Mixing identification processing method and device, equipment and medium

Abstract

The invention relates to the technical field of voice recognition, in particular to a mixed recognition processing method and device, equipment and a medium, and the method comprises the steps: recognizing command word data based on a command word recognition model, selecting command word data with an error rate lower than a preset range, generating a corresponding phoneme embedding representation, and forming a phoneme embedding dictionary; determining a mixed recognition command word corresponding to the current command word, and generating a mixed recognition list; identifying an audio embedding sequence corresponding to the current command word; obtaining a phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on a phoneme embedding dictionary; and calculating the similarity between an audio embedding sequence and each phoneme embedding sequence to obtain an identification result. The method and the device can improve recognition accuracy and reduce mixed recognition of command words.

Description

Technical Field

[0001] The present application relates to the field of speech recognition, and in particular to a mixed recognition processing method, device, equipment and medium. Background Art

[0002] In the field of smart home, such as smart voice speakers, headphones, lights, fans and other devices, command word recognition is one of the key functions. Existing command word recognition mainly relies on the CTC (Connectionist Temporal Classification) algorithm, which does not require sequence alignment and has the advantages of fast speed and memory saving, so it is widely used in embedded devices. However, a significant problem with the CTC algorithm is that when processing command words with similar pronunciations, it is easy to have mixed recognition, such as "play the previous song" and "play the next song", "eleven degrees" and "seventeen degrees", and "turn on blood pressure" and "turn on blood oxygen", etc. In this case, the system may not be able to accurately execute the user's instructions.

[0003] Therefore, the technical problem of easily confusing command words in the existing speech recognition process needs to be solved urgently. Summary of the invention

[0004] The main purpose of the present application is to provide a mixed recognition processing method, device, equipment and medium, aiming to solve the technical problem that command words are easily mixed in the existing speech recognition process.

[0005] In order to achieve the above-mentioned invention object, the present application proposes a mixed recognition processing method, the method comprising:

[0006] Recognize the command word data based on the command word recognition model, select the command word data with an error rate lower than a preset range, generate corresponding phoneme embedding representations, and form a phoneme embedding dictionary;

[0007] Determine the mixed recognition command word corresponding to the current command word and generate a mixed recognition list;

[0008] Identify the audio embedding sequence corresponding to the current command word;

[0009] Obtaining a phoneme embedding sequence for each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary;

[0010] The similarity between the audio embedding sequence and each of the phoneme embedding sequences is calculated to obtain a recognition result.

[0011] Furthermore, before the step of identifying the command word data based on the command word recognition model, the step includes:

[0012] Feature extraction based on general corpus;

[0013] Based on the extracted features, the model is trained through a preset model architecture to obtain a preliminary model;

[0014] Selecting adjustment data based on a preset command word data set;

[0015] Based on the preset fine-tuning parameters, the adjustment data is input to train the preliminary model to obtain a command word recognition model.

[0016] Furthermore, the step of identifying command word data based on the command word recognition model, selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary includes:

[0017] Using the command word recognition model to recognize the selected command word data, and generating a recognition result;

[0018] Calculate the error rate of the recognition result and compare it with the preset range;

[0019] Selecting command word data with an error rate lower than a preset range as valid command word data;

[0020] Generate corresponding phoneme embedding representation based on valid command word data;

[0021] All generated phone embedding representations are combined into a phone embedding dictionary.

[0022] Furthermore, the step of determining the mixed recognition command word corresponding to the current command word and generating a mixed recognition list includes:

[0023] Identify the phoneme sequence corresponding to the current command word;

[0024] Calculating the edit distance between the current command word phoneme sequence and the phoneme sequence corresponding to the command word contained in the command word data;

[0025] Identifying the command word whose phoneme sequence edit distance with the current command word is less than a preset threshold;

[0026] The corresponding command words are determined to be mixed recognition command words, and are sorted into a mixed recognition list; wherein the mixed recognition command words include the current command word.

[0027] Furthermore, the step of identifying the audio embedding sequence corresponding to the current command word further includes:

[0028] Traverse each command word contained in the command word data;

[0029] evaluating the similarity between the command word and the current command word;

[0030] According to the result of similarity evaluation, a mixed recognition command word is determined, and the mixed recognition command word is

[0031] Add to the mixed recognition candidate list.

[0032] Furthermore, the step of acquiring the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary includes:

[0033] According to the mixed recognition candidate list, extract the phoneme sequence corresponding to each mixed recognition command word from the phoneme embedding dictionary;

[0034] For each phoneme sequence corresponding to the mixed recognition command word, the corresponding phoneme embedding is identified from the phoneme embedding dictionary to generate a phoneme embedding sequence.

[0035] Furthermore, the step of calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain the recognition result includes:

[0036] Normalize each embedding vector in the audio embedding sequence and the phoneme embedding sequence;

[0037] Based on the normalized audio embedding sequence and the phoneme embedding sequence, a dynamic programming algorithm is used to calculate a similarity score between the audio embedding sequence and each of the phoneme embedding sequences;

[0038] The similarity scores are compared, and the phoneme embedding sequence with the highest similarity to the audio embedding sequence is screened out to obtain a recognition result.

[0039] The second aspect of the present application provides a mixed identification processing device, comprising:

[0040] A generation module, used to recognize command word data based on a command word recognition model, select command word data with an error rate lower than a preset range, generate corresponding phoneme embedding representations, and form a phoneme embedding dictionary;

[0041] A determination module, used to determine a mixed recognition command word corresponding to the current command word and generate a mixed recognition list;

[0042] A recognition module, used to recognize the audio embedding sequence corresponding to the current command word;

[0043] An acquisition module, used for acquiring a phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on a phoneme embedding dictionary;

[0044] The calculation module is used to calculate the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain a recognition result.

[0045] The third aspect of the present application also includes a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0046] The fourth aspect of the present application also includes a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0047] Beneficial Effects

[0048] The command word data is recognized based on the command word recognition model, and the command word data with an error rate lower than a preset range is selected to generate a phoneme embedding dictionary. The present invention does not require retraining of the model. Phoneme embedding is generated based on audio embedding, which ensures that the audio and text are in the same feature space and avoids the differences in different feature spaces. Based on the phoneme embedding dictionary, the phoneme embedding sequence of each command word in the mixed recognition list is obtained to ensure the consistency of audio and text embedding. The dynamic programming algorithm is used to calculate the similarity between the audio embedding sequence and each phoneme embedding sequence, and the phoneme embedding sequence with the highest similarity is screened out to obtain the final recognition result. By setting a similarity threshold, the recognition result is further confirmed to reduce the probability of misrecognition. The accuracy and reliability of command word recognition are significantly improved, the calculation is efficient, the resource usage is small, and it is easy to deploy on embedded devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flow chart of a mixed identification processing method according to an embodiment of the present application;

[0050] Figure 2 This is a schematic block diagram of the structure of a mixed identification processing device according to an embodiment of the present application;

[0051] Figure 3 A schematic block diagram of the structure of a computer device according to an embodiment of the present application.

[0052] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "a", "an", "above", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any module and all combinations of one or more associated listed items.

[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as herein.

[0056] Reference Figure 1 The embodiment of the present invention provides a mixed identification processing method, including steps S1-S5, specifically:

[0057] S1. Recognize command word data based on the command word recognition model, select command word data with an error rate lower than a preset range, generate corresponding phoneme embedding representations, and form a phoneme embedding dictionary;

[0058] S2, determining a mixed recognition command word corresponding to the current command word, and generating a mixed recognition list;

[0059] S3, identifying the audio embedding sequence corresponding to the current command word;

[0060] S4, obtaining a phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary;

[0061] S5. Calculate the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain a recognition result.

[0062] As described in step S1 above, in this embodiment, first, a fully trained command word recognition model is required. This model is usually pre-trained on a large amount of general corpus, and then fine-tuned on a specific command word data set to adapt to the specific command word recognition task. The command word recognition model is used to recognize the command word data, with the purpose of screening out high-quality command word data, which will be used to generate phoneme embedding representations. The recognition process involves converting the command word audio into text form, while recording the recognition results of each command word. In order to ensure the quality of the generated phoneme embedding representation, only those command word data with a recognition error rate lower than a preset range are selected. The preset range is selected based on the needs of the actual application scenario, and is usually a small value, such as 5% or lower. The selected command word data are considered to be of high quality because they show high accuracy in the recognition process. Next, the corresponding phoneme embedding representation is generated based on these high-quality command word data. The process of generating the phoneme embedding representation includes aligning the command word audio, finding the corresponding relationship between the audio and the phoneme, and then calculating the embedding vector of each phoneme. These embedding vectors will eventually be combined into a phoneme embedding dictionary, which will be used in the subsequent mixed recognition processing steps.

[0063] As described in step S2 above, when determining the mixed recognition command word corresponding to the current command word, it is first necessary to identify the phoneme sequence of the current command word. This step is completed based on the command word recognition model, and the model converts the audio of the current command word into its corresponding phoneme sequence. Next, the edit distance between the phoneme sequence of the current command word and the phoneme sequence of other command words is calculated. Edit distance is a method to measure the similarity of two sequences, which defines the minimum number of single-character editing operations (insertion, deletion or replacement) required to convert one sequence into another sequence. By calculating the edit distance, the similarity between different command words can be quantified. If the edit distance between the phoneme sequence of a command word and the phoneme sequence of the current command word is less than a preset threshold, it is considered that the two command words are easily confused, and the command word will be added to the mixed recognition list. The preset threshold can be adjusted according to the actual situation, and a value that can balance the sensitivity and accuracy of mixed recognition detection is usually selected. It should be noted that the mixed recognition list not only contains other command words that may be confused with the current command word, but also the current command word itself. The process of generating a mixed recognition list helps to more accurately identify command words in subsequent steps and reduce misrecognition caused by similar pronunciation. There is no need to set a threshold for distinguishing mixed recognition. It is obtained based on the nearest distance, and the calculation is simple and efficient.

[0064] As described in step S3 above, the command word recognition model is first used to process the audio of the current command word and extract the feature representation of the audio. These feature representations are usually a series of high-dimensional vectors, each of which represents a small segment of the audio. In order to make these vectors better reflect the phoneme characteristics of the audio, they need to be normalized to ensure that the modulus of each vector is 1. The normalization process can be completed by a simple mathematical operation, that is, each vector is divided by its own modulus. The normalized vector can be directly used to calculate the cosine similarity, which is the basis for comparing the similarity between the audio embedding sequence and the phoneme embedding sequence in the subsequent steps. Next, according to the output of the command word recognition model, the left and right boundaries of the current command word are determined. The determination of the left and right boundaries is completed by analyzing the CTC matrix output by the model, finding the maximum score path, and using the backtracking algorithm to find the alignment path of the audio and the phoneme. The alignment path indicates which parts of the audio correspond to which phonemes of the command word, which is crucial for accurately extracting the audio embedding sequence. The extracted audio embedding sequence will be used for similarity comparison with the phoneme embedding sequence in the subsequent steps to determine the most likely command word. At the same time, the recognized command words can be further confirmed to reduce the probability of misrecognition.

[0065] As described in step S4 above, first, each command word in the mixed recognition list needs to be traversed, and for each command word, its corresponding phoneme sequence is searched from the phoneme embedding dictionary. The phoneme embedding dictionary is generated in step S1, and contains the phonemes of all command words and their corresponding embedding vectors. The phoneme sequence of each command word consists of a series of phonemes, and these phonemes have corresponding embedding vectors in the dictionary. By searching these embedding vectors, a phoneme embedding sequence can be generated for each command word. The generated phoneme embedding sequence is a two-dimensional array, in which each element is an embedding vector, representing a phoneme in the command word. These phoneme embedding sequences will be used for similarity comparison with the audio embedding sequence in subsequent steps, so as to determine the most likely command word. In this way, it can be ensured that when the similarity is compared, the audio embedding sequence and the phoneme embedding sequence are in the same feature space, which improves the accuracy and efficiency of the comparison. Since the phoneme sequence is generally short, the present invention is efficient in calculation, occupies very few computing resources, and is easy to deploy. The short characteristics of the phoneme sequence greatly reduce the computational complexity, and are suitable for running on embedded devices with limited resources.

[0066] As described in step S5 above, after the generation of the audio embedding sequence and the phoneme embedding sequence is completed, the next step is to calculate the similarity between the two to determine the most likely command word. This process first requires normalizing each embedding vector in the audio embedding sequence and the phoneme embedding sequence to ensure that the modulus length of each vector is 1. Normalization allows the vector dot multiplication to be directly equivalent to the cosine similarity calculation, simplifying the similarity calculation process. The normalized audio embedding sequence and phoneme embedding sequence are denoted as Sa and St, respectively, where the shape of Sa is T×d, the shape of St is L×d, T represents the length of the audio embedding sequence, L represents the length of the command word phoneme sequence, and d represents the dimension of the embedding vector. Next, the dynamic programming algorithm is used to calculate the similarity between Sa and St. Calculate from back to front, map the last k1 of the audio sequence to the last k2 of the text sequence, and calculate the values ​​that fill the dynamic programming table (DP table) in turn. The shape of the DP table is L×T, and each cell dp[i][j] stores the maximum similarity score between the first j elements of the audio embedding sequence and the first i elements of the phoneme embedding sequence. Finally, the similarity score is dp[L-1][0] / m, where m is a normalization factor used to adjust the range of the similarity score. By comparing the similarity scores of all mixed recognition command words, the phoneme embedding sequence with the highest similarity to the audio embedding sequence can be screened out, and the corresponding command word is the final recognition result. In addition, a threshold can be set. If the maximum similarity score is lower than the threshold, the recognition result is considered unreliable and the recognition can be discarded to avoid misrecognition. This method can not only improve the accuracy of command word recognition, but also effectively reduce misrecognition caused by similar pronunciation and improve user experience. The embedding obtained based on the ability of the command word model itself effectively combines the recognition ability of the model. The embedding of phonemes is generated based on the embedding of audio, so that the two modalities of audio and text are embedded in the same feature space, avoiding the differences between different feature spaces, ensuring that the generated embedding vector can accurately reflect the characteristics of the command word, and further improving the accuracy and robustness of recognition.

[0067] Steps S1-S5 generate a phoneme embedding dictionary by selecting command word data with low error rates, ensuring high quality of phoneme embedding and effectively identifying easily confused command words. The audio embedding sequence of the current command word is extracted, and then the phoneme embedding sequence of each command word in the mixed recognition list is obtained based on the phoneme embedding dictionary, ensuring that the audio and text are embedded in the same feature space. By calculating the similarity between the audio embedding sequence and the phoneme embedding sequence, the most likely command word is finally determined. The whole process not only improves the accuracy of recognition, but also simplifies the calculation and reduces the probability of misrecognition through the dynamic programming algorithm and the nearest distance acquisition method. It is efficient in calculation, occupies less resources, and is easy to deploy on embedded devices.

[0068] In one embodiment, before the step of identifying the command word data based on the command word recognition model, the step includes:

[0069] S10, feature extraction based on general corpus;

[0070] S11, training is performed based on the extracted features through a preset model architecture to obtain a preliminary model;

[0071] S12, selecting adjustment data based on a preset command word data set;

[0072] S13. Based on the preset fine-tuning parameters, input adjustment data to train the preliminary model to obtain a command word recognition model.

[0073] In this embodiment, first, a large amount of general corpus is used for feature extraction. These general corpus can come from a variety of sources, such as public speech data sets or recording data in a specific field. The process of feature extraction usually involves converting the audio signal into a set of meaningful feature vectors. Common features include Mel-frequency cepstral coefficients (MFCC), filter bank energy (FBANK), etc. These feature vectors can capture important information of the audio signal and provide a basis for subsequent model training. After the feature extraction is completed, the extracted features are trained using a preset model architecture (such as convolutional neural network CNN, recurrent neural network RNN ​​or Transformer, etc.). During the training process, the model will learn how to map the input feature vector to the corresponding label (i.e., the text representation of the command word). By conducting large-scale training on general corpus, the model can learn the basic pattern and structure of the speech signal and form a preliminary command word recognition model. In order to make the preliminary model better adapt to the specific command word recognition task, it is necessary to fine-tune it on a specific command word dataset. First, select a part of the data or all of the data from the preset command word dataset as adjustment data. These data usually contain audio samples of various command words and their corresponding labels, covering various situations that may occur in actual application scenarios. The selection of adjustment data should ensure diversity and representativeness to improve the generalization ability of the model. Finally, the adjustment data is input into the preliminary model for further training using the preset fine-tuning parameters. Fine-tuning parameters include learning rate, batch size, number of training rounds, etc. The selection of these parameters needs to be optimized according to the characteristics of the specific task and dataset. Through fine-tuning, the model can further improve its performance on a specific command word dataset, and finally obtain a model specifically for command word recognition. This model will be used in subsequent steps to recognize command word data, generate high-quality phoneme embedding representations, and provide support for mixed recognition processing.

[0074] In one embodiment, the step of identifying command word data based on the command word recognition model, selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary includes:

[0075] S20, using the command word recognition model to recognize the selected command word data and generate a recognition result;

[0076] S21, calculating the error rate of the recognition result and comparing it with a preset range;

[0077] S22, selecting command word data with an error rate lower than a preset range and taking it as valid command word data;

[0078] S23, generating a corresponding phoneme embedding representation based on the valid command word data;

[0079] S24. Combine all generated phoneme embedding representations into a phoneme embedding dictionary.

[0080] In this embodiment, first, the selected command word data are recognized using the trained command word recognition model. These command word data are usually preprocessed audio files, each file corresponding to a command word. The model converts these audio files into corresponding text representations and generates recognition results. The recognition results include the predicted label of each command word and its confidence. In this process, the model uses the CTC (Connectionist Temporal Classification) algorithm for decoding to generate the final recognition result. Next, the error rate of the recognition result of each command word is calculated. The error rate can be calculated by comparing the predicted label of the model with the true label. Common indicators include word error rate (WER) or word error rate (CER). The calculated error rate is compared with a preset error rate range. The preset error rate range is usually a small value, such as 5% or lower, and the specific value can be adjusted according to the needs of the actual application scenario. According to the comparison results of the error rate, those command word data with an error rate lower than the preset range are selected. These data are considered to be of high quality because they show high accuracy in the recognition process. These high-quality command word data will be retained as the basis for the subsequent generation of phoneme embedding representations. For the selected valid command word data, the corresponding phoneme embedding representation is generated. The process of generating the phoneme embedding representation includes aligning the command word audio, finding the corresponding relationship between the audio and the phoneme, and then calculating the embedding vector of each phoneme. The specific operation is as follows: Use the forward algorithm of the CTC algorithm to calculate the path score and find the maximum score path. This step determines the alignment path of the audio and the phoneme through the backtracking algorithm. After the alignment path is determined, the audio segment corresponding to each phoneme is extracted, and the embedding vectors of these segments are calculated. These embedding vectors reflect the characteristics of the phoneme and are used to generate the phoneme embedding representation. Finally, all the generated phoneme embedding representations are combined into a phoneme embedding dictionary. The phoneme embedding dictionary is a set of phonemes and their corresponding embedding vectors containing all valid command words. This dictionary will be used in subsequent steps to help identify the mixed recognition command words of the current command word and perform similarity calculations. Through the above steps, the present invention can not only generate high-quality phoneme embedding representations, but also ensure that these representations have high accuracy and reliability in subsequent mixed recognition processing. In particular, the present invention does not require retraining of the model, and the embedding of the phoneme is generated based on the embedding of the audio, so that the two modalities of audio and text are embedded in the same feature space, avoiding the differences in different feature spaces. This method not only improves the accuracy of recognition, but also simplifies the calculation process, reduces the probability of misrecognition, and is efficient in calculation, occupies less resources, and is easy to deploy on embedded devices. By using the maximum score path of CTC for alignment, the accuracy and consistency of phoneme embedding are ensured.

[0081] In one embodiment, the step of determining the mixed recognition command word corresponding to the current command word and generating the mixed recognition list includes:

[0082] S30, identifying the phoneme sequence corresponding to the current command word;

[0083] S31, calculating the edit distance between the current command word phoneme sequence and the phoneme sequence corresponding to the command word contained in the command word data;

[0084] S32, identifying the command word whose phoneme sequence edit distance with the current command word is less than a preset threshold;

[0085] S33: Determine the corresponding command word as a mixed recognition command word, and sort it into a mixed recognition list; wherein the mixed recognition command word includes the current command word.

[0086] In this embodiment, first, the command word recognition model is used to identify the phoneme sequence of the current command word. This step is performed by inputting the audio of the current command word into the command word recognition model, and the model outputs the phoneme sequence of the command word. The phoneme sequence is composed of a series of phonemes, each phoneme representing a pronunciation unit. For example, the phoneme sequence of the command word "play the previous song" may be "bo1fang4shang4yi1shou3". Next, the edit distance between the phoneme sequence of the current command word and the phoneme sequences of other command words is calculated. The edit distance (LevenshteinDistance) is a method for measuring the similarity between two sequences, which defines the minimum number of single-character editing operations (insertion, deletion or replacement) required to convert one sequence into another sequence. By calculating the edit distance, the similarity between different command words can be quantified. For example, by calculating the edit distance between the phoneme sequences of "play the previous song" and "play the next song", it can be found that the phoneme sequences of the two command words are very similar and the edit distance is very small. According to the calculated edit distance, the command word whose edit distance with the phoneme sequence of the current command word is less than a preset threshold is identified. The selection of the preset threshold is based on the needs of the actual application scenario, and is usually a smaller value, such as 2 or 3. If the edit distance between the phoneme sequence of a command word and the phoneme sequence of the current command word is less than the preset threshold, it is considered that the two command words are easily confused, and the command word will be marked as a mixed recognition command word. Finally, the identified mixed recognition command words are sorted into a mixed recognition list. The mixed recognition list should not only include other command words that are easily confused with the current command word, but also the current command word itself. For example, if the current command word is "play the previous song", the mixed recognition list may include "play the previous song", "play the next song", "play this song", etc. The generated mixed recognition list will be used in subsequent steps to help further confirm the recognition results and reduce the probability of misrecognition.

[0087] Through the above steps, the present invention can effectively identify other command words that are easily confused with the current command word and generate a mixed recognition list. There is no need to set a threshold for distinguishing mixed recognition, and it is acquired based on the nearest distance, so the calculation is simple and efficient. This method not only simplifies the calculation process, but also improves the accuracy and efficiency of mixed recognition, ensuring the reliability of the final recognition result. At the same time, since the phoneme sequence is generally short, the present invention is efficient in calculation, occupies very few computing resources, and is easy to deploy. The short nature of the phoneme sequence greatly reduces the computational complexity and is suitable for running on embedded devices with limited resources.

[0088] In one embodiment, the step of identifying the audio embedding sequence corresponding to the current command word further includes:

[0089] S40, traversing each command word included in the command word data;

[0090] S41, evaluating the similarity between the command word and the current command word;

[0091] S42: Determine a mixed recognition command word according to the result of the similarity evaluation, and add the mixed recognition command word to a mixed recognition candidate list.

[0092] In this embodiment, first, each command word contained in the command word data set is traversed. The purpose of this step is to evaluate all possible command words to determine which command words are similar to the current command word and may cause mixed recognition. The command word data set usually contains audio samples of various command words and their corresponding labels, covering various situations that may occur in actual application scenarios. For each command word, its similarity with the current command word is evaluated. Similarity evaluation can be performed by a variety of methods, and common methods include calculating the cosine similarity or Euclidean distance between audio embedding sequences. Based on the results of the similarity evaluation, command words with higher similarity to the current command word are determined, and these command words are added to the mixed recognition candidate list. By traversing all command words and directly evaluating the similarity, the method is simple and direct and easy to implement.

[0093] In one embodiment, the step of acquiring the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary includes:

[0094] S50, extracting a phoneme sequence corresponding to each mixed recognition command word from the phoneme embedding dictionary according to the mixed recognition candidate list;

[0095] S51. For each phoneme sequence corresponding to the mixed recognition command word, identify the corresponding phoneme embedding from the phoneme embedding dictionary to generate a phoneme embedding sequence.

[0096] In this embodiment, first, according to the generated mixed recognition candidate list, the phoneme sequence corresponding to each mixed recognition command word is extracted from the phoneme embedding dictionary. The mixed recognition candidate list contains command words with high similarity to the current command word, and these command words may cause mixed recognition. The phoneme embedding dictionary is a set of phonemes and their corresponding embedding vectors of all command words. The specific steps are as follows: traverse each command word in the mixed recognition candidate list. Find the phoneme sequence corresponding to each command word from the phoneme embedding dictionary. The phoneme sequence is composed of a series of phonemes, and each phoneme represents a pronunciation unit. For example, the phoneme sequence of the command word "play the previous song" may be "bo1fang4shang4yi1shou3". For the phoneme sequence corresponding to each mixed recognition command word, the corresponding phoneme embedding is identified from the phoneme embedding dictionary to generate a phoneme embedding sequence. For example: for each phoneme, the corresponding embedding vector is found from the phoneme embedding dictionary. The phoneme embedding dictionary stores the embedding vectors of each phoneme, and these vectors can capture the characteristics of the phoneme. The found embedding vectors are combined in sequence into a phoneme embedding sequence. The phoneme embedding sequence is a two-dimensional array, where each element is an embedding vector representing a phoneme in the command word. For example, if the phoneme sequence of the command word "play previous song" is "bo1fang4shang4yi1shou3", the generated phoneme embedding sequence is [embed_bo1,embed_fang4,embed_shang4,embed_yi1,embed_shou3]. By directly looking up the phoneme embedding from the phoneme embedding dictionary, the accuracy and consistency of the phoneme embedding are ensured, and the complexity of recalculating the embedding vector is avoided. The phoneme embedding dictionary has been pre-generated, and the process of looking up the phoneme embedding is simple and efficient, with low consumption of computing resources. This method makes full use of the existing knowledge of the command word recognition model to ensure that the generated embedding vector can accurately reflect the characteristics of the command word, further improving the accuracy and robustness of recognition.

[0097] In one embodiment, the step of calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain a recognition result includes:

[0098] S60, normalizing each embedding vector in the audio embedding sequence and the phoneme embedding sequence;

[0099] S61, based on the normalized audio embedding sequence and the phoneme embedding sequence, using a dynamic programming algorithm to calculate the similarity score between the audio embedding sequence and each of the phoneme embedding sequences;

[0100] S62: Compare the similarity scores, select the phoneme embedding sequence with the highest similarity to the audio embedding sequence, and obtain a recognition result.

[0101] In one embodiment, first, each embedding vector in the audio embedding sequence and the phoneme embedding sequence is normalized. The purpose of the normalization process is to ensure that the modulus length of each vector is 1, so that the cosine similarity can be calculated directly by vector dot multiplication. For example: extract each embedding vector from the audio embedding sequence and the phoneme embedding sequence. Calculate the modulus length of each embedding vector, that is, the Euclidean norm of the vector. Divide each embedding vector by its modulus length to obtain a normalized embedding vector. Next, based on the normalized audio embedding sequence and phoneme embedding sequence, a dynamic programming algorithm is used to calculate the similarity score between the audio embedding sequence and each phoneme embedding sequence. The core idea of ​​the dynamic programming algorithm is to calculate from back to front, map the last k1 of the audio sequence to the last k2 of the text sequence, and calculate the values ​​of filling the dynamic programming table (DP table) in turn. For example: create a dynamic programming table dp with a shape of L×T, where L is the length of the phoneme embedding sequence and T is the length of the audio embedding sequence. Using the normalized audio embedding sequence and phoneme embedding sequence, the cosine similarity of each pair of embedding vectors is calculated, and the dynamic programming table is filled. Each cell dp[i][j] of the dynamic programming table stores the maximum similarity score between the first j elements of the audio embedding sequence and the first i elements of the phoneme embedding sequence. The similarity score between the audio embedding sequence and each phoneme embedding sequence is calculated by the dynamic programming algorithm. The final similarity score can be obtained from the last cell dp[L-1][T-1] of the dynamic programming table. Finally, the similarity scores of all phoneme embedding sequences are compared, and the phoneme embedding sequence with the highest similarity to the audio embedding sequence is screened out to obtain the final recognition result. The similarity scores of all phoneme embedding sequences are compared to find the phoneme embedding sequence with the highest score. The command word corresponding to the phoneme embedding sequence with the highest similarity is taken as the final recognition result. If the highest similarity score is lower than the preset threshold, the recognition result can be considered unreliable, and the recognition can be discarded to avoid misrecognition. The similarity score is calculated by the dynamic programming algorithm, which ensures the accuracy and reliability of the similarity calculation and improves the accuracy of recognition. By comparing the similarity scores, the most likely command word is screened out, and the recognition result is further confirmed, which reduces the probability of misrecognition.

[0102] Reference Figure 2 , is a structural block diagram of a mixed identification processing device in an embodiment of the present application, the device includes:

[0103] A generating module 100 is used to identify command word data based on a command word recognition model, and select command word data with an error rate lower than a preset range, generate corresponding phoneme embedding representations, and form a phoneme embedding dictionary;

[0104] A determination module 200, for determining a mixed recognition command word corresponding to the current command word, and generating a mixed recognition list;

[0105] Recognition module 300, used to recognize the audio embedding sequence corresponding to the current command word;

[0106] An acquisition module 400 is used to acquire a phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on a phoneme embedding dictionary;

[0107] The calculation module 500 is used to calculate the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain a recognition result.

[0108] In one embodiment, the above device further includes a training module, wherein the training module includes:

[0109] An extraction unit, used for feature extraction based on general corpus;

[0110] A first training unit is used to perform training based on the extracted features through a preset model architecture to obtain a preliminary model;

[0111] A selection unit, used for selecting adjustment data based on a preset command word data set;

[0112] The fine-tuning unit is used to train the preliminary model based on preset fine-tuning parameters and input adjustment data to obtain a command word recognition model.

[0113] In one embodiment, the generation module 100 includes:

[0114] A first recognition unit, used to recognize the selected command word data using a command word recognition model to generate a recognition result;

[0115] A comparison unit, used to calculate the error rate of the recognition result and compare it with a preset range;

[0116] A valid selection unit, used for selecting command word data with an error rate lower than a preset range as valid command word data;

[0117] A representation unit, configured to generate a corresponding phoneme embedding representation based on valid command word data;

[0118] A combination unit is used to combine all generated phoneme embedding representations into a phoneme embedding dictionary.

[0119] In one embodiment, the determination module 200 includes:

[0120] A second recognition unit, used to recognize a phoneme sequence corresponding to the current command word;

[0121] a distance calculation unit, used to calculate the edit distance between the current command word phoneme sequence and the phoneme sequence corresponding to the command word contained in the command word data;

[0122] A less than recognition unit, used to identify the command word whose phoneme sequence edit distance with the current command word is less than a preset threshold;

[0123] A sorting unit is used to determine the corresponding command word as a mixed recognition command word and sort it into a mixed recognition list; wherein the mixed recognition command word includes the current command word.

[0124] In one embodiment, the determination module 200 further includes:

[0125] A traversal unit, used for traversing each command word contained in the command word data;

[0126] An evaluation unit, used for evaluating the similarity between the command word and the current command word;

[0127] A unit is added to determine a mixed recognition command word according to the result of the similarity evaluation, and to convert the mixed recognition command word into

[0128] Add to the mixed recognition candidate list.

[0129] In one embodiment, the acquisition module 400 includes:

[0130] An extraction unit, used for extracting a phoneme sequence corresponding to each mixed recognition command word from a phoneme embedding dictionary according to the mixed recognition candidate list;

[0131] The embedding unit is used to identify the corresponding phoneme embedding from the phoneme embedding dictionary for the phoneme sequence corresponding to each mixed recognition command word, and generate a phoneme embedding sequence.

[0132] In one embodiment, the calculation module 500 includes:

[0133] A normalization unit, used for normalizing each embedding vector in the audio embedding sequence and the phoneme embedding sequence;

[0134] A dynamic programming unit, configured to calculate a similarity score between the audio embedding sequence and each of the phoneme embedding sequences using a dynamic programming algorithm based on the normalized audio embedding sequence and the phoneme embedding sequence;

[0135] The comparison unit is used to compare the similarity scores, select the phoneme embedding sequence with the highest similarity to the audio embedding sequence, and obtain the recognition result.

[0136] Reference Figure 3 In the embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3As shown. The computer device includes a processor, an internal memory, a storage medium (non-volatile storage medium) and a network interface connected via a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes the above-mentioned storage medium (non-volatile storage medium) and the internal memory. The storage medium (non-volatile storage medium) stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the storage medium (non-volatile storage medium). The database of the computer device is used to store usage data in a mixed identification processing method process, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. Furthermore, the above-mentioned computer device can also be provided with an input device and a display screen, etc. When the above computer program is executed by the processor, a mixed recognition processing method is implemented, including the following steps: recognizing command word data based on a command word recognition model, and selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary; determining the mixed recognition command word corresponding to the current command word, and generating a mixed recognition list; recognizing the audio embedding sequence corresponding to the current command word; obtaining the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary; calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences, and obtaining the recognition result. Those skilled in the art can understand that Figure 3 The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.

[0137] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, a mixed recognition processing method is implemented, including the following steps: recognizing command word data based on a command word recognition model, and selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary; determining a mixed recognition command word corresponding to the current command word, and generating a mixed recognition list; recognizing an audio embedding sequence corresponding to the current command word; obtaining a phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary; calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences, and obtaining a recognition result. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0138] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0139] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

[0140] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A mixed recognition processing method, characterized in that: The method comprises: Recognize the command word data based on the command word recognition model, select the command word data with an error rate lower than a preset range, generate corresponding phoneme embedding representations, and form a phoneme embedding dictionary; Determine the mixed recognition command word corresponding to the current command word and generate a mixed recognition list; Identify the audio embedding sequence corresponding to the current command word; Obtaining a phoneme embedding sequence for each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary; The similarity between the audio embedding sequence and each of the phoneme embedding sequences is calculated to obtain a recognition result.

2. The mixed identification processing method according to claim 1, characterized in that: Before the step of identifying the command word data based on the command word recognition model, the method includes: Feature extraction based on general corpus; Based on the extracted features, the preset model architecture is used for training to obtain a preliminary model; Selecting adjustment data based on a preset command word data set; Based on the preset fine-tuning parameters, the adjustment data is input to train the preliminary model to obtain a command word recognition model.

3. The mixed identification processing method according to claim 1, characterized in that: The step of identifying command word data based on the command word recognition model, selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary includes: Using the command word recognition model to recognize the selected command word data, and generating a recognition result; Calculate the error rate of the recognition result and compare it with the preset range; Selecting command word data with an error rate lower than a preset range as valid command word data; Generate corresponding phoneme embedding representation based on valid command word data; All generated phone embedding representations are combined into a phone embedding dictionary.

4. The mixed identification processing method according to claim 1, characterized in that: The step of determining the mixed recognition command word corresponding to the current command word and generating a mixed recognition list includes: Identify the phoneme sequence corresponding to the current command word; Calculating the edit distance between the current command word phoneme sequence and the phoneme sequence corresponding to the command word contained in the command word data; Identifying the command word whose phoneme sequence edit distance with the current command word is less than a preset threshold; The corresponding command words are determined to be mixed recognition command words, and are sorted into a mixed recognition list; wherein the mixed recognition command words include the current command word.

5. The mixed identification processing method according to claim 1, characterized in that: The step of identifying the audio embedding sequence corresponding to the current command word also includes: Traverse each command word contained in the command word data; evaluating the similarity between the command word and the current command word; According to the result of the similarity evaluation, a mixed recognition command word is determined, and the mixed recognition command word is added to the mixed recognition candidate list.

6. The mixed identification processing method according to claim 1, characterized in that: The step of acquiring the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary comprises: According to the mixed recognition candidate list, extract the phoneme sequence corresponding to each mixed recognition command word from the phoneme embedding dictionary; For each phoneme sequence corresponding to the mixed recognition command word, the corresponding phoneme embedding is identified from the phoneme embedding dictionary to generate a phoneme embedding sequence.

7. The mixed identification processing method according to claim 1, characterized in that: The step of calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain a recognition result comprises: Normalize each embedding vector in the audio embedding sequence and the phoneme embedding sequence; Based on the normalized audio embedding sequence and the phoneme embedding sequence, a dynamic programming algorithm is used to calculate a similarity score between the audio embedding sequence and each of the phoneme embedding sequences; The similarity scores are compared, and the phoneme embedding sequence with the highest similarity to the audio embedding sequence is screened out to obtain a recognition result.

8. A mixed recognition processing device, characterized in that: include: A generation module, used to recognize command word data based on a command word recognition model, select command word data with an error rate lower than a preset range, generate corresponding phoneme embedding representations, and form a phoneme embedding dictionary; A determination module, used to determine a mixed recognition command word corresponding to the current command word and generate a mixed recognition list; A recognition module, used to recognize the audio embedding sequence corresponding to the current command word; An acquisition module, used for acquiring a phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on a phoneme embedding dictionary; The calculation module is used to calculate the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain a recognition result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images