Hybrid recognition processing method, apparatus, device, and medium

Abstract

This application relates to the field of speech recognition technology, and in particular to a mixed recognition processing method, apparatus, device, and medium. The method includes: recognizing command word data based on a command word recognition model, selecting command word data with an error rate below a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary; determining mixed recognition command words corresponding to the current command word, and generating a mixed recognition list; recognizing the audio embedding sequence corresponding to the current command word; obtaining the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary; calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences, and obtaining the recognition result. This application can improve recognition accuracy and reduce mixed recognition of command words.

Description

Technical Field

[0001] This application relates to the field of speech recognition, and in particular to a mixed recognition processing method, apparatus, device and medium. Background Technology

[0002] In the smart home field, command word recognition is a key function in devices such as smart speakers, headphones, lights, and fans. Existing command word recognition primarily relies on the CTC (Connectionist Temporal Classification) algorithm. This algorithm does not require sequence alignment and has advantages such as speed and low memory consumption, thus it is widely used in embedded devices. However, a significant problem with the CTC algorithm is that it is prone to misrecognition when processing command words with similar pronunciations, such as "play previous song" and "play next song," "eleven degrees" and "seventeen degrees," and "turn on blood pressure" and "turn on blood oxygen." In such cases, the system may fail to accurately execute the user's instructions.

[0003] Therefore, the technical problem of misidentification of command words in the existing speech recognition process urgently needs to be solved. Summary of the Invention

[0004] The main objective of this application is to provide a mixed recognition processing method, apparatus, device, and medium, which aims to solve the technical problem of mixed recognition of command words in the existing speech recognition process.

[0005] To achieve the above-mentioned objectives, this application proposes a mixed identification processing method, the method comprising:

[0006] The command word recognition model is used to identify command word data, and command word data with an error rate lower than a preset range are selected to generate corresponding phoneme embedding representations, thus forming a phoneme embedding dictionary.

[0007] Identify the mixed recognition command words corresponding to the current command word and generate a mixed recognition list;

[0008] Identify the audio embedding sequence corresponding to the current command word;

[0009] The phoneme embedding sequence of each mixed recognition command word in the mixed recognition list is obtained based on the phoneme embedding dictionary;

[0010] The similarity between the audio embedding sequence and each of the phoneme embedding sequences is calculated to obtain the recognition result.

[0011] Furthermore, prior to the step of recognizing command word data based on the command word recognition model, the following steps are included:

[0012] Feature extraction based on general corpus;

[0013] Based on the extracted features, a preliminary model is obtained by training the model using a pre-defined model architecture.

[0014] Data is selected and adjusted based on a pre-defined command word dataset;

[0015] Based on preset fine-tuning parameters, the preliminary model is trained by inputting adjustment data to obtain a command word recognition model.

[0016] Further, the step of identifying command word data based on the command word recognition model, selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary includes:

[0017] The command word recognition model is used to identify the selected command word data and generate recognition results;

[0018] Calculate the error rate of the recognition results and compare it with the preset range;

[0019] Select command word data with an error rate below the preset range and use it as valid command word data;

[0020] Generate corresponding phoneme embedding representations based on valid command word data;

[0021] All generated phoneme embeddings are combined into a phoneme embedding dictionary.

[0022] Further, the step of determining the mixed recognition command word corresponding to the current command word and generating the mixed recognition list includes:

[0023] Identify the phoneme sequence corresponding to the current command word;

[0024] Calculate the edit distance between the current command word phoneme sequence and the phoneme sequence corresponding to the command word contained in the command word data;

[0025] Identify command words whose phoneme sequence edit distance from the current command word is less than a preset threshold;

[0026] The corresponding command word is identified as a mixed recognition command word and organized into a mixed recognition list; wherein the mixed recognition command word includes the current command word.

[0027] Furthermore, the step of identifying the audio embedding sequence corresponding to the current command word further includes:

[0028] Iterate through each command word contained in the command word data;

[0029] Evaluate the similarity between the stated command word and the current command word;

[0030] Based on the similarity assessment results, the mixed recognition command words are identified, and the mixed recognition command words are...

[0031] Add to the mixed recognition candidate list.

[0032] Further, the step of obtaining the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary includes:

[0033] Based on the candidate list of mixed recognition, extract the phoneme sequence corresponding to each mixed recognition command word from the phoneme embedding dictionary;

[0034] For each mixed recognition command word, the corresponding phoneme sequence is identified from the phoneme embedding dictionary, and a phoneme embedding sequence is generated.

[0035] Further, the step of calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain the recognition result includes:

[0036] Normalize each embedding vector in the audio embedding sequence and the phoneme embedding sequence;

[0037] Based on the normalized audio embedding sequence and phoneme embedding sequence, a dynamic programming algorithm is used to calculate the similarity score between the audio embedding sequence and each of the phoneme embedding sequences.

[0038] By comparing the similarity scores, the phoneme embedding sequence with the highest similarity to the audio embedding sequence is selected to obtain the recognition result.

[0039] A second aspect of this application provides a mixed identification processing device, comprising:

[0040] The generation module is used to identify command word data based on the command word recognition model, select command word data with an error rate lower than a preset range, generate corresponding phoneme embedding representations, and form a phoneme embedding dictionary;

[0041] The determination module is used to determine the mixed recognition command words corresponding to the current command word and generate a mixed recognition list;

[0042] The recognition module is used to identify the audio embedding sequence corresponding to the current command word;

[0043] The acquisition module is used to obtain the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary;

[0044] The calculation module is used to calculate the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain the recognition result.

[0045] A third aspect of this application also includes a computer device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of any of the methods described above.

[0046] The fourth aspect of this application also includes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0047] Beneficial effects

[0048] This invention identifies command word data based on a command word recognition model, selects command word data with an error rate below a preset range, and generates a phoneme embedding dictionary. This invention does not require model retraining; phoneme embeddings are generated based on audio embeddings, ensuring that audio and text reside in the same feature space and avoiding differences between different feature spaces. Based on the phoneme embedding dictionary, the phoneme embedding sequence for each command word in the mixed recognition list is obtained, ensuring consistency between audio and text embeddings. A dynamic programming algorithm is used to calculate the similarity between the audio embedding sequence and each phoneme embedding sequence, selecting the phoneme embedding sequence with the highest similarity to obtain the final recognition result. By setting a similarity threshold, the recognition result is further confirmed, reducing the probability of misrecognition. This significantly improves the accuracy and reliability of command word recognition, is computationally efficient, consumes few resources, and is easy to deploy on embedded devices. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating a mixed identification processing method according to an embodiment of this application;

[0050] Figure 2 This is a schematic block diagram of a mixed identification processing device according to an embodiment of this application;

[0051] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application.

[0052] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0055] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0056] Reference Figure 1 This invention provides a mixed identification processing method, including steps S1-S5, specifically:

[0057] S1. Based on the command word recognition model, the command word data is recognized, and command word data with an error rate lower than the preset range is selected to generate the corresponding phoneme embedding representation and form a phoneme embedding dictionary;

[0058] S2. Determine the mixed recognition command words corresponding to the current command word and generate a mixed recognition list;

[0059] S3. Identify the audio embedding sequence corresponding to the current command word;

[0060] S4. Obtain the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary;

[0061] S5. Calculate the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain the recognition result.

[0062] As described in step S1 above, in this embodiment, a well-trained command word recognition model is first required. This model is typically pre-trained on a large general corpus and then fine-tuned on a specific command word dataset to adapt to the specific command word recognition task. The command word recognition model is used to identify command word data, with the aim of selecting high-quality command word data that will be used to generate phoneme embedding representations. The recognition process involves converting the command word audio into text form while recording the recognition result for each command word. To ensure the quality of the generated phoneme embedding representations, only command word data with an error rate below a preset range is selected. The preset range is chosen based on the needs of the actual application scenario and is usually a small value, such as 5% or lower. The selected command word data is considered high-quality because it exhibits high accuracy during the recognition process. Next, corresponding phoneme embedding representations are generated based on these high-quality command word data. The process of generating phoneme embedding representations includes aligning the command word audio, finding the correspondence between audio and phonemes, and then calculating the embedding vector for each phoneme. These embedding vectors are ultimately combined into a phoneme embedding dictionary, which will be used in subsequent mixed recognition processing steps.

[0063] As described in step S2 above, when determining the mixed-recognition command word corresponding to the current command word, it is first necessary to identify the phoneme sequence of the current command word. This step is completed based on the command word recognition model, which converts the audio of the current command word into its corresponding phoneme sequence. Next, the edit distance between the phoneme sequence of the current command word and the phoneme sequences of other command words is calculated. Edit distance is a method to measure the similarity between two sequences; it defines the minimum number of single-character editing operations (insertion, deletion, or replacement) required to transform one sequence into another. By calculating the edit distance, the similarity between different command words can be quantified. If the edit distance between the phoneme sequence of a command word and the phoneme sequence of the current command word is less than a preset threshold, the two command words are considered to be easily confused, and the command word will be added to the mixed-recognition list. The preset threshold can be adjusted according to the actual situation, and usually a value that can balance the sensitivity and accuracy of mixed-recognition detection is chosen. It should be noted that the mixed-recognition list should not only include other command words that may be confused with the current command word, but also the current command word itself. The process of generating the mixed-recognition list helps to more accurately identify command words in subsequent steps and reduce misidentification caused by similar pronunciation. The distinction between mixed recognition does not require setting a threshold, and is based on the nearest distance, making the calculation simple and efficient.

[0064] As described in step S3 above, the audio of the current command word is first processed using a command word recognition model to extract its feature representation. These feature representations are typically a series of high-dimensional vectors, each representing a small segment of the audio. To better reflect the phoneme features of the audio, these vectors need to be normalized to ensure that the magnitude of each vector is 1. Normalization can be achieved through simple mathematical operations, i.e., dividing each vector by its own magnitude. The normalized vectors can be directly used to calculate cosine similarity, which is the basis for comparing the similarity between the audio embedding sequence and the phoneme embedding sequence in subsequent steps. Next, the left and right boundaries of the current command word are determined based on the output of the command word recognition model. The determination of the left and right boundaries is accomplished by analyzing the CTC matrix output by the model, finding the path with the maximum score, and using a backtracking algorithm to find the alignment path between the audio and the phonemes. The alignment path indicates which parts of the audio correspond to which phonemes of the command word, which is crucial for accurately extracting the audio embedding sequence. The extracted audio embedding sequence will be used in subsequent steps for similarity comparison with the phoneme embedding sequence to determine the most likely command word. At the same time, it can further confirm the identified command words, reducing the probability of misidentification.

[0065] As described in step S4 above, it is first necessary to traverse each command word in the mixed recognition list. For each command word, its corresponding phoneme sequence is searched from the phoneme embedding dictionary. The phoneme embedding dictionary, generated in step S1, contains the phonemes of all command words and their corresponding embedding vectors. The phoneme sequence of each command word consists of a series of phonemes, each with a corresponding embedding vector in the dictionary. By searching these embedding vectors, a phoneme embedding sequence can be generated for each command word. The generated phoneme embedding sequence is a two-dimensional array, where each element is an embedding vector representing a phoneme in the command word. These phoneme embedding sequences will be used for similarity comparison with the audio embedding sequence in subsequent steps to determine the most likely command word. In this way, it can be ensured that the audio embedding sequence and the phoneme embedding sequence are in the same feature space during similarity comparison, improving the accuracy and efficiency of the comparison. Since the phoneme sequences are generally short, this invention is computationally efficient, consumes very few computing resources, and is easy to deploy. The shortness of the phoneme sequences greatly reduces the computational complexity, making it suitable for running on resource-constrained embedded devices.

[0066] As described in step S5 above, after generating the audio embedding sequence and the phoneme embedding sequence, the next step is to calculate their similarity to determine the most likely command word. This process first requires normalizing each embedding vector in the audio embedding sequence and the phoneme embedding sequence to ensure that the magnitude of each vector is 1. Normalization makes the vector dot product directly equivalent to the cosine similarity calculation, simplifying the similarity calculation process. The normalized audio embedding sequence and the phoneme embedding sequence are denoted as Sa and St, respectively, where Sa has a shape of T×d and St has a shape of L×d, T represents the length of the audio embedding sequence, L represents the length of the command word phoneme sequence, and d represents the dimension of the embedding vector. Next, the similarity between Sa and St is calculated using a dynamic programming algorithm. Calculating from the end to the beginning, the last k1 elements of the audio sequence are mapped to the last k2 elements of the text sequence, and the values ​​filling the dynamic programming table (DP table) are calculated sequentially. The DP table has a shape of L×T, and each cell dp[i][j] stores the maximum similarity score between the first j elements of the audio embedding sequence and the first i elements of the phoneme embedding sequence. Finally, the similarity score is calculated as dp[L-1][0] / m, where m is a normalization factor used to adjust the range of the similarity score. By comparing the similarity scores of all mixed recognition command words, the phoneme embedding sequence with the highest similarity to the audio embedding sequence can be selected, and the corresponding command word is the final recognition result. In addition, a threshold can be set. If the maximum similarity score is lower than the threshold, the recognition result is considered unreliable and can be discarded to avoid misrecognition. This method not only improves the accuracy of command word recognition but also effectively reduces misrecognition caused by similar pronunciation, thus improving the user experience. The embedding obtained based on the capabilities of the command word model itself effectively combines the model's recognition capabilities. The phoneme embedding is generated based on the audio embedding, so that the embeddings of the audio and text modalities are in the same feature space, avoiding the differences between different feature spaces and ensuring that the generated embedding vector can accurately reflect the features of the command word, further improving the accuracy and robustness of recognition.

[0067] Steps S1-S5 generate a phoneme embedding dictionary by selecting command word data with low error rates, ensuring high-quality phoneme embedding and effectively identifying easily confused command words. The audio embedding sequence of the current command word is extracted, and then the phoneme embedding sequence of each command word in the mixed recognition list is obtained based on the phoneme embedding dictionary, ensuring that the audio and text embeddings are in the same feature space. By calculating the similarity between the audio embedding sequence and the phoneme embedding sequence, the most likely command word is finally determined. This entire process not only improves recognition accuracy but also simplifies computation and reduces the probability of misidentification through dynamic programming algorithms and nearest-neighbor methods. Furthermore, it is computationally efficient, resource-efficient, and easy to deploy on embedded devices.

[0068] In one embodiment, before the step of recognizing command word data based on the command word recognition model, the following steps are included:

[0069] S10. Feature extraction based on general corpus;

[0070] S11. Based on the extracted features, a preliminary model is obtained by training the model using a pre-defined model architecture.

[0071] S12. Select and adjust data based on the preset command word dataset;

[0072] S13. Based on preset fine-tuning parameters, input adjustment data to train the preliminary model to obtain a command word recognition model.

[0073] In this embodiment, feature extraction is first performed using a large general corpus. This general corpus can come from various sources, such as publicly available speech datasets or domain-specific recordings. The feature extraction process typically involves converting the audio signal into a set of meaningful feature vectors. Common features include Mel-frequency cepstral coefficients (MFCCs) and filter bank energy (FBANK). These feature vectors capture important information from the audio signal, providing a foundation for subsequent model training. After feature extraction, the extracted features are trained using a pre-defined model architecture (such as a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or a Transformer). During training, the model learns how to map the input feature vectors to their corresponding labels (i.e., the textual representations of command words). Through large-scale training on a general corpus, the model learns the basic patterns and structures of speech signals, forming a preliminary command word recognition model. To better adapt the preliminary model to specific command word recognition tasks, fine-tuning is required on a specific command word dataset. First, a portion or all of the data from the pre-defined command word dataset is selected as adjustment data. This data typically contains audio samples of various command words and their corresponding labels, covering various situations that may occur in real-world application scenarios. The selection of data for adjustment should ensure diversity and representativeness to improve the model's generalization ability. Finally, using preset fine-tuning parameters, the adjusted data is input into the initial model for further training. Fine-tuning parameters include the learning rate, batch size, and number of training epochs; these parameters need to be optimized based on the specific task and dataset characteristics. Through fine-tuning, the model can further improve its performance on a specific command word dataset, ultimately resulting in a model specifically designed for command word recognition. This model will be used in subsequent steps to recognize command word data, generating high-quality phoneme embedding representations to support mixed recognition processing.

[0074] In one embodiment, the step of identifying command word data based on a command word recognition model, selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary includes:

[0075] S20. Use the command word recognition model to recognize the selected command word data and generate recognition results;

[0076] S21. Calculate the error rate of the recognition results and compare it with the preset range;

[0077] S22. Select command word data with an error rate lower than the preset range and use it as valid command word data;

[0078] S23. Generate the corresponding phoneme embedding representation based on the effective command word data;

[0079] S24. Combine all generated phoneme embedding representations into a phoneme embedding dictionary.

[0080] In this embodiment, firstly, a pre-trained command word recognition model is used to recognize selected command word data. This command word data is typically pre-processed audio files, with each file corresponding to one command word. The model converts these audio files into corresponding text representations, generating recognition results. The recognition results include the predicted label and confidence score for each command word. During this process, the model uses the CTC (Connectionist Temporal Classification) algorithm for decoding, generating the final recognition results. Next, the error rate of the recognition results for each command word is calculated. The error rate can be calculated by comparing the model's predicted label with the true label; commonly used metrics include word error rate (WER) or grammatical error rate (CER). The calculated error rate is compared with a preset error rate range. The preset error rate range is usually a small value, such as 5% or lower, and the specific value can be adjusted according to the needs of the actual application scenario. Based on the error rate comparison results, command word data with error rates below the preset range are selected. This data is considered high-quality because it exhibits high accuracy during the recognition process. This high-quality command word data will be retained as the basis for subsequent generation of phoneme embedding representations. For the selected valid command word data, corresponding phoneme embedding representations are generated. The process of generating phoneme embedding representations includes aligning the command word audio, finding the correspondence between audio and phonemes, and then calculating the embedding vector for each phoneme. Specifically, the forward algorithm of the CTC algorithm is used to calculate the path score, and the path with the highest score is found. This step uses a backtracking algorithm to determine the alignment path between audio and phonemes. After the alignment path is determined, the audio segments corresponding to each phoneme are extracted, and the embedding vectors of these segments are calculated. These embedding vectors reflect the features of the phonemes and are used to generate phoneme embedding representations. Finally, all generated phoneme embedding representations are combined into a phoneme embedding dictionary. The phoneme embedding dictionary is a collection containing the phonemes of all valid command words and their corresponding embedding vectors. This dictionary will be used in subsequent steps to help identify mixed recognition command words of the current command word and perform similarity calculations. Through the above steps, this invention can not only generate high-quality phoneme embedding representations but also ensure that these representations have high accuracy and reliability in subsequent mixed recognition processing. In particular, this invention does not require model retraining. Phoneme embedding is generated based on audio embedding, ensuring that the embeddings of both audio and text modalities reside in the same feature space, avoiding differences between different feature spaces. This method not only improves recognition accuracy but also simplifies the computation process, reduces the probability of false recognition, and is computationally efficient, resource-saving, and easy to deploy on embedded devices. Alignment using the maximum score path of CTC ensures the accuracy and consistency of phoneme embedding.

[0081] In one embodiment, the step of determining the mixed recognition command word corresponding to the current command word and generating a mixed recognition list includes:

[0082] S30. Identify the phoneme sequence corresponding to the current command word;

[0083] S31. Calculate the edit distance between the current command word phoneme sequence and the phoneme sequence corresponding to the command word contained in the command word data;

[0084] S32. Identify command words whose phoneme sequence edit distance from the current command word is less than a preset threshold;

[0085] S33. Determine the corresponding command word as a mixed recognition command word and organize it into a mixed recognition list; wherein the mixed recognition command word includes the current command word.

[0086] In this embodiment, firstly, a command word recognition model is used to identify the phoneme sequence of the current command word. This step involves inputting the audio of the current command word into the command word recognition model, which then outputs the phoneme sequence of the command word. A phoneme sequence consists of a series of phonemes, each representing a unit of articulation. For example, the phoneme sequence of the command word "play the previous song" might be "bo1fang4shang4yi1shou3". Next, the edit distance between the phoneme sequence of the current command word and the phoneme sequences of other command words is calculated. Edit distance is a method for measuring the similarity between two sequences, defining the minimum number of single-character editing operations (insertion, deletion, or replacement) required to transform one sequence into another. By calculating the edit distance, the similarity between different command words can be quantified. For example, calculating the edit distance between the phoneme sequences of "play the previous song" and "play the next song" reveals that the phoneme sequences of these two command words are very similar, with a small edit distance. Based on the calculated edit distance, command words whose edit distance to the phoneme sequence of the current command word is less than a preset threshold are identified. The preset threshold is chosen based on the needs of the actual application scenario and is usually a small value, such as 2 or 3. If the edit distance between the phoneme sequence of a command word and the phoneme sequence of the current command word is less than the preset threshold, the two command words are considered easily confused, and the command word will be marked as a mixed recognition command word. Finally, the identified mixed recognition command words are compiled into a mixed recognition list. The mixed recognition list should not only include other command words that are easily confused with the current command word, but also the current command word itself. For example, if the current command word is "play the previous song", the mixed recognition list may include "play the previous song", "play the next song", "play this song", etc. The generated mixed recognition list will be used in subsequent steps to help further confirm the recognition results and reduce the probability of misrecognition.

[0087] Through the above steps, this invention can effectively identify other command words that are easily confused with the current command word and generate a mixed recognition list. The distinction between mixed recognition and other words does not require setting a threshold; it is based on nearest-neighbor acquisition, making the calculation simple and efficient. This method not only simplifies the calculation process but also improves the accuracy and efficiency of mixed recognition, ensuring the reliability of the final recognition result. Furthermore, since phoneme sequences are generally short, this invention is computationally efficient, consumes minimal computing resources, and is easy to deploy. The short length of the phoneme sequences significantly reduces computational complexity, making it suitable for operation on resource-constrained embedded devices.

[0088] In one embodiment, the step of identifying the audio embedding sequence corresponding to the current command word further includes:

[0089] S40. Iterate through each command word contained in the command word data;

[0090] S41. Evaluate the similarity between the command word and the current command word;

[0091] S42. Based on the similarity assessment results, determine the mixed recognition command words and add the mixed recognition command words to the mixed recognition candidate list.

[0092] In this embodiment, firstly, every command word in the command word dataset is traversed. The purpose of this step is to evaluate all possible command words to determine which command words are similar to the current command word and may cause misidentification. The command word dataset typically contains audio samples of various command words and their corresponding labels, covering various situations that may occur in real-world applications. For each command word, its similarity to the current command word is evaluated. Similarity evaluation can be performed using various methods, common methods including calculating the cosine similarity or Euclidean distance between audio embedding sequences. Based on the similarity evaluation results, command words with high similarity to the current command word are identified and added to the misidentification candidate list. By traversing all command words and directly evaluating similarity, the method is simple, direct, and easy to implement.

[0093] In one embodiment, the step of obtaining the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary includes:

[0094] S50. Based on the mixed recognition candidate list, extract the phoneme sequence corresponding to each mixed recognition command word from the phoneme embedding dictionary;

[0095] S51. For each mixed recognition command word, identify the corresponding phoneme embedding from the phoneme embedding dictionary and generate a phoneme embedding sequence.

[0096] In this embodiment, firstly, based on the generated mixed recognition candidate list, the phoneme sequence corresponding to each mixed recognition command word is extracted from the phoneme embedding dictionary. The mixed recognition candidate list contains command words with high similarity to the current command word, which may cause mixed recognition. The phoneme embedding dictionary is a collection containing the phonemes of all command words and their corresponding embedding vectors. The specific steps are as follows: traverse each command word in the mixed recognition candidate list. Find the phoneme sequence corresponding to each command word from the phoneme embedding dictionary. A phoneme sequence is composed of a series of phonemes, each phoneme representing a pronunciation unit. For example, the phoneme sequence of the command word "play the previous song" might be "bo1fang4shang4yi1shou3". For the phoneme sequence corresponding to each mixed recognition command word, identify the corresponding phoneme embedding from the phoneme embedding dictionary and generate a phoneme embedding sequence. For example, for each phoneme, find the corresponding embedding vector from the phoneme embedding dictionary. The phoneme embedding dictionary stores the embedding vector of each phoneme, and these vectors can capture the features of the phoneme. Combine the found embedding vectors in sequence into a phoneme embedding sequence. The phoneme embedding sequence is a two-dimensional array where each element is an embedding vector representing a phoneme in the command word. For example, if the phoneme sequence for the command word "play the previous song" is "bo1fang4shang4yi1shou3", then the generated phoneme embedding sequence is [embed_bo1,embed_fang4,embed_shang4,embed_yi1,embed_shou3]. By directly looking up phoneme embeddings from the phoneme embedding dictionary, the accuracy and consistency of the phoneme embeddings are ensured, avoiding the complexity of recalculating the embedding vectors. The phoneme embedding dictionary has been pre-generated, and the process of looking up phoneme embeddings is simple, efficient, and consumes few computational resources. This method fully utilizes the existing knowledge of the command word recognition model, ensuring that the generated embedding vectors accurately reflect the features of the command word, further improving the accuracy and robustness of recognition.

[0097] In one embodiment, the step of calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain the recognition result includes:

[0098] S60. Normalize each embedding vector in the audio embedding sequence and the phoneme embedding sequence;

[0099] S61. Based on the normalized audio embedding sequence and phoneme embedding sequence, a dynamic programming algorithm is used to calculate the similarity score between the audio embedding sequence and each of the phoneme embedding sequences.

[0100] S62. Compare the similarity scores, select the phoneme embedding sequence with the highest similarity to the audio embedding sequence, and obtain the recognition result.

[0101] In one embodiment, firstly, each embedding vector in the audio embedding sequence and phoneme embedding sequence is normalized. The purpose of normalization is to ensure that the magnitude of each vector is 1, allowing direct calculation of cosine similarity via vector dot product. For example, each embedding vector is extracted from the audio embedding sequence and phoneme embedding sequence. The magnitude of each embedding vector, i.e., the Euclidean norm of the vector, is calculated. Each embedding vector is divided by its magnitude to obtain the normalized embedding vector. Next, based on the normalized audio embedding sequence and phoneme embedding sequence, a dynamic programming algorithm is used to calculate the similarity score between the audio embedding sequence and each phoneme embedding sequence. The core idea of ​​the dynamic programming algorithm is to calculate from the end to the beginning, mapping the last k1 elements of the audio sequence to the last k2 elements of the text sequence, and sequentially calculating the values ​​to fill the dynamic programming table (DP table). For example, a dynamic programming table dp is created with a shape of L×T, where L is the length of the phoneme embedding sequence and T is the length of the audio embedding sequence. Using the normalized audio embedding sequence and phoneme embedding sequence, the cosine similarity of each pair of embedding vectors is calculated, and the dynamic programming table is filled. Each cell dp[i][j] in the dynamic programming table stores the maximum similarity score between the first j elements of the audio embedding sequence and the first i elements of the phoneme embedding sequence. The similarity score between the audio embedding sequence and each phoneme embedding sequence is calculated using a dynamic programming algorithm. The final similarity score can be obtained from the last cell dp[L-1][T-1] of the dynamic programming table. Finally, the similarity scores of all phoneme embedding sequences are compared, and the phoneme embedding sequence with the highest similarity to the audio embedding sequence is selected to obtain the final recognition result. The similarity scores of all phoneme embedding sequences are compared to find the phoneme embedding sequence with the highest score. The command word corresponding to the phoneme embedding sequence with the highest similarity is used as the final recognition result. If the highest similarity score is lower than a preset threshold, the recognition result can be considered unreliable, and this recognition can be discarded to avoid misidentification. Calculating the similarity score using a dynamic programming algorithm ensures the accuracy and reliability of the similarity calculation, improving the accuracy of recognition. By comparing the similarity scores, the most likely command word is selected to further confirm the recognition result, reducing the probability of misidentification.

[0102] Reference Figure 2 This is a structural block diagram of a mixed identification processing device according to an embodiment of this application. The device includes:

[0103] The generation module 100 is used to identify command word data based on the command word recognition model, select command word data with an error rate lower than a preset range, generate corresponding phoneme embedding representations, and form a phoneme embedding dictionary;

[0104] The determination module 200 is used to determine the mixed recognition command word corresponding to the current command word and generate a mixed recognition list;

[0105] The recognition module 300 is used to recognize the audio embedding sequence corresponding to the current command word;

[0106] The acquisition module 400 is used to acquire the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary;

[0107] The calculation module 500 is used to calculate the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain the recognition result.

[0108] In one embodiment, the above-described apparatus further includes a training module, the training module comprising:

[0109] The extraction unit is used for feature extraction based on a general corpus.

[0110] The first training unit is used to train a preliminary model based on the extracted features using a pre-defined model architecture.

[0111] The selection unit is used to select and adjust data based on a preset command word dataset;

[0112] The fine-tuning unit is used to train the preliminary model based on preset fine-tuning parameters and input adjustment data to obtain a command word recognition model.

[0113] In one embodiment, the generation module 100 includes:

[0114] The first recognition unit is used to recognize the selected command word data using a command word recognition model and generate recognition results;

[0115] The comparison unit is used to calculate the error rate of the recognition result and compare it with a preset range;

[0116] The effective selection unit is used to select command word data with an error rate lower than a preset range and treat it as valid command word data.

[0117] A representation unit is used to generate a corresponding phoneme embedding representation based on valid command word data;

[0118] Combination unit, used to combine all generated phoneme embedding representations into a phoneme embedding dictionary.

[0119] In one embodiment, the determining module 200 includes:

[0120] The second recognition unit is used to recognize the phoneme sequence corresponding to the current command word;

[0121] The distance calculation unit is used to calculate the edit distance between the current command word phoneme sequence and the phoneme sequence corresponding to the command word contained in the command word data;

[0122] The smaller-than-a-threshold recognition unit is used to recognize the command word whose phoneme sequence edit distance from the current command word is less than a preset threshold.

[0123] The sorting unit is used to determine the corresponding command word as a mixed recognition command word and sort it into a mixed recognition list; wherein the mixed recognition command word includes the current command word.

[0124] In one embodiment, the determining module 200 further includes:

[0125] The traversal unit is used to traverse each command word contained in the command word data;

[0126] An evaluation unit is used to evaluate the similarity between the command word and the current command word;

[0127] The joining unit is used to determine the mixed recognition command words based on the similarity evaluation results, and to add the mixed recognition command words.

[0128] Add to the mixed recognition candidate list.

[0129] In one embodiment, the acquisition module 400 includes:

[0130] The extraction unit is used to extract the phoneme sequence corresponding to each mixed recognition command word from the phoneme embedding dictionary based on the mixed recognition candidate list;

[0131] The embedding unit is used to identify the corresponding phoneme embedding from the phoneme embedding dictionary for each phoneme sequence corresponding to the mixed recognition command word, and generate a phoneme embedding sequence.

[0132] In one embodiment, the computing module 500 includes:

[0133] The normalization unit is used to normalize each embedding vector in the audio embedding sequence and the phoneme embedding sequence;

[0134] A dynamic programming unit is used to calculate the similarity score between the audio embedding sequence and each of the phoneme embedding sequences based on the normalized audio embedding sequence and phoneme embedding sequence using a dynamic programming algorithm.

[0135] The comparison unit is used to compare the similarity scores, filter out the phoneme embedding sequence with the highest similarity to the audio embedding sequence, and obtain the recognition result.

[0136] Reference Figure 3 This application also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 3As shown, the computer device includes a processor, internal memory, a storage medium (non-volatile storage medium), and a network interface connected via a system bus. The processor provides computing and control capabilities. The computer device's memory includes the aforementioned storage medium (non-volatile storage medium) and internal memory. The storage medium (non-volatile storage medium) stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the storage medium (non-volatile storage medium). The computer device's database stores usage data, such as data used in a mixed-identity processing method. The computer device's network interface is used for communication with external terminals via a network connection. Furthermore, the computer device may also include input devices and a display screen. When the aforementioned computer program is executed by a processor, it implements a mixed recognition processing method, including the following steps: recognizing command word data based on a command word recognition model, selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary; determining mixed recognition command words corresponding to the current command word, and generating a mixed recognition list; recognizing the audio embedding sequence corresponding to the current command word; obtaining the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary; calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences, and obtaining the recognition result. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0137] One embodiment of this application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements a mixed recognition processing method, including the following steps: recognizing command word data based on a command word recognition model, selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary; determining mixed recognition command words corresponding to the current command word, and generating a mixed recognition list; recognizing the audio embedding sequence corresponding to the current command word; obtaining the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary; calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences, and obtaining the recognition result. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAM bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0139] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0140] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A mixed identification processing method, characterized in that, The method includes: The command word recognition model is used to identify command word data, and command word data with an error rate lower than a preset range are selected to generate corresponding phoneme embedding representations, thus forming a phoneme embedding dictionary. Identify the mixed recognition command words corresponding to the current command word and generate a mixed recognition list; Identify the audio embedding sequence corresponding to the current command word; The phoneme embedding sequence of each mixed recognition command word in the mixed recognition list is obtained based on the phoneme embedding dictionary; The similarity between the audio embedding sequence and each of the phoneme embedding sequences is calculated to obtain the recognition result; The step of calculating the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain the recognition result includes: Normalize each embedding vector in the audio embedding sequence and the phoneme embedding sequence; Based on the normalized audio embedding sequence and phoneme embedding sequence, a dynamic programming algorithm is used to calculate the similarity score between the audio embedding sequence and each of the phoneme embedding sequences. The similarity scores are compared, and the phoneme embedding sequence with the highest similarity to the audio embedding sequence is selected to obtain the recognition result. If the highest similarity score is lower than the preset threshold, the recognition result is considered unreliable and the recognition is discarded.

2. The mixed identification processing method according to claim 1, characterized in that, Before the step of recognizing command word data based on the command word recognition model, the following steps are included: Feature extraction based on general corpus; Based on the extracted features, a preliminary model is obtained by training the model using a pre-defined model architecture. Data is selected and adjusted based on a pre-defined command word dataset; Based on preset fine-tuning parameters, the preliminary model is trained by inputting adjustment data to obtain a command word recognition model.

3. The mixed identification processing method according to claim 1, characterized in that, The steps of identifying command word data based on the command word recognition model, selecting command word data with an error rate lower than a preset range, generating corresponding phoneme embedding representations, and forming a phoneme embedding dictionary include: The command word recognition model is used to identify the selected command word data and generate recognition results; Calculate the error rate of the recognition results and compare it with the preset range; Select command word data with an error rate below the preset range and use it as valid command word data; Generate corresponding phoneme embedding representations based on valid command word data; All generated phoneme embeddings are combined into a phoneme embedding dictionary.

4. The mixed identification processing method according to claim 1, characterized in that, The step of determining the mixed recognition command word corresponding to the current command word and generating a mixed recognition list includes: Identify the phoneme sequence corresponding to the current command word; Calculate the edit distance between the current command word phoneme sequence and the phoneme sequence corresponding to the command word contained in the command word data; Identify command words whose phoneme sequence edit distance from the current command word is less than a preset threshold; The corresponding command word is identified as a mixed recognition command word and organized into a mixed recognition list; wherein the mixed recognition command word includes the current command word.

5. The mixed identification processing method according to claim 1, characterized in that, The step of identifying the audio embedding sequence corresponding to the current command word further includes: Iterate through each command word contained in the command word data; Evaluate the similarity between the stated command word and the current command word; Based on the similarity assessment results, the mixed recognition command words are identified, and the mixed recognition command words are... Add to the mixed recognition candidate list.

6. The mixed identification processing method according to claim 1, characterized in that, The step of obtaining the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary includes: Based on the candidate list of mixed recognition, extract the phoneme sequence corresponding to each mixed recognition command word from the phoneme embedding dictionary; For each mixed recognition command word, the corresponding phoneme sequence is identified from the phoneme embedding dictionary, and a phoneme embedding sequence is generated.

7. A mixed identification processing device, used to implement the mixed identification processing method according to any one of claims 1-6, characterized in that, include: The generation module is used to identify command word data based on the command word recognition model, select command word data with an error rate lower than a preset range, generate corresponding phoneme embedding representations, and form a phoneme embedding dictionary; The determination module is used to determine the mixed recognition command words corresponding to the current command word and generate a mixed recognition list; The recognition module is used to identify the audio embedding sequence corresponding to the current command word; The acquisition module is used to obtain the phoneme embedding sequence of each mixed recognition command word in the mixed recognition list based on the phoneme embedding dictionary; The calculation module is used to calculate the similarity between the audio embedding sequence and each of the phoneme embedding sequences to obtain the recognition result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images