A voice command recognition method and apparatus

Abstract

The application discloses a voice instruction recognition method and device, comprising: processing a to-be-executed voice instruction through a pre-constructed voice recognition model to obtain a decoding result and a decoding vector; obtaining a plurality of primary candidate instructions based on the character similarity between the decoding result and each instruction sequence in a pre-converted instruction sequence set; substituting each primary candidate instruction into the decoding vector to obtain the average score of each primary candidate instruction; screening a plurality of secondary candidate instructions meeting the requirements from the plurality of primary candidate instructions based on the average score, and determining the average score of each secondary candidate instruction; determining the number of common characters of each secondary candidate instruction; determining the comprehensive score of each secondary candidate instruction based on the average score and the number of common characters of each secondary candidate instruction; and determining a specified instruction sequence meeting the requirements based on the comprehensive score.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a voice instruction recognition method and device. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, voice recognition (ASR) has been widely applied in many fields, especially in voice instruction interaction scenarios such as smart home, vehicle-mounted system, mobile terminal, etc. Users control the device to perform corresponding operations through voice instructions, such as “play the previous song” “volume up 20%”, etc., which greatly improves the convenience and naturalness of human-computer interaction.

[0003] At present, end-to-end voice recognition models (such as Paraformer) gradually become mainstream due to their simple structure and high recognition efficiency. However, in the actual application process, due to environmental noise, user pronunciation difference, limited model generalization ability, etc., the voice recognition result may still have errors or omissions, such as misrecognizing “turn on X video” as “X video”, or missing the key word “turn on”; recognizing “play the previous song” as “the previous song”; recognizing “volume up 20%” as “up 20%”, etc. Such errors are particularly significant in scenarios where the instruction structure is relatively fixed but the expression method is diverse. SUMMARY

[0004] One or more embodiments of the present specification provide a voice instruction recognition method, device and equipment to solve the technical problems proposed in the background.

[0005] One or more embodiments of the present specification adopt the following technical solutions:

[0006] A voice instruction recognition method provided by one or more embodiments of the present specification, the method comprises:

[0007] processing the to-be-executed voice instruction through a pre-constructed voice recognition model to obtain a decoding result and a decoding vector;

[0008] obtaining a plurality of primary candidate instructions based on the character similarity between the decoding result and each instruction sequence in the pre-converted instruction sequence set;

[0009] substituting each primary candidate instruction into the decoding vector to obtain the average score of each primary candidate instruction;

[0010] filtering a plurality of secondary candidate instructions that meet the requirements from the plurality of primary candidate instructions based on the average score, and determining the average score of each secondary candidate instruction;

[0011] determining the number of common characters of each secondary candidate instruction;

[0012] determine a comprehensive score of each secondary candidate instruction based on the average score and the common character number of each secondary candidate instruction;

[0013] determine the designated instruction sequence in accordance with the comprehensive score.

[0014] It should be noted that the present application generates a decoding result and a decoding vector by using a speech recognition model, wherein the decoding vector contains deep semantic understanding of the speech signal by the model; then, by calculating the character similarity between the decoding result and the preset instruction sequence, a plurality of candidate instructions are preliminarily screened out, which helps to capture the potential variants of the instruction and avoid complete matching failure caused by pronunciation difference or noise. Then, the present application substitutes each candidate instruction into the decoding vector to calculate an average score, thereby evaluating the semantic consistency of the candidate instruction and the original speech, and screening out secondary candidate instructions with higher scores, which effectively filters out options with large deviation from the speech intent. Further, the present application introduces the common character number as an evaluation index to identify common key elements (such as verbs “open” “play” etc.) in the secondary candidate instructions, thereby emphasizing the integrity of the instruction structure; finally, the comprehensive score is calculated by combining the average score and the common character number, which balances the model confidence and the retention of instruction key components, ensuring that the finally selected instruction sequence not only meets the probability characteristics of the speech input, but also covers the core logic of the instruction.

[0015] Further, before processing the to-be-executed speech instruction by the pre-constructed speech recognition model, the method further comprises:

[0016] obtaining a pre-constructed speech instruction set, wherein the speech instruction set is a collection of a plurality of pre-set speech instruction texts;

[0017] converting each speech instruction text in the speech instruction set into an instruction sequence set.

[0018] It should be noted that when the system obtains a set of various but established instruction texts such as "open X video", "play the previous song", "increase the volume by 20%", and the like, and converts them into a unified instruction sequence. This pre-converted instruction sequence set defines what is an effective instruction that the system can correctly understand and execute. It is this explicit and limited specification set that makes the subsequent "candidate instruction filtering based on character similarity" key step meaningful and possible. It enables the system to efficiently and targetedly match and compare within a closed but comprehensive range of all expected instructions, rather than blindly searching in an infinite number of speech recognition results. This ensures that the system's processing goal is always consistent with the pre-set application scenario and functional boundary from the beginning, effectively avoiding completely irrelevant or illogical conclusions that may arise from model free decoding, providing a fundamental guarantee for the final accurate locking of user instructions, and improving the accuracy and predictability of the interaction.

[0019] Further, the processing of the to-be-executed voice instruction by the pre-constructed speech recognition model to obtain a decoding result and a decoding vector comprises:

[0020] A blank voice instruction is filled at the front end of the to-be-executed voice instruction to obtain a to-be-processed voice instruction;

[0021] The to-be-processed voice instruction is processed by the speech recognition model to obtain a decoding result and a decoding vector.

[0022] It should be noted that in a real voice interaction scenario, the user's instruction is usually short and starts abruptly. The end-to-end model needs a very short warm-up time to establish a stable acoustic and decoding state from silence to start analyzing the voice signal. If the instruction is directly input into the model, the model may not have entered the best analysis state in the first few frames of the instruction, which may easily lead to the omission of key verbs such as "open" and "play" at the beginning of the sentence. By artificially adding a blank voice at the front end of the to-be-executed voice instruction, it is equivalent to providing a signal-free buffer zone for the model, which enables the model's encoder and decoder to complete a smooth transition from silence to listening before it actually contacts the real instruction voice. This enhances the robustness and accuracy of the system in identifying short voice instructions, especially the starting part.

[0023] Further, the processing of the to-be-executed voice instruction by the pre-constructed speech recognition model to obtain a decoding result and a decoding vector comprises:

[0024] The to-be-executed voice instruction is converted into mel-spectrum data;

[0025] encoding the mel-spectrogram data by an encoder of the speech recognition model to extract acoustic features in the mel-spectrogram data;

[0026] decoding the acoustic features by a decoder of the speech recognition model to obtain the decoding result and the decoding vector.

[0027] It should be noted that converting the original speech waveform into mel-spectrogram data is the first key step, because the mel scale simulates the nonlinear perception characteristics of the human ear to different frequency sounds, which enables the model to focus on the most important frequency band information for human speech recognition, thereby strengthening the key acoustic clues related to the content in the speech in the preprocessing stage, while suppressing irrelevant spectral details to some extent. Subsequently, the encoder deeply encodes the mel-spectrogram, which is aimed at going beyond simple spectral analysis, but capturing the inherent temporal correlation and context patterns from continuous acoustic frames, thereby refining robust acoustic features containing pronunciation habits, phoneme sequences, and even coarse-grained lexical information. This high-level acoustic representation is more resistant to interference from environmental noise and pronunciation variations than the original spectrum. Finally, the decoder decodes based on these deep acoustic features, which not only generates a decoding result (i.e., the most likely text sequence), but more importantly, produces a decoding vector containing probability information for each candidate word or character. This decoding vector essentially encapsulates the "belief distribution" of the model for all possible text sequences corresponding to the acoustic input, rather than just a best guess. It is this information-rich vector that creates the prerequisite for the subsequent key re-scoring step of "calculating the average score of the candidate instruction in the decoding vector". The entire process ensures that the system does not simply rely on a one-time, possibly erroneous text decoding result, but can backtrack to a more informative acoustic feature representation level to verify the candidate instruction again, thereby greatly improving the potential and fault tolerance of restoring the user's true instruction intent from complex acoustic signals.

[0028] Further, the plurality of primary candidate instructions are obtained based on a character similarity between the decoding result and each instruction sequence in the pre-converted instruction sequence set, including:

[0029] calculating a character number similarity between the decoding result and each instruction sequence in the speech instruction set;

[0030] selecting a first candidate instruction that has a character number similarity meeting a preset similarity threshold;

[0031] calculating a character set intersection size between the decoding result and the first candidate instruction;

[0032] selecting a second candidate instruction that has a character set intersection size meeting a preset intersection threshold as the primary candidate instruction.

[0033] It needs to be noted that the character number similarity between the decoding result and the instruction sequence is a high-efficiency and low-computational-cost preliminary filtering strategy. It is based on a reasonable assumption that a misrecognized instruction and its true intention usually do not differ greatly in character length. Through a preset similarity threshold, those instructions that are obviously not matched in length can be quickly excluded, greatly reducing the number of instructions that need to enter the next step of fine comparison and improving the overall efficiency. However, length screening alone may be too rough and may retain some instructions that are similar in length but completely irrelevant in content. Therefore, it is crucial to introduce a second screening calculation of the character intersection size. This step considers the "composition" of the instruction and focuses on the number of actual characters shared between the decoding result and the candidate instruction. This ensures that the "primary candidate instructions" screened not only have similar lengths to the input, but more importantly, they share a considerable number of key characters (such as "play", "volume", "20%", etc.). This double safeguard mechanism means that the primary candidate instruction set that enters the next stage of "average score" calculation has already been highly relevant in structure and vocabulary composition to the voice decoding result. This avoids wasting computational resources on instructions that appear to be similar in length but are completely irrelevant in content, and greatly reduces the possibility of misjudging completely irrelevant instructions as high-quality candidates, thereby ensuring that the subsequent semantic reevaluation process based on decoding vectors, which is more computationally expensive, can be carried out in a more reliable and focused candidate pool, ultimately improving the accuracy and efficiency of the entire system in screening the correct instruction.

[0034] Further, the substituting each primary candidate instruction into the decoding vector to obtain an average score of each primary candidate instruction comprises:

[0035] For each primary candidate instruction, substituting each primary candidate instruction into the decoding vector to obtain a score of each primary candidate instruction;

[0036] Based on the number of characters in each primary candidate instruction, determining an average score of each primary candidate instruction.

[0037] It should be noted that the initial candidate instructions selected based on character similarity only represent the surface correlation between text strings, but cannot determine which candidate is more similar to the user's actual pronunciation in terms of acoustics and probability. The decoding vector is a rich data carrier containing probability distribution information of various possible character sequences generated by the speech recognition model after deep analysis of the input speech. Substituting each candidate instruction text into this decoding vector is essentially verifying the matching degree of the candidate instruction as a hypothetical sequence with the original speech signal and obtaining an overall probability score. This score reflects how likely the speech recognition model is to "hear" the candidate instruction. However, directly using the total score may not be fair to long instructions, so by calculating the "average score" (i.e. total score divided by instruction character number), a fair comparison of instructions of different lengths is achieved, focusing the evaluation standard on "average confidence per character", thus eliminating the bias caused by instruction length. This mechanism means that a candidate instruction with a large character intersection but possibly misrecognized by the model due to unclear pronunciation or noise interference may have a lower average score; on the contrary, a candidate with slightly different characters but high acoustic feature matching degree can obtain a higher score. This provides a scientific basis for the next step of secondary screening based on the average score, ensuring that the candidate instructions entering the final comprehensive evaluation are not only "look-alike" instructions, but also "sound-alike" instructions according to the model, greatly enhancing the system's ability to distinguish between confusing instructions.

[0038] Further, the filtering of the secondary candidate instructions from the plurality of initial candidate instructions based on the average scores comprises:

[0039] Shifting the instruction characters of each initial candidate instruction one position to the left and one position to the right, respectively, and recalculating the front average score and the rear average score of the shifted instruction characters;

[0040] Filtering a third candidate instruction whose average score, front average score and rear average score are all greater than a preset mean threshold value, and taking the third candidate instruction as the secondary candidate instruction.

[0041] It should be noted that the decoding vector output by the speech recognition model contains time dimension information, that is, the probability distribution of aligning acoustic frames with text characters. Directly substituting the candidate instruction into the calculated average score is based on an assumed optimal alignment position. However, due to the continuity of speech and pronunciation changes, the starting and ending positions of the character sequence of the correct instruction on the time axis may have a slight shift of one frame. By actively shifting the instruction sequence left and right by one position and recalculating the average score, it is equivalent to "scanning" and verifying the candidate instruction at three adjacent potential alignment points. A truly matched instruction should have a stable acoustic-text alignment relationship, so it should maintain a high confidence (that is, the average score, the previous average score, and the subsequent average score all exceed the threshold) at the correct alignment point and its immediate left and right positions. On the contrary, an instruction that gets a high score only because of accidental alignment will usually drop significantly in score once there is a slight shift. This screening mechanism greatly enhances the robustness requirement of the alignment result, and it ensures that the final winning secondary candidate instruction is not only an instruction with a "high score at a certain alignment point", but also an instruction that "performs stably within a small neighborhood around the optimal alignment point". This effectively prevents misselection caused by slight deviations in speech segmentation or model alignment, and ensures that subsequent decisions based on the number of common characters and the comprehensive score are based on a set of high-quality candidates with reliable alignment and solid acoustic evidence, thereby improving the accuracy of the final recognition result from another dimension.

[0042] Further, the determination of the number of common characters of each secondary candidate instruction comprises:

[0043] Calculating the edit distance of each secondary candidate instruction and the decoding result;

[0044] Based on the edit distance, the number of common characters of each secondary candidate instruction is obtained, and the number of common characters is the total number of characters of all common substrings with a length greater than a preset value.

[0045] It should be noted that the application uses the calculation process of the edit distance to extract all continuous common substrings between two strings whose length exceeds a preset value in reverse, and counts the total number of characters of these substrings. This "common character number" index has stronger semantic directionality. Because in actual voice instructions, those continuous and longer common substrings often correspond to indivisible keywords or phrases, such as "play", "turn up 20%" and the like. A longer common substring can better indicate the matching of a core semantic unit than multiple scattered short common characters. Therefore, this "common character number" index actually measures the degree of overlap between the candidate instruction and the initial recognition result in "meaningful information blocks". In the subsequent calculation of the comprehensive score, this index focusing on the retention degree of key semantic fragments is introduced, combined with the "average score" focusing on the acoustic-probability matching degree, so that the basis for decision of the comprehensive score is more comprehensive: it not only considers the probability confidence that the model considers that the candidate instruction "sounds" like the original voice, but also considers the degree of fit of the candidate instruction to the initial recognition result in "core content". This helps to preferentially select instructions that not only have high acoustic matching degree, but also have complete key information, thereby significantly improving the ability of the system to correct instructions when the key words (such as verbs and quantifiers) of the instructions are misrecognized or omitted.

[0046] Further, the determination of the comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the common character number comprises:

[0047] Obtaining a preset weight value between the average score and the common character number;

[0048] Determining the comprehensive score based on the weight value, the average score of each secondary candidate instruction, and the common character number.

[0049] It needs to be explained that in the last stage of the whole screening process, the system has obtained two sets of key data: one is the "average score" derived from the decoding vector, which represents the confidence degree of the speech recognition model from the perspective of acoustic probability for each candidate instruction; the other is the "common character number" based on the analysis of the edit distance, which quantifies the coincidence degree of the key continuous character fragments with actual semantics between the candidate instruction and the initial recognition result. However, these two indicators may not always be consistent, for example, a certain instruction may have a high acoustic score but miss a key word (low common character number), while another instruction has a slightly lower acoustic score but contains all the core words. At this time, simply relying on either indicator may bring risks. The introduction of preset weight values and linear combination is essentially to establish an interpretable and stable decision rule. This weight distribution reflects the priori cognition and trade-off of the system designer on the relative importance of "acoustic matching accuracy" and "semantic structure integrity". Through this weighting process, the comprehensive score is no longer an isolated value in two dimensions, but a unified composite criterion that combines sound evidence and text evidence. This enables the system to avoid the bias caused by excessive reliance on a single indicator, and instead tends to select the candidate instruction that performs well in both acoustic and semantic dimensions, or slightly deficient in one dimension but extremely outstanding in the other dimension, as the final output as a whole is more balanced and reliable.

[0050] The one or more embodiments of the present specification provide a voice instruction recognition device, comprising:

[0051] An instruction processing unit processes the to-be-executed voice instruction through a pre-constructed voice recognition model to obtain a decoding result and a decoding vector;

[0052] A preliminary selection unit obtains a plurality of primary candidate instructions based on the character similarity between the decoding result and each instruction sequence in the pre-converted instruction sequence set;

[0053] An instruction substitution unit substitutes each primary candidate instruction into the decoding vector to obtain the average score of each primary candidate instruction;

[0054] An instruction screening unit screens a plurality of secondary candidate instructions that meet the requirements from the plurality of primary candidate instructions based on the average score, and determines the average score of each secondary candidate instruction;

[0055] A character number determination unit determines the common character number of each secondary candidate instruction;

[0056] A score determination unit determines the comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the common character number;

[0057] An instruction determination unit determines the designated instruction sequence that meets the requirements based on the comprehensive score.

[0058] One or more embodiments of the present specification provide a voice instruction recognition device, comprising:

[0059] at least one processor and a bus; and

[0060] a memory in communication connection with the at least one processor; wherein

[0061] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0062] process the to-be-executed voice instruction through a pre-constructed voice recognition model to obtain a decoding result and a decoding vector;

[0063] obtain a plurality of primary candidate instructions based on the character similarity between the decoding result and each instruction sequence in a pre-converted instruction sequence set;

[0064] substitute each primary candidate instruction into the decoding vector to obtain an average score of each primary candidate instruction;

[0065] filter a plurality of secondary candidate instructions that meet the requirements from the plurality of primary candidate instructions based on the average score, and determine an average score of each secondary candidate instruction;

[0066] determine the number of common characters of each secondary candidate instruction;

[0067] determine a comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the number of common characters;

[0068] determine a specified instruction sequence that meets the requirements based on the comprehensive score.

[0069] The above at least one technical solution adopted by the embodiments of the present specification can achieve the following beneficial effects:

[0070] The application generates a decoding result and a decoding vector by using a speech recognition model, wherein the decoding vector contains deep semantic understanding of the speech signal by the model; then, by calculating the character similarity between the decoding result and a preset instruction sequence, a plurality of candidate instructions are preliminarily screened out, which helps to capture potential variants of the instruction and avoid complete matching failure caused by pronunciation difference or noise. Then, the application substitutes each candidate instruction into the decoding vector to calculate an average score, thereby evaluating the semantic consistency of the candidate instruction with the original speech, and screening out secondary candidate instructions with higher scores, which effectively filters out options with large deviation from the speech intent. Further, the application introduces the number of public characters as an evaluation index to identify common key elements (such as the verbs "open" and "play") in the secondary candidate instructions, thereby emphasizing the integrity of the instruction structure; finally, by calculating a comprehensive score by integrating the average score and the number of public characters, the model confidence and the retention of instruction key components are balanced, ensuring that the finally selected instruction sequence not only meets the probability characteristics of the speech input, but also covers the core logic of the instruction. BRIEF DESCRIPTION OF DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor. In the drawings:

[0072] Figure 1 An application environment diagram of a voice instruction recognition method provided by one or more embodiments of the present specification;

[0073] Figure 2 A flowchart of a voice instruction recognition method provided by one or more embodiments of the present specification;

[0074] Figure 3 A flowchart of an instruction recognition provided by one or more embodiments of the present specification;

[0075] Figure 4 A structural diagram of a voice instruction recognition device provided by one or more embodiments of the present specification;

[0076] Figure 5 A structural diagram of a voice instruction recognition device provided by one or more embodiments of the present specification. DETAILED DESCRIPTION

[0077] The embodiments of the present specification provide a voice instruction recognition method, device and equipment.

[0078] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0079] The solution proposed in this application can be applied to voice command recognition scenarios in voice command recognition terminals. Figure 1 This specification illustrates an application environment diagram of a voice command recognition method provided in an embodiment, such as... Figure 1 As shown, terminal 102 communicates with server 103 via a network. Data storage system 101 can store data that server 103 needs to process. Data storage system 101 can be integrated on server 103 or placed on the cloud or other network servers. Terminal 102 can acquire historical behavior data of users in different operating scenarios and the operating status of XR glasses; extract common behavioral information of users from the historical behavior data of users in each operating scenario; extract common status information of XR glasses from the operating status of XR glasses in each operating scenario; combine the common behavioral information and the common status information to generate the glasses habit status of XR glasses in the operating scenario; associate the operating scenario with the glasses habit status to obtain behavioral habit tags. Alternatively, the process of constructing the tags described above can be executed on server 103. That is, the server obtains the user's historical behavior data in different operating scenarios and the operating status of the XR glasses; extracts common behavioral information from the user's historical behavior data in each operating scenario; extracts common status information of the XR glasses from the operating status of the XR glasses in each operating scenario; combines the common behavioral information and the common status information to generate the glasses habit status of the XR glasses in the operating scenario; and associates the operating scenario with the glasses habit status to obtain behavioral habit tags.

[0080] Specifically, voice command recognition terminals can include smartphones, smart home appliances, tablets, virtual reality headsets (VR headsets), augmented reality glasses (AR glasses), electronic displays, and mixed reality (MR) devices, etc. MR devices can include MR glasses, MR helmets, MR cameras, etc. In-vehicle systems can include in-vehicle chips, in-vehicle devices (such as in-vehicle infotainment systems, in-vehicle computers, sensors with voice recognition capabilities, etc.).

[0081] Figure 2A flowchart of a voice instruction recognition method is provided for one or more embodiments of the present specification, which can be executed by a voice instruction recognition system. Some input parameters or intermediate results in the flow allow human intervention to adjust to help improve accuracy.

[0082] The method flow steps of the embodiments of the present specification are as follows:

[0083] S201, processing the to-be-executed voice instruction through a pre-constructed voice recognition model to obtain a decoding result and a decoding vector.

[0084] In the embodiments of the present specification, the original voice signal input by the user can be collected and preprocessed, including noise reduction and silence removal. Then, the preprocessed voice signal is converted into mel spectrum data to better represent the acoustic features sensitive to the human ear. Then, the mel spectrum data is input into the encoder of the pre-trained end-to-end voice recognition model (such as Paraformer) to extract a high-level acoustic feature sequence. Finally, the acoustic feature sequence is input into the decoder of the model for decoding; the decoding process not only outputs the text sequence with the highest probability as the decoding result, but also retains the complete data generated during the decoding process, which contains the probability distribution information of each candidate character at each time step, as the decoding vector.

[0085] S202, obtaining a plurality of primary candidate instructions based on the character similarity between the decoding result and each instruction sequence in the pre-converted instruction sequence set.

[0086] In the embodiments of the present specification, the decoding result obtained in S201 can be compared with each instruction sequence in the instruction sequence set constructed in advance for the application scenario. First, the character quantity similarity (for example, based on the string length ratio) between the decoding result and each instruction sequence is calculated, and a first candidate instruction set with a similarity higher than a preset threshold is selected. Subsequently, the maximum common substring length or the character intersection size between the decoding result and each first candidate instruction is calculated, and the instructions with a value higher than another preset threshold are selected again, which are determined as the primary candidate instructions.

[0087] S203, substituting each primary candidate instruction into the decoding vector to obtain the average score of each primary candidate instruction.

[0088] In the embodiments of the present specification, for each primary candidate instruction selected in S202, it is regarded as a candidate text sequence, which can be substituted into S201 to obtain the decoding vector. Using the probability information of each time step corresponding to the character in the decoding vector, the overall appearance probability (or score) of the candidate instruction sequence is calculated. Then, the average probability of each character is obtained by dividing the total score of the candidate instruction by the number of characters in the candidate instruction, which is the average score of the primary candidate instruction.

[0089] S204, filtering a plurality of secondary candidate instructions meeting the requirements from the plurality of primary candidate instructions based on the average scores, and determining an average score of each secondary candidate instruction.

[0090] In the embodiments of the present disclosure, an average score threshold can be set, and all the primary candidate instructions with the average scores higher than the threshold in S203 are filtered out as the secondary candidate instructions. The average scores of these secondary candidate instructions calculated in S203 are directly reserved as their respective average scores for subsequent steps.

[0091] S205, determining a number of common characters of each secondary candidate instruction.

[0092] In the embodiments of the present disclosure, for each secondary candidate instruction obtained in S204, the edit distance between the secondary candidate instruction and the original decoding result in S201 is calculated. In the process of calculating the edit distance or based on the result of the edit distance, all continuous common substrings with lengths exceeding a preset value existing in the secondary candidate instruction and the decoding result are found. The number of characters of these continuous common substrings meeting the condition is accumulated, and the accumulated value is the number of common characters of the secondary candidate instruction.

[0093] S206, determining a comprehensive score of each secondary candidate instruction based on the average score and the number of common characters of the secondary candidate instruction.

[0094] In the embodiments of the present disclosure, appropriate weight values can be pre-set for the average score and the number of common characters respectively. For each secondary candidate instruction, the average score determined in S204 and the number of common characters determined in S205 are multiplied by the respective corresponding weight values respectively, and then the two weighted values are added to obtain the comprehensive score of the secondary candidate instruction.

[0095] S207, determining a designated instruction sequence meeting the requirements based on the comprehensive score.

[0096] In the embodiments of the present disclosure, the comprehensive scores of all the secondary candidate instructions calculated in S206 can be compared. The secondary candidate instruction with the highest comprehensive score is selected and finally determined as the designated instruction sequence meeting the original voice instruction of the user, which is output to the system for execution.

[0097] It should be noted that the voice recognition model is used to generate a decoding result and a decoding vector, wherein the decoding vector contains deep semantic understanding of the voice signal by the model; then, by calculating the character similarity between the decoding result and the preset instruction sequence, a plurality of candidate instructions are preliminarily screened out, which helps to capture the potential variants of the instruction and avoid complete matching failure caused by pronunciation difference or noise. Then, the average score of each candidate instruction is calculated by substituting the candidate instruction into the decoding vector, so as to evaluate the semantic consistency of the candidate instruction and the original voice, and to screen out secondary candidate instructions with higher scores, which effectively filters the options with large deviation from the voice intent. Further, the public character number is introduced as an evaluation index to identify common key elements (such as verbs such as "open" and "play") in the secondary candidate instructions, so as to emphasize the integrity of the instruction structure; finally, the comprehensive score is calculated by comprehensively calculating the average score and the public character number, balancing the model confidence and the retention of the key components of the instruction, so as to ensure that the finally selected instruction sequence not only meets the probability characteristics of the voice input, but also covers the core logic of the instruction.

[0098] Further, before processing the to-be-executed voice instruction by the pre-constructed voice recognition model, a pre-constructed voice instruction set can be acquired, wherein the voice instruction set is a collection of a plurality of pre-set voice instruction texts; and each voice instruction text in the voice instruction set is converted into an instruction sequence set.

[0099] It should be noted that for the above content, the embodiments of the present specification can systematically extract all valid voice instruction texts from the entire function description document or configuration file supported by the system. These texts should directly correspond to executable operations, such as "turn on the air conditioner", "turn up the volume", "play the next song", etc., forming a non-redundant voice instruction text set covering all possible instructions.

[0100] At the same time, by using the same segmentation and mapping rules as the subsequent voice recognition model, each instruction text in the voice instruction set is converted into a token list composed of discrete units. Specifically, first, a piece of instruction text is segmented, and then each word or character segmented is mapped to a unique, numerical token identifier. Finally, each instruction text corresponds to a token list, and all these token lists together form an instruction sequence set.

[0101] It should be noted that when the system obtains a set of various but established instruction texts such as "open X video", "play the previous song", "increase the volume by 20%", and converts it into a unified instruction sequence. This pre-converted instruction sequence set defines what is the effective instruction that the system can correctly understand and execute. It is this explicit and limited specification set that makes the subsequent "candidate instruction filtering based on character similarity" step meaningful and possible. It makes the system not blindly search in the infinite possibilities of speech recognition results, but can efficiently and targetedly match and compare within a closed but all-expected instruction range. The whole method ensures from the beginning that the system's processing goal is always consistent with the pre-set application scenario and functional boundary, effectively avoiding completely irrelevant or illogical conclusions that may be caused by model free decoding, providing a fundamental guarantee for the final accurate locking of user instructions, and improving the accuracy and predictability of interaction.

[0102] Further, in real voice interaction scenarios, user instructions are usually short and start abruptly. The end-to-end model needs very short warm-up time to establish stable acoustic and decoding state from silence to start analyzing voice signals. If the instruction is directly input into the model, the model may not have entered the best analysis state in the first few frames of the instruction, which may cause the key verbs such as "open" and "play" at the beginning of the sentence to be missed. Therefore, Figure 3 A specific flowchart of a voice instruction recognition method provided by one or more embodiments of the present specification is provided. Some input parameters or intermediate results in the flow allow manual intervention to adjust to help improve accuracy.

[0103] The method flow steps of the embodiments of the present specification are as follows:

[0104] S301, fill a blank voice instruction at the front end of the voice instruction to be executed to obtain a voice instruction to be processed.

[0105] In the embodiments of the present specification, a digital mute signal with fixed duration and short duration is generated by the system, the signal amplitude is zero, and the sampling rate is consistent with the processing requirement of the voice recognition model. This signal is defined as "blank voice instruction". Then, through audio data splicing operation, the blank voice instruction is placed before the start position of the "to-be-executed voice instruction" input by the user, and the two are connected end to end to form a new longer audio data block. The newly generated audio data block is formally defined as "to-be-processed voice instruction" and is immediately delivered to the subsequent voice recognition model for processing.

[0106] S302, processing the to-be-processed voice instruction through the voice recognition model to obtain a decoding result and a decoding vector.

[0107] In the embodiments of the present specification, see S201 described above, which will not be repeated here.

[0108] S303, obtaining a plurality of primary candidate instructions based on the character similarity between the decoding result and each instruction sequence in the pre-converted instruction sequence set.

[0109] In the embodiments of the present specification, see S202 described above, which will not be repeated here.

[0110] S304, substituting each primary candidate instruction into the decoding vector to obtain the average score of each primary candidate instruction.

[0111] In the embodiments of the present specification, see S203 described above, which will not be repeated here.

[0112] S305, screening a plurality of secondary candidate instructions that meet the requirements from the plurality of primary candidate instructions based on the average score, and determining the average score of each secondary candidate instruction.

[0113] In the embodiments of the present specification, see S204 described above, which will not be repeated here.

[0114] S306, determining the number of common characters of each secondary candidate instruction.

[0115] In the embodiments of the present specification, see S205 described above, which will not be repeated here.

[0116] S307, determining the comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the number of common characters.

[0117] In the embodiments of the present specification, see S206 described above, which will not be repeated here.

[0118] S308, determining the designated instruction sequence that meets the requirements based on the comprehensive score.

[0119] In the embodiments of the present specification, see S207 described above, which will not be repeated here.

[0120] It should be noted that by artificially adding a blank voice at the front end of the to-be-executed voice instruction, a buffer zone without signal is provided for the model, which makes the encoder and decoder of the model complete a smooth transition from silence to listening before the real instruction voice is contacted. Thus, the robustness and accuracy of the system in identifying short voice instructions, especially the starting part of the short voice instructions, are enhanced as a whole.

[0121] Further, when the pre-constructed speech recognition model is used to process the to-be-executed voice instruction, the to-be-executed voice instruction can be converted into mel-spectrogram data; the mel-spectrogram data is encoded by an encoder of the speech recognition model to extract acoustic features in the mel-spectrogram data; and the acoustic features are decoded by a decoder of the speech recognition model to obtain the decoding result and the decoding vector.

[0122] It should be noted that for the above content, the embodiments of the present specification can receive the "to-be-processed voice instruction" (i.e., the complete audio signal that may have been filled with blank speech in advance) output by the previous pre-processing. First, the audio signal is pre-processed by frame division, windowing, and short-time Fourier transform to convert it from a time-domain signal to a linear frequency spectrum. Then, a set of mel-scale filters simulating human auditory characteristics are applied to filter and compress the linear frequency spectrum, and finally convert it to mel-spectrogram data that can highlight the key features of the speech.

[0123] Then, the mel-spectrogram data obtained in the previous step is input into the encoder of the pre-trained speech recognition model. The encoder processes the mel-spectrogram data frame by frame or based on an attention mechanism to extract a high-level robust acoustic feature sequence containing context information.

[0124] Finally, the acoustic feature sequence output by the encoder is input into the decoder component of the model. The decoder searches based on the acoustic feature sequence. The decoding process finally outputs a token sequence with the highest probability as the "decoding result". At the same time, the probability distribution of all possible tokens corresponding to each time step in the decoding process is recorded completely, and this complete probability distribution dataset is the "decoding vector".

[0125] It should be noted that the conversion of the original speech waveform into the mel-spectrogram data is a key first step, because the mel-scale simulates the non-linear perception characteristics of the human ear to different frequency sounds, which enables the model to focus on the most important frequency band information for human speech recognition, thereby strengthening the key acoustic clues related to the content in the speech in the preprocessing stage, while suppressing irrelevant spectral details to some extent. Subsequently, the encoder deeply encodes the mel-spectrogram, which is aimed at going beyond simple spectral analysis, but capturing the inherent temporal correlation and context patterns from continuous acoustic frames, thereby refining robust acoustic features containing pronunciation habits, phoneme sequences, and even coarse-grained lexical information. This high-level acoustic representation is more resistant to interference from environmental noise and pronunciation variations than the original spectrum. Finally, the decoder decodes based on these deep acoustic features, which not only generates a decoding result (i.e., the most likely text sequence), but more importantly, produces a decoding vector containing probability information for each candidate word or character. This decoding vector essentially encapsulates the model's "belief distribution" for all possible text sequences corresponding to the acoustic input, rather than just a best guess. It is this information-rich vector that creates the prerequisite for the subsequent key re-scoring step of "calculating the average score of the candidate instruction by substituting it into the decoding vector". The entire process ensures that the system does not simply rely on a one-time, possibly erroneous text decoding result, but can backtrack to a more informative acoustic feature representation level to verify the candidate instruction again, thereby greatly improving the potential and fault tolerance of restoring the user's true instruction intent from complex acoustic signals.

[0126] Further, when the plurality of primary candidate instructions are obtained based on the character similarity between the decoding result and each instruction sequence in the pre-converted instruction sequence set, the character number similarity between the decoding result and each instruction sequence in the speech instruction set can be calculated; a first candidate instruction whose character number similarity meets a preset similarity threshold is screened out; the size of the character set intersection between the decoding result and the first candidate instruction is calculated; a second candidate instruction whose character set intersection size meets a preset intersection threshold is screened out, and the second candidate instruction is taken as the primary candidate instruction.

[0127] It should be noted that the decoding result (a token list) generated in S101 and the pre-constructed instruction sequence set (each instruction sequence is also a token list) are obtained. For each instruction sequence in the instruction sequence set, the similarity between the length of the token list thereof and the length of the token list of the decoding result is calculated. This similarity calculation can be realized by comparing the relative proportion or absolute difference between the lengths, and finally a quantitative similarity value is obtained.

[0128] Then, a character number similarity threshold is set. All the similarity values calculated above are compared with the threshold. Instruction sequences with similarity values higher than (or meeting) the threshold are filtered out and classified as the first candidate instruction set.

[0129] After that, for each first candidate instruction filtered out above, its token list is compared with the token list of the decoding result. The number of tokens that are common between the two lists is calculated and defined as the character intersection size.

[0130] Finally, a character intersection size threshold is set. The character intersection size of each first candidate instruction calculated above is compared with the threshold. Instruction sequences with character intersection size higher than (or meeting) the threshold are filtered out and determined as the second candidate instruction, i.e., the final primary candidate instruction set.

[0131] It is noted that calculating the character number similarity between the decoding result and the instruction sequence is an efficient and low computational cost preliminary filtering strategy. It is based on a reasonable assumption that a misrecognized instruction usually does not differ much from its true intention in terms of character length. Through the pre-set similarity threshold, those instructions that are obviously not matched in length can be quickly excluded, greatly reducing the number of instructions that need to enter the next step of fine comparison and improving the overall efficiency. However, relying solely on length screening can be too rough and may retain some instructions that are similar in length but completely irrelevant in content. Therefore, it is crucial to introduce the second screening to calculate the character intersection size. This step considers the "composition" of the instruction and focuses on the number of actual characters that are common between the decoding result and the candidate instruction. This ensures that the "primary candidate instruction" filtered out not only has similar length to the input, but more importantly, it shares a considerable number of key characters (such as "play", "volume", "20%", etc.). This double safeguard mechanism means that the primary candidate instruction set that enters the next stage of "average score" calculation has already been highly relevant to the voice decoding result in terms of structure and vocabulary composition. This avoids wasting computational resources on instructions that appear to be similar in length but are completely irrelevant in content, and greatly reduces the possibility of misjudging completely irrelevant instructions as high-quality candidates. This ensures that the subsequent semantic reevaluation process based on decoding vectors, which is more computationally expensive, can be carried out in a more reliable and focused candidate pool, ultimately improving the accuracy and efficiency of the entire system in filtering out the correct instruction.

[0132] Further, when the initial candidate instruction is substituted into the decoding vector to obtain the average score of the initial candidate instruction, for each initial candidate instruction, the initial candidate instruction is substituted into the decoding vector to obtain a score of the initial candidate instruction; and based on the number of characters in each initial candidate instruction, an average score of each initial candidate instruction is determined.

[0133] It should be noted that the token sequence of the initial candidate instruction can be matched with the decoding vector obtained in step S101. The decoding vector contains the probability distribution information of the speech recognition model on each time step for all possible tokens. By aligning the token sequence of the initial candidate instruction with the probability distribution of the corresponding time step in the decoding vector in turn, and extracting the probability value of each token at the corresponding time step, these probability values are combined, and the total score of the initial candidate instruction as a whole sequence is calculated by multiplication or other probability aggregation methods.

[0134] Subsequently, in order to eliminate the influence of the instruction length on the score, the obtained total score is normalized based on the number of tokens contained in the initial candidate instruction itself. Specifically, the calculated total score is divided by the number of tokens of the instruction, and the result obtained is the average score of the initial candidate instruction. This average score represents the average probability confidence of each token.

[0135] It should be noted that the initial candidate instructions screened based on character similarity only represent the surface correlation between text strings, but cannot determine which candidate is more acoustically and probabilistically close to the user's actual pronunciation. The decoding vector is a rich data carrier containing probability distribution information of various possible character sequences generated by the speech recognition model after deep analysis of the input speech. Substituting each candidate instruction text into this decoding vector is essentially verifying the matching degree of the candidate instruction as a hypothetical sequence with the original speech signal and obtaining an overall probability score. This score reflects how likely the speech recognition model is to "hear" the candidate instruction. However, directly using the total score may not be fair to long instructions, so by calculating the "average score" (i.e., the total score divided by the number of instruction characters), a fair comparison of instructions of different lengths is achieved, focusing the evaluation standard on the "average confidence per character", thereby eliminating the bias caused by instruction length. This mechanism means that a candidate instruction with a large character intersection but possibly misrecognized by the model due to ambiguous pronunciation or noise interference may have a lower average score; conversely, a candidate with slightly different characters but high acoustic feature matching degree can obtain a higher score. This provides a scientific basis for the next step of secondary screening based on the average score, ensuring that the candidate instructions entering the final comprehensive evaluation are not only "look-alike" instructions, but also instructions that the model considers "sound-alike", greatly enhancing the system's discrimination ability when faced with confusing instructions.

[0136] Further, when the required secondary candidate instructions are screened from the plurality of initial candidate instructions based on the average scores, the instruction characters of each initial candidate instruction can be shifted one position to the left and one position to the right, respectively, and the pre-average score and the post-average score of the shifted instruction characters are recalculated; a third candidate instruction whose average score, pre-average score and post-average score are all greater than a preset mean threshold value is screened, and the third candidate instruction is taken as the secondary candidate instruction.

[0137] It should be noted that the token sequence of the initial candidate instruction can be shifted one position to the left on the time axis, i.e., the starting position of the instruction sequence is assumed to be one time step earlier than the original alignment position, and then the shifted sequence is substituted into the decoding vector obtained in S101 to calculate a new average score, called pre-average score. Similarly, the same instruction sequence is shifted one position to the right, i.e., the starting position is assumed to be one time step later, and the average score is recalculated to obtain the post-average score.

[0138] Next, a preset average score threshold is set. For each primary candidate instruction, it is checked whether its original average score, previous average score and subsequent average score are all greater than the preset average score threshold. Only those primary candidate instructions that satisfy the condition are retained, and these retained instructions are classified as third candidate instructions, i.e., final secondary candidate instructions. At the same time, the original average scores of these secondary candidate instructions are directly recorded as their average scores for subsequent steps.

[0139] It should be noted that the decoding vector output by the speech recognition model contains time dimension information, i.e., a probability distribution of aligning acoustic frames with text characters. The average score calculated by directly substituting the candidate instruction is based on an assumed optimal alignment position. However, due to the continuity of speech and pronunciation changes, the start and end positions of the character sequence of a correct instruction on the time axis may have a slight shift of one frame. By actively shifting the instruction sequence left and right by one position and recalculating the average score, it is equivalent to "scanning" and verifying the candidate instruction at three adjacent potential alignment points. A truly matched instruction should have a stable acoustic-text alignment relationship, so it should maintain a high confidence (i.e., average score, previous average score, and subsequent average score all exceed the threshold) at the correct alignment point and its immediate left and right positions. On the contrary, an instruction that obtains a high score only because of accidental alignment will usually decrease significantly in score once a slight shift occurs. This screening mechanism greatly enhances the robustness requirement of the alignment result, which ensures that the final winning secondary candidate instruction is not only "high in score at a certain alignment point", but also "stable within a small neighborhood around the optimal alignment point". This effectively prevents misselection caused by slight deviations in speech segmentation or model alignment, and ensures that subsequent decisions based on the number of common characters and the comprehensive score are based on a set of high-quality candidates with reliable alignment and solid acoustic evidence, thereby improving the accuracy of the final recognition result from another dimension.

[0140] Further, when determining the number of common characters of each secondary candidate instruction, the edit distance between each secondary candidate instruction and the decoding result can be calculated; based on the edit distance, the number of common characters of each secondary candidate instruction is obtained, and the number of common characters is the total number of characters of all common substrings with a length greater than a preset value.

[0141] It should be noted that for each secondary candidate instruction obtained from step S104, the embodiment of the present specification performs a string comparison between it and the decoding result obtained in step S101. Specifically, the edit distance between the character sequence of the secondary candidate instruction and the character sequence of the decoding result is calculated, and the edit distance is calculated by a dynamic programming method or the like to evaluate the degree of difference between the two sequences. Then, based on the character matching information identified in the edit distance calculation process, or by an independent common substring finding algorithm, all continuous and identical character subsequences, i.e. common substrings, are extracted from the secondary candidate instruction and the decoding result. Next, a length threshold is set to filter out all substrings in the common substrings whose length is greater than the preset value. Finally, the number of characters in the filtered common substrings is accumulated, and the sum obtained is the number of common characters of the secondary candidate instruction. This number of common characters quantifies the degree of consistency between the secondary candidate instruction and the original decoding result in the key continuous character segment, providing an important input for the subsequent step S106 to calculate the comprehensive score.

[0142] It should be noted that the present application uses the calculation process of the edit distance to extract all continuous common substrings between the two strings whose length exceeds the preset value in reverse, and counts the total number of characters of these substrings. This "number of common characters" indicator has a stronger semantic direction. Because in actual voice instructions, those continuous and longer common substrings often correspond to indivisible key words or phrases, such as "play", "turn up 20%" and the like. A longer common substring can better indicate the matching of a core semantic unit than multiple scattered short common characters. Therefore, this "number of common characters" indicator actually measures the degree of overlap between the candidate instruction and the initial recognition result in the "meaningful information block". When calculating the comprehensive score later, this indicator focusing on the preservation degree of key semantic segments is introduced in combination with the "average score" focusing on the acoustic-probability matching degree, so that the decision basis of the comprehensive score is more comprehensive: it not only considers the probability confidence that the model thinks the candidate instruction "sounds" like the original voice, but also considers the degree of fit between the candidate instruction and the initial recognition result in "core content". This helps to preferentially select instructions that not only have high acoustic matching degree, but also have complete key information, thereby significantly improving the ability of the system to correct the key words (such as verbs and quantifiers) when they are misrecognized or omitted.

[0143] Further, when determining the comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the number of common characters, a preset weight value between the average score and the number of common characters can be obtained; and the comprehensive score is determined based on the weight value, the average score of each secondary candidate instruction, and the number of common characters.

[0144] It should be noted that the embodiments of the present specification obtain the weight values respectively set in advance for the average score and the number of common characters from the system configuration or parameter setting. The two weight values are used to quantify the relative importance of the average score and the number of common characters in the final decision.

[0145] Then, for each secondary candidate instruction obtained from the S104 and S105 steps, the corresponding average score and the number of common characters are combined with the respective weight values. Specifically, the average score of each secondary candidate instruction is multiplied by the weight value of the average score to obtain the weighted average score contribution; at the same time, the number of common characters of the instruction is multiplied by the weight value of the number of common characters to obtain the weighted number of common characters contribution. Finally, the two weighted contribution values are added to obtain the comprehensive score of the secondary candidate instruction.

[0146] It should be noted that in the final stage of the entire screening process, the system has obtained two sets of key data: one is the "average score" derived from the decoding vector, which represents the confidence degree of the speech recognition model from the perspective of acoustic probability for each candidate instruction; the other is the "number of common characters" based on the analysis of the edit distance, which quantifies the coincidence degree of the key continuous character fragments with actual semantics between the candidate instruction and the initial recognition result. However, these two indicators may not always be consistent, for example, a certain instruction may have a high acoustic score but miss a key word (low number of common characters), while another instruction has a slightly lower acoustic score but contains all the core words. At this time, simply relying on either indicator may pose a risk. The introduction of preset weight values and linear combination is essentially to establish an interpretable and stable decision rule. This weight distribution reflects the prior knowledge and trade-off of the system designer on the relative importance of "acoustic matching accuracy" and "semantic structure integrity". Through this weighting process, the comprehensive score is no longer an isolated value in two dimensions, but a unified composite criterion that combines acoustic evidence and textual evidence. This enables the system to avoid the bias caused by excessive reliance on a single indicator, and instead tends to select the candidate instruction that performs well in both acoustic and semantic dimensions, or slightly deficient in one dimension but extremely outstanding in the other dimension, as a whole more balanced and reliable, as the final output.

[0147] It should be noted that the present application proposes a post-processing method for paraformer speech recognition model, which corrects the fixed instruction set and improves the instruction recognition accuracy. The paraformer speech recognition model improves the problem of wrong and missing words, and realizes efficient correction on the fixed instruction set.

[0148] It should be noted that the present application introduces a post-processing method of a speech recognition model paraformer, and the application scenario is speech instruction interaction. In the speech instruction interaction scenario, the result of asr (speech recognition, speech to text) needs to be corrected to match the correct speech instruction and complete the corresponding interaction. For example, "open X video, play the previous song, and increase the volume by 20%", when applying paraformer for speech recognition, recognition errors and missing words may occur, such as "X video, open X video, previous song, and increase by 20%", the present application corrects the speech recognition result for the scene of a fixed set of speech instructions. The specific scheme is as follows:

[0149] 1. Data preparation

[0150] Instruction set arrangement: assume a set of key speech instructions for operating devices and applications, such as "open X video, close X video, volume up, close volume, previous song, next song", etc. Arrange into an instruction set list.

[0151] Convert to token: convert each instruction text into a token list and save it with the instruction text.

[0152] 2. Speech recognition inference:

[0153] Speech data padding: input a speech, in order to avoid missing the first word, add a length of 10000 of blank data in front.

[0154] Speech data encoding: convert the speech data into mel spectrum data, and use a one-dimensional convolution and several residual attention layers to form an encoding network to encode the data.

[0155] Decoding: use an embedding layer and several residual attention layers to form a decoding network to decode the above speech data encoding, first get the decoding token length L, then get the two-dimensional vector S of L times the model vocabulary length, and the decoding result T of the maximum score.

[0156] 3. Result post-processing:

[0157] Combine the two-dimensional vector S and the instruction set to correct the decoding result T.

[0158] First screening: First, find the closest text distance in the instruction set list with the decoding result T as the candidate result. In order to speed up the processing speed, the text distance is defined as the operation of T and the candidate instruction character set: if the number of characters in the two character sets is different (the number of candidate instruction characters is more than 30% of the number of T characters, and the number of candidate instruction characters is less than 30% of the number of T characters), discard the candidate instruction, otherwise further compare; Then calculate the number of characters in the intersection of the two character sets, if it is less than 0.7 of the minimum of the two character sets, discard the candidate instruction, otherwise keep it.

[0159] Second screening: Bring the candidate instruction into the two-dimensional vector S to get the score X of each instruction, and divide it by the number of tokens in the instruction to get the average score of each instruction. Considering the character offset, shift the candidate instruction characters left and right by one token respectively, and then calculate and get the average score twice. Keep the candidate instruction whose average score is greater than a preset threshold (such as 0.2). The token of the special character in the instruction is normalized, such as ocr may be segmented into 'oc@@' and 'r' or 'o','c','r', which causes misalignment when brought in.

[0160] Third screening: For the remaining candidate instructions, calculate the edit distance of each instruction and T, and save the result with the maximum common character number Y as the final result. Common character number calculation: get all common substrings of the two strings with length greater than 1, and the sum of the number of characters in all common substrings as the common character number. Keep the maximum common character number and the instructions less than the maximum value * 0.9. Among the remaining instructions, judge by the score X and the common character number Y of the instruction, such as calculating the instruction with the maximum X*0.6+Y*0.4 as the final result.

[0161] If there is no remaining instruction, the voice does not match the instruction in any fixed instruction set.

[0162] It should be noted that the instruction screening is designed from easy to difficult in combination with the complexity of character distance and score distance calculation, which realizes the fast screening of a large number of instruction sets under the paraformer model structure, thereby realizing the correction of the fixed instruction set in voice interaction and improving the instruction recognition rate.

[0163] Figure 4 A voice instruction recognition device is provided for one or more embodiments of the present specification, comprising: an instruction processing unit 401, a preliminary selection unit 402, an instruction substitution unit 403, an instruction screening unit 404, a character number determination unit 405, a score determination unit 406, and an instruction determination unit 407.

[0164] The instruction processing unit 401 processes the to-be-executed voice instruction through the pre-constructed voice recognition model to obtain a decoding result and a decoding vector;

[0165] The initial selection unit 402 obtains a plurality of primary candidate instructions based on character similarity between the decoding result and each instruction sequence in the pre-transformed instruction sequence set;

[0166] The instruction substitution unit 403 substitutes each primary candidate instruction into the decoding vector to obtain an average score of each primary candidate instruction;

[0167] The instruction screening unit 404 screens a plurality of secondary candidate instructions meeting the requirements from the plurality of primary candidate instructions based on the average score and determines an average score of each secondary candidate instruction;

[0168] The character number determination unit 405 determines a common character number of each secondary candidate instruction;

[0169] The score determination unit 406 determines a comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the common character number;

[0170] The instruction determination unit 407 determines a designated instruction sequence meeting the requirements based on the comprehensive score.

[0171] Figure 5 A voice instruction recognition device provided for one or more embodiments of the present specification comprises:

[0172] At least one processor and a bus; and,

[0173] A memory in communication connection with the at least one processor; wherein,

[0174] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0175] Process the to-be-executed voice instruction through a pre-constructed voice recognition model to obtain a decoding result and a decoding vector;

[0176] Obtain a plurality of primary candidate instructions based on character similarity between the decoding result and each instruction sequence in a pre-transformed instruction sequence set;

[0177] Substitute each primary candidate instruction into the decoding vector to obtain an average score of each primary candidate instruction;

[0178] Screen a plurality of secondary candidate instructions meeting the requirements from the plurality of primary candidate instructions based on the average score and determine an average score of each secondary candidate instruction;

[0179] Determine a common character number of each secondary candidate instruction;

[0180] determine a comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the common character number;

[0181] determine the designated instruction sequence according to the comprehensive score.

[0182] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the device, equipment, and non-volatile computer storage medium embodiments are basically similar to the method embodiments, and thus the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0183] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the device, equipment, and non-volatile computer storage medium embodiments are basically similar to the method embodiments, and thus the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0184] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the specification can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Professionals can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0185] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are only schematic. The division of the modules or units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0186] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0187] In addition, each of the function units in each of the embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above units can be realized in the form of hardware or in the form of software.

[0188] The integrated module / unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0189] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A voice command recognition method, characterized by, The method comprises: processing the to-be-executed voice instruction through a pre-constructed voice recognition model to obtain a decoding result and a decoding vector; obtaining a plurality of primary candidate instructions based on the character similarity between the decoding result and each instruction sequence in a pre-converted instruction sequence set; substituting each primary candidate instruction into the decoding vector to obtain an average score of each primary candidate instruction; screening a plurality of secondary candidate instructions that meet the requirements from the plurality of primary candidate instructions based on the average score, and determining an average score of each secondary candidate instruction; determining the number of common characters of each secondary candidate instruction; determining a comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the number of common characters; determining a specified instruction sequence that meets the requirements based on the comprehensive score. The method further comprises, before processing the to-be-executed voice instruction through the pre-constructed voice recognition model: obtaining a pre-constructed voice instruction set, the voice instruction set being a set of a plurality of pre-set voice instruction texts; converting each voice instruction text in the voice instruction set into an instruction sequence set.

2. The method of claim 1, wherein, The method further comprises: filling a blank voice instruction at the front end of the to-be-executed voice instruction to obtain a to-be-processed voice instruction; processing the to-be-processed voice instruction through the voice recognition model to obtain a decoding result and a decoding vector.

3. The method of claim 1, wherein, The method further comprises: converting the to-be-executed voice instruction into mel-spectrogram data; encoding the mel-spectrogram data through an encoder of the voice recognition model to extract acoustic features in the mel-spectrogram data; 4. The method of claim 1, wherein, decoding the acoustic features through a decoder of the voice recognition model to obtain the decoding result and the decoding vector. The method further comprises: calculating the character number similarity between the decoding result and each instruction sequence in the voice instruction set; screening a first candidate instruction whose character number similarity meets a pre-set similarity threshold; 5. The method of claim 1, wherein, calculating the size of the character set intersection between the decoding result and the first candidate instruction; screening a second candidate instruction whose character set intersection size meets a pre-set intersection threshold, and taking the second candidate instruction as the primary candidate instruction. The method further comprises: for each primary candidate instruction, substituting each primary candidate instruction into the decoding vector to obtain a score of each primary candidate instruction. ​ 6. The method of claim 1, wherein, ​ ​ Determine average scores of the primary candidate instructions based on the number of characters in each primary candidate instruction.

7. The method of claim 6, wherein, The filtering of the multiple secondary candidate instructions from the multiple primary candidate instructions based on the average scores comprises: Shift the instruction characters of each primary candidate instruction by one position to the left and to the right respectively, and recalculate the front average score and the rear average score of the shifted primary candidate instruction; Filter a third candidate instruction whose average score, front average score and rear average score are all greater than a preset average threshold, and take the third candidate instruction as the secondary candidate instruction.

8. The method of claim 1, wherein, The determination of the comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the common character number comprises: Obtain a preset weight value between the average score and the common character number; Determine the comprehensive score based on the weight value, the average score of each secondary candidate instruction and the common character number.

9. A voice command recognition device, characterized by Comprise: At least one processor and a bus; And, A memory in communication connection with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Process the to-be-executed voice instruction through a pre-constructed voice recognition model to obtain a decoding result and a decoding vector; Obtain multiple primary candidate instructions based on the character similarity between the decoding result and each instruction sequence in a pre-converted instruction sequence set; Substitute each primary candidate instruction into the decoding vector to obtain the average score of each primary candidate instruction; Filter multiple secondary candidate instructions meeting the requirements from the multiple primary candidate instructions based on the average scores, and determine the average score of each secondary candidate instruction; Determine the common character number of each secondary candidate instruction; Determine the comprehensive score of each secondary candidate instruction based on the average score of each secondary candidate instruction and the common character number; Determine the designated instruction sequence meeting the requirements based on the comprehensive score; The determination of the common character number of each secondary candidate instruction comprises: Calculate the edit distance of each secondary candidate instruction and the decoding result; Based on the edit distance, obtain the common character number of each secondary candidate instruction, and the common character number is the total character number of all common substrings with a length greater than a preset value.

Images