Voice command recognition method, device and electronic device

Abstract

The present invention discloses a method, device and electronic device for voice command recognition, wherein the method comprises: collecting a target voice signal and extracting command element information from the target voice signal; arranging and combining the command elements in a pre-stored command element table to generate a plurality of commands; filtering the commands that do not conform to the preset language habit rules in the plurality of commands to obtain the remaining commands; and selecting the command with the highest degree of matching from the remaining commands based on the extracted command element information as the execution command. The technical solution provided by the present invention takes into account the presence of frequently appearing reusable command elements in a large number of different voice commands, and the arrangement and combination of these command elements can cover a large number of commands used in the field of smart homes. Therefore, based on the identified command elements, the command with the highest degree of matching is selected from the commands generated by the arrangement and combination, thereby reducing the storage volume of command entries while ensuring the accuracy of command recognition.

Description

Technical Field

[0001] The present invention relates to the fields of speech recognition and natural language processing, and in particular to a method, device and electronic device for recognizing speech instructions. Background Art

[0002] With the progress of society and the development of science and technology, voice has gradually become a convenient way of human-computer interaction, and more and more families have smart devices with voice interaction functions. In the smart home scenario, the existing voice control system relies on smart speakers or voice treasure products. The voice control system based on smart speakers needs to support a large number of voice control commands and requires free expression. It needs to run continuous speech recognition models and semantic understanding models at the same time. Moreover, these two models are large in scale (usually GB scale) and have high requirements for computing power. Therefore, they cannot be run directly on end-side devices with limited storage and computing power. Existing end-side voice products (such as smart speakers) can only support a limited number of command word recognition models, which require complete matching of specific command words, such as Figure 1 As shown, the system supports a maximum of approximately 100 words. If the user's command "Turn on all the lights in the living room" is one of the pre-stored words, it will be recognized. However, if the user's command is not one of the pre-stored words, it will not be accurately recognized. Therefore, if users want voice products to accurately recognize their voice commands, the voice products need to store a large number of voice commands, resulting in high costs. Alternatively, users may have to remember every voice command accurately, resulting in a poor user experience. Therefore, how to reduce the storage capacity of command words on the end-side device while ensuring command recognition accuracy is an unresolved issue. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method, apparatus, and electronic device for voice command recognition, thereby reducing the storage capacity of command entries on a terminal device while ensuring command recognition accuracy.

[0004] The technical solution adopted in the present invention is:

[0005] According to the first aspect, an embodiment of the present invention provides a voice instruction recognition method, which includes: collecting a target voice signal and extracting instruction element information from the target voice signal; arranging and combining the instruction elements in a pre-stored instruction element table to generate multiple instructions; filtering out instructions from the multiple instructions that do not conform to preset language habit rules to obtain remaining instructions; and selecting the instruction with the highest degree of matching from the remaining instructions based on the extracted instruction element information as the execution instruction.

[0006] Optionally, extracting instruction element information from the target speech signal includes: extracting high-dimensional acoustic features of the target speech signal; performing instruction element recognition on the high-dimensional acoustic features respectively through two neural network models with different receptive fields to obtain two recognition results, and the two recognition results are used to characterize the probability of each instruction element contained in the high-dimensional acoustic features; and fusing the two recognition results to obtain the instruction element information.

[0007] Optionally, the method of selecting the instruction with the highest matching degree from the remaining instructions based on the extracted instruction element information as the execution instruction includes: generating a decoding graph based on the remaining instructions; filling the instruction element probabilities contained in the instruction element information into the corresponding instruction element nodes in the decoding graph; finding the optimal path from the filled decoding graph, and using the instruction corresponding to the optimal path as the execution instruction.

[0008] Optionally, the finding of the optimal path from the filled decoding graph includes: inverting the instruction element probability of the instruction element node in the decoding graph to obtain the cost value of each instruction element node; calculating the cumulative cost value corresponding to each path in the decoding graph based on the pre-generated transfer arc cost value and the cost value of the instruction element node in the decoding graph; and taking the path with the smallest cumulative cost value as the optimal path.

[0009] Optionally, the two neural network models with different receptive field ranges are used to respectively identify the instruction elements of the high-dimensional acoustic features to obtain two recognition results, including: identifying the instruction elements in the high-dimensional acoustic features through the first neural network model to obtain a first recognition result; extracting a first preset number of frames from the high-dimensional acoustic features to remove redundant parts in the high-dimensional acoustic features to obtain a second high-dimensional acoustic feature; identifying the instruction elements in the second high-dimensional acoustic features through the second neural network to obtain a second recognition result; wherein, the receptive field of the first neural network is smaller than the receptive field of the second neural network.

[0010] Optionally, the first recognition result includes the probability of an instruction start frame, the probabilities of several instruction elements and the probability of an instruction end frame, wherein the instruction start frame and the instruction end frame respectively represent the second preset number of speech frames before the instruction starts and after the instruction ends in the high-dimensional acoustic features.

[0011] Optionally, the two recognition results are fused to obtain the instruction element information, including: extracting the sub-recognition result corresponding to the first preset frame number from the first recognition result; judging whether the probability of the instruction end frame is greater than a preset threshold; if the probability of the instruction end frame is greater than the preset threshold, weighted multiplication of the sub-recognition result and the second recognition result to obtain the instruction element information.

[0012] Optionally, the step of generating the pre-stored instruction element table includes: disassembling instruction elements from the collected control instructions and labeling the types of the disassembled instruction elements; separately counting the high-frequency instruction elements in each type whose occurrence frequency is greater than a preset frequency threshold; and summarizing the high-frequency instruction elements in each type into the pre-stored instruction element table.

[0013] According to the second aspect, an embodiment of the present invention provides a voice command recognition device, which includes: an instruction element extraction module for collecting a target voice signal and extracting instruction element information from the target voice signal; an instruction generation module for arranging and combining instruction elements in a pre-stored instruction element table to generate multiple instructions; an instruction filtering module for filtering instructions that do not conform to preset language habit rules from the multiple instructions to obtain remaining instructions; and an execution instruction matching module for selecting the instruction with the highest degree of matching from the remaining instructions based on the extracted instruction element information as the execution instruction.

[0014] According to the third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method described in the first aspect or any optional embodiment of the first aspect by executing the computer instructions.

[0015] According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in the first aspect or any optional embodiment of the first aspect.

[0016] The technical solution provided by this application has the following advantages:

[0017] The technical solution provided by this application pre-stores a table of command elements on the end-side device, including control actions, locations, device ranges, and device names. The number of command entries generated by permutations and combinations of command elements is far greater than that of pre-stored fixed command entries, and the command scenarios involved are more diverse. If the command entries generated by the permutations and combinations can be accurately recognized, the difficulty of users memorizing a large number of command entries can be greatly reduced. The end-side device also does not need to store a large number of fixed command entries, and only needs to permutate and combine the command words during recognition. On this basis, simply ensuring that the user's spoken command accurately matches the command entries generated by the permutations and combinations can solve the problems in the prior art. Therefore, the embodiments of the present invention perform command element recognition on the collected voice signal to accurately identify the command elements in the voice signal, including but not limited to words, characters, and phrases. The identified command elements and their order are then matched with multiple instructions generated by the permutations and combinations. The instruction with the highest degree of matching is then selected as the execution instruction, thereby achieving accurate execution of the instruction. This reduces the storage volume of command entries on the end-side device while ensuring the accuracy of command recognition.

[0018] In addition, in an embodiment of the present invention, the high-dimensional acoustic features of the speech signal are used to identify the command elements through two neural networks with different receptive fields, and then the two recognition results are fused, thereby simultaneously using the global information and local information of the high-dimensional acoustic features, further suppressing the misrecognition between different command elements, thereby improving the accuracy of command element information recognition.

[0019] In addition, in an embodiment of the present invention, the more accurate instruction element information obtained by the above fusion step is input into the instruction decoding graph to match the execution instructions, thereby further improving the accuracy of the execution instruction matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0021] Figure 1 A schematic diagram showing the steps of a voice command recognition method according to one embodiment of the present invention is shown;

[0022] Figure 2 A flow chart of a method for voice command recognition in one embodiment of the present invention is shown;

[0023] Figure 3 Another flow chart of a method for voice command recognition according to one embodiment of the present invention is shown;

[0024] Figure 4A schematic diagram of the structure of a feature extraction neural network in one embodiment of the present invention is shown;

[0025] Figure 5 A schematic diagram of the structure of a first neural network in one embodiment of the present invention is shown;

[0026] Figure 6 A schematic diagram of the structure of a decoding graph in one embodiment of the present invention is shown;

[0027] Figure 7 A schematic structural diagram of a voice command recognition device according to one embodiment of the present invention is shown;

[0028] Figure 8 A schematic structural diagram of an electronic device in one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0029] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0030] See also Figure 1 and Figure 2 In one embodiment, a method for voice command recognition includes the following steps:

[0031] Step S101: collecting a target voice signal and extracting instruction element information from the target voice signal.

[0032] Step S102: Arrange and combine the instruction elements in the pre-stored instruction element table to generate multiple instructions.

[0033] Step S103: filtering instructions that do not conform to the preset language convention rules among the multiple instructions to obtain remaining instructions.

[0034] Step S104: Based on the extracted instruction element information, the instruction with the highest matching degree is selected from the remaining instructions as the execution instruction.

[0035] Specifically, embodiments of the present invention pre-store a table of command elements on the end-side device. The table includes control actions, locations, device ranges, device names, and other information. Command elements include, but are not limited to, words, characters, and phrases. By permuting and combining command elements, the number of command entries generated is far greater than that of pre-stored fixed command entries, and the command scenarios involved are more diverse. If the command entries generated by the permutation and combination can be accurately recognized, the difficulty of users memorizing a large number of command entries can be greatly reduced. The end-side device also does not need to store a large number of fixed command entries, and only needs to permutate and combine the command words during recognition. On this basis, simply ensuring that the user's spoken command accurately matches the command entry generated by the permutation and combination can solve the problems in the prior art. Therefore, embodiments of the present invention perform command element recognition on the collected voice signal to accurately identify the command elements in the voice signal. The identified command elements and their order are then matched with multiple instructions generated by the permutation and combination. Matching methods include, but are not limited to, decoders and similarity. The instruction with the highest matching degree is then selected from the multiple instructions generated by the permutation and combination as the execution instruction, thereby achieving accurate execution of the instruction. This reduces the amount of command entries stored on the device while ensuring command recognition accuracy. Before matching multiple commands, commands that do not conform to the language rules are filtered according to pre-set language conventions, such as "Turn on the desk lamp in the bedroom." This prevents users from speaking confusing commands that could cause confusion in command execution, while also helping users develop good command-speaking habits.

[0036] For example, existing smart home control systems include products such as Voice Treasure, which require the collection of a large number of command words. The number of command words that the trained model can support is relatively limited, and the command words must be fully matched to be recognized. Users cannot speak freely according to actual needs. For example, the user's vocabulary is "turn on the study light", "turn off the study light", "turn on the dining room fan", and "turn off the dining room fan". If the user wants to operate the study fan by commanding "turn on the study fan", this is not feasible. The embodiment of the present invention proposes a solution for recognizing voice commands as command elements such as "control actions", "locations", "device ranges", and "devices". The output of the network model in the embodiment of the present invention is a series of "control actions" such as "turn on" and "turn off", a series of "locations" such as "study" and "dining room", a series of "device ranges" such as "all" and "all", a series of "device names" such as "lights" and "fans", and then the decoder parses the recognized command elements into specific control commands, so that users can easily achieve free control in smart home scenarios.

[0037] Specifically, in one embodiment, the step of generating the pre-stored instruction element table includes:

[0038] 1. Disassemble the instruction elements from the collected control instructions and label the type of the disassembled instruction elements.

[0039] 2. Count the high-frequency instruction elements of each type whose occurrence frequency is greater than the preset frequency threshold.

[0040] 3. Summarize the high-frequency instruction elements of each type into the pre-stored instruction element table.

[0041] Specifically, in this embodiment, the command elements within a large number of pre-collected control instructions are broken down and then labeled by type, such as which instruction elements are control actions and which are positional. For each type, the most frequently occurring command elements are then counted and compiled into a command element table. This ensures that the command elements in the command element table cover all types and remove redundant command elements, making the generated command element table more accurate.

[0042] Specifically, in one embodiment, Figure 3 As shown, the above step S101 specifically includes the following steps:

[0043] Step 1: Extract high-dimensional acoustic features of the target speech signal.

[0044] Step 2: Use two neural network models with different receptive fields to identify the command elements of the high-dimensional acoustic features respectively, and obtain two recognition results. The two recognition results are used to represent the probability of each command element contained in the high-dimensional acoustic features.

[0045] Step 3: Fuse the two recognition results to obtain instruction element information.

[0046] Specifically, the embodiment of the present invention realizes the recognition of the instruction elements of the target voice signal by deploying a multi-task fusion neural network model. First, the target voice signal input by the user is received, and the signal contains human voice and non-human voice background noise, and the received voice signal is pre-processed by pre-emphasis, framing, windowing and other pre-processing operations, and then the framed time domain voice signal is Fourier transformed frame by frame to convert it into a frequency domain signal; then the frequency domain signal is Mel-filtered to obtain Mel features that are more in line with the human voice mechanism and used as voice features; then the Mel features are input into the feature extraction neural network to extract high-dimensional acoustic features in the voice features. Among them, the feature extraction neural network can be composed of one or more combinations of convolutional layers, long-short time memory layers, fully connected layers, etc., including but not limited to CDNN, TDNN, TDNN-F, cascaded LSTM, etc.; in the embodiment of the present invention, Figure 4As shown, a recognition network with a TDNN-F structure is used as the feature extraction neural network. In actual use, the network structure is not limited. During feature extraction neural network training, a TDNN-F layer is added to map the network output to the first posterior probability information. Mel features are input into the constructed feature extraction neural network. Phonemes, syllables (pinyin), or characters are used as labels for the network output. A loss function is calculated between the predicted posterior probability and the label. The loss function can be selected from cross entropy, CTC, etc., and backpropagation training is performed. After model training converges, the output of the feature extraction neural network is used as the high-dimensional acoustic feature.

[0047] Afterwards, the embodiment of the present invention retrains two neural network models with different receptive fields, inputting high-dimensional acoustic features into the two neural networks with different receptive fields, thereby outputting different recognition results, representing the posterior probability of recognizing the command element. The neural network with a smaller receptive field focuses on the local information of the signal, while the neural network with a larger receptive field focuses on the global information of the signal. By extracting both local and global information from the high-dimensional acoustic features, the fusion of these two types of information further suppresses misidentification of different command elements, thereby improving the accuracy of command element information recognition.

[0048] Specifically, in one embodiment, the above step 2 specifically includes the following steps:

[0049] Step 4: Identify the instruction elements in the high-dimensional acoustic features through the first neural network model to obtain a first recognition result.

[0050] Step 5: Extract the first preset number of frames from the high-dimensional acoustic feature to remove redundant parts from the high-dimensional acoustic feature to obtain a second high-dimensional acoustic feature.

[0051] Step 6: Identify the instruction element in the second high-dimensional acoustic feature through the second neural network to obtain a second recognition result; wherein the receptive field of the first neural network is smaller than the receptive field of the second neural network.

[0052] Specifically, in the embodiment of the present invention, a first neural network is first constructed to determine the posterior probability of control elements including "control action", "position", "device range" and "device" corresponding to the high-dimensional acoustic features, and output a first recognition result; the first neural network can be composed of one or more combinations of convolutional layers, long and short time memory layers, and fully connected layers, so as to pay more attention to the local information in the speech signal. In the embodiment of the present invention, if Figure 5As shown, a recognition network composed of a multi-layer CNN, LSTM, and DNN structure is used as the first neural network. In actual use, the network structure is not limited. High-dimensional acoustic features are input into the constructed first neural network, and instruction elements are used as labels for the network output. A loss function (cross entropy, CTC, etc.) is calculated between the prediction result and the label. Backpropagation training is performed. After the model training converges, the posterior probability output by the trained neural network is used as the first recognition result.

[0053] Then build a second neural network. In an embodiment of the present invention, the second neural network adopts a recognition network composed of a Transformer structure based on an attention mechanism. The network has a larger receptive field and pays more attention to global information. This is just an example and is not limited to this. The advantage of extracting global information by the Transformer model based on the attention mechanism is combined with the advantage of local information extracted by the first neural network. At the same time, global information and local information are used for fusion, so as to re-score the first recognition result through the second recognition result, and further suppress the misidentification between different instruction elements. When training the second neural network model, the instruction element is used as the label of the network output, and the loss function between the prediction result and the label is calculated. The loss function can choose cross entropy, etc., and perform back propagation training; after the model training converges, the posterior probability output by the second neural network is used as the second recognition result.

[0054] In this embodiment, before the high-dimensional acoustic features are fed into the second neural network, they are dimensionalized using a first buffer. This buffer only captures a first preset number of frames to remove redundancy. For example, the voice signal collected by a smart speaker is approximately 30 seconds long, but only 10 seconds of it is actually useful. Therefore, a preset number of frames of high-dimensional acoustic features are selected for global processing to further improve the accuracy of command element recognition.

[0055] Specifically, in one embodiment, when training the first neural network, in addition to the instruction element labels, the instruction start frame and the instruction end frame are also used as training samples for training. Therefore, in the first recognition result, in addition to the posterior probability of the instruction element, the probability of the instruction start frame and the probability of the instruction end frame are also included. Among them, the embodiment of the present invention defines that the instruction start frame SOI (Start Of Instruction) and the instruction end frame EOI (End Of Instruction) occupy N voice frames (a second preset number of frames) respectively, and the specific number of N can be adjusted according to actual conditions. SOI is the N voice frames before the start of the voice instruction; EOI is the N voice frames after the end of the voice instruction, so that the first neural network is trained based on the instruction start frame, instruction element, and instruction end frame, which can more accurately locate and identify the voice signal containing the real user instruction, further improving the accuracy of subsequent instruction element recognition.

[0056] Specifically, in one embodiment, based on steps 4 to 6 above, step 3 above specifically includes the following steps:

[0057] Step 7: Extracting sub-recognition results corresponding to a first preset number of frames from the first recognition result.

[0058] Step 8: Determine whether the probability of the instruction end frame is greater than a preset threshold.

[0059] Step 9: If the probability of the instruction end frame is greater than the preset threshold, the sub-recognition result and the second recognition result are weighted multiplied to obtain the instruction element information.

[0060] Specifically, in this embodiment, before performing recognition result fusion, the second buffer is first used to extract the sub-recognition results corresponding to the first preset number of frames in the first recognition result, that is, the posterior probability of the first preset number of frames, thereby adjusting the dimension of the first recognition result to match the matrix dimension of the second recognition result, facilitating matrix multiplication. It is then determined whether the posterior probability of the instruction end frame is greater than a preset threshold. If so, it is considered that a complete sentence has been recognized, the information in the first and second buffers is valid, and the fusion operation is valid. Otherwise, the voice command recognition is abandoned to improve the efficiency and accuracy of voice command recognition.

[0061] Afterwards, when fusing the recognition results, the two recognition results are weighted and multiplied together, which is equivalent to re-scoring the local information with global information, thereby generating instruction element information with higher accuracy, providing a guarantee for improving the accuracy of subsequent instruction matching.

[0062] Let me explain this with a specific example:

[0063] For example, the input high-dimensional acoustic feature of the first neural network is X_n2, the dimension of X_n2 is B*T*F, and B is the batch size (Batch Size), T is the total number of frames of input speech, F is the dimension of high-dimensional acoustic features, the output of the first neural network is Y_n2, with a dimension of B*T*C, C is the number of categories, including the total number of categories such as instruction elements, SOI and EOI; after the high-dimensional acoustic features pass through the first buffer, only the high-dimensional acoustic features of Tc frames are retained, recorded as X_n2_c, and the dimension becomes B*Tc*F; after X_n2_c is input into the second neural network, the output is Y_n3, with a dimension of B*Tc*C, where Y_n3 is a weighted matrix; then Y_n2 passes through the second buffer, and only the posterior probability of Tc frames is retained, recorded as Y_n2_c, and the dimension becomes B*Tc*C; since Y_n3 and Y_n2_c have the same dimension, they can be directly multiplied to obtain Y_n2_resore=Y_n2_c.*Y_n3, where Y_n2_resore is the instruction element information obtained by re-scoring the first recognition result with the second recognition result.

[0064] Specifically, if Figure 6 As shown, in one embodiment, the above step S104 specifically includes the following steps:

[0065] Step 10: Generate a decoding graph based on the remaining instructions.

[0066] Step 11: Fill the instruction element probability contained in the instruction element information into the corresponding instruction element node in the decoding graph.

[0067] Step 12: Find the optimal path from the filled decoding graph, and use the instruction corresponding to the optimal path as the execution instruction.

[0068] Specifically, in this embodiment, the decoder module determines the execution instruction corresponding to the speech signal based on the instruction element information and the decoding graph constructed by all possible instruction element combinations. The instruction element probability of each instruction element in the instruction element information is filled into the corresponding instruction element node in the decoding graph (that is, the instruction element probability is filled into the corresponding instruction element node according to the instruction element). Figure 6 Then, the probability of the instruction element node and the cost of each path are scored to find the optimal path, and the instruction corresponding to the optimal path is determined as the execution instruction.

[0069] In this embodiment, the specific steps of searching for the optimal path are:

[0070] 1. Invert the instruction element probabilities of the instruction element nodes in the decoding graph to obtain the cost value of each instruction element node.

[0071] 2. Calculate the cumulative cost value corresponding to each path in the decoding graph based on the pre-generated transfer arc cost value and the cost value of the instruction element node in the decoding graph.

[0072] 3. The path with the smallest cumulative cost is taken as the optimal path.

[0073] Specifically, after the voice features pass through the neural network model, the instruction element information is obtained, and the posterior probability of each instruction element in the instruction element information is inverted to obtain the instruction element cost. The larger the posterior probability, the smaller the instruction element cost; obtain all possible voice instruction words pre-stored on the terminal device in a certain application scenario, and mark the instruction element type for the voice instruction word, count the frequently appearing instruction elements by type, form an instruction element table, and arrange and combine the instruction elements, and filter out the combinations that do not conform to daily usage habits, and map them into corresponding instruction element sequences. Then, all voice instruction elements and their corresponding instruction element sequences are input into the WFST tool (WFST is a commonly used tool in speech recognition) to generate a decoding graph. The decoding graph contains state nodes ( Figure 6 The circle in the middle, i.e. the instruction element node), the emission transfer arc between the state nodes (such as Figure 6 solid arrows) or non-emitting transfer arcs (such as Figure 6 As shown by the dashed arrows in the middle), and the decoding graph cost corresponding to the transmitting transfer arc or the non-transmitting transfer arc (i.e. Figure 6 The solid line and the dotted line generate the equivalent value. Figure 6(not marked in the figure), the cost of the instruction element needs to be filled in the corresponding state node. In this embodiment, the cumulative cost of the transmitted transfer arc = the cumulative cost of the previous state node + the instruction element cost of the current frame + the decoding graph cost on the current transfer arc; the cumulative cost of the non-transmitted transfer arc = the cumulative cost of the previous state node + the decoding graph cost on the current transfer arc. The state nodes in the decoding path are sequentially subjected to the above-mentioned calculation formula for the cumulative cost of the transmitted transfer arc or the calculation formula for the cumulative cost of the non-transmitted transfer arc until the cumulative cost corresponding to the last state node is calculated and generated, that is, the cumulative cost corresponding to the decoding path. The selection of the decoding path is determined according to the cumulative cost of the decoding path. The smaller the cumulative cost of the decoding path, the better the decoding path, so the cumulative costs corresponding to the decoding paths are arranged from small to large, and the decoding path with the smallest cumulative cost is selected as the final result (that is, the corresponding voice instruction). The embodiment of the present invention combines the instruction element information obtained by the fusion model with the decoding graph to obtain effective and accurate execution instructions. It supports flexible command element combinations, enabling free control of multiple actions, multiple locations, and multiple devices in smart home scenarios. Its simple grammar conforms to natural expression habits, reducing the burden of memorizing numerous command words and effectively improving the user experience. Furthermore, the network model's parameters and computing power requirements are significantly reduced, allowing it to run on low-computing, low-storage end-side smart devices. Furthermore, the devices operate offline, without internet access, effectively protecting user privacy.

[0074] Taking the whole-house lighting control scenario in a smart home as an example, the technical solution of this application is introduced:

[0075] Suppose a user wants to turn on the living room ceiling light. According to existing command word solutions, the user needs to fully match the voice command "Turn on the living room ceiling light" for the device to recognize it. This solution doesn't support user-modified voice commands (this includes changing the order of command elements, combining command elements according to different rules, etc.), such as "Turn on the living room ceiling light," "Turn on the ceiling light," and so on. Our solution breaks down voice commands into command elements such as "control action," "location," "device range," and "device." The order of command elements can be freely combined, but at least "control action" and "device" are required. Control actions include but are not limited to those listed in Table 1, locations include but are not limited to those listed in Table 2, device ranges include but are not limited to those listed in Table 3, and devices include but are not limited to those listed in Table 4.

[0076] Table 1 Control action list

[0077]

[0078] Table 2 Location list

[0079]

[0080]

[0081] Table 3 Equipment range list

[0082]

[0083] Table 4 Equipment List

[0084]

[0085] In the new technology solution, the system recognizes not the entire command word, but the command elements such as "control action", "location", "device range", and "device" contained in the voice command, and at least "control action" and "device" must be included. Other command elements can be empty. For example, the user's voice command is "Turn on the living room ceiling light", and the network recognition is: location = living room, device = ceiling light, control action = turn on, device range = empty, and the recognition results are given in the order in which the user gives the voice command. After all the voice commands are completed, the decoder module searches for the execution instruction on the pre-configured decoding map. Here, the final recognition is "turn on" the "ceiling light" located in the "living room".

[0086] Through the above steps, the technical solution provided by this application pre-stores a table of command elements on the end-side device, including control actions, locations, device ranges, and device names. The number of command entries generated by permutations and combinations of command elements is far greater than that of pre-stored fixed command entries, and the command scenarios involved are more diverse. If the command entries generated by the permutations and combinations can be accurately recognized, the difficulty of users memorizing a large number of command entries can be greatly reduced. The end-side device also does not need to store a large number of fixed command entries, and only needs to permutate and combine the command words during recognition. On this basis, simply ensuring that the user's spoken command accurately matches the command entries generated by the permutations and combinations can solve the problems of the prior art. Therefore, the embodiments of the present invention perform command element recognition on the collected voice signal to accurately identify the command elements in the voice signal, including but not limited to words, characters, and phrases. The identified command elements and their order are then matched with multiple instructions generated by the permutations and combinations. The instruction with the highest degree of match is then selected as the execution instruction, thereby achieving accurate execution of the instruction. This reduces the storage volume of command entries on the end-side device while ensuring the accuracy of command recognition.

[0087] In addition, in an embodiment of the present invention, the high-dimensional acoustic features of the speech signal are used to identify the command elements through two neural networks with different receptive fields, and then the two recognition results are fused, thereby simultaneously using the global information and local information of the high-dimensional acoustic features, further suppressing the misrecognition between different command elements, thereby improving the accuracy of command element information recognition.

[0088] In addition, in an embodiment of the present invention, the more accurate instruction element information obtained by the above fusion step is input into the instruction decoding graph to match the execution instructions, thereby further improving the accuracy of the execution instruction matching.

[0089] like Figure 7 As shown, this embodiment also provides a voice command recognition device, which includes:

[0090] The instruction element extraction module 101 is used to collect the target speech signal and extract instruction element information from the target speech signal. For details, please refer to the relevant description of step S101 in the above method embodiment, which will not be repeated here.

[0091] Instruction generation module 102 is configured to generate multiple instructions by permuting and combining instruction elements in a pre-stored instruction element table. The instruction element table is generated from control instructions whose occurrence frequency exceeds a preset frequency threshold among the collected control instructions. For details, see the description of step S102 in the above method embodiment and will not be repeated here.

[0092] The instruction filtering module 103 is used to filter instructions that do not conform to the preset language convention rules from the multiple instructions to obtain the remaining instructions. For details, please refer to the relevant description of step S103 in the above method embodiment, which will not be repeated here.

[0093] The execution instruction matching module 104 is used to select the instruction with the highest matching degree from the remaining instructions based on the extracted instruction element information as the execution instruction. For details, please refer to the relevant description of step S104 in the above method embodiment, which will not be repeated here.

[0094] The voice command recognition device provided in the embodiment of the present invention is used to execute the voice command recognition method provided in the above embodiment. Its implementation method and principle are the same. For details, please refer to the relevant description of the above method embodiment and will not be repeated here.

[0095] Through the above-mentioned components, the technical solution provided by this application pre-stores a table of command elements on the end-side device, including control actions, locations, device ranges, device names, and so on. The number of command entries generated by permutations and combinations of command elements is far greater than that of pre-stored fixed command entries, and the command scenarios involved are more diverse. If the command entries generated by the permutations and combinations can be accurately recognized, the difficulty of users memorizing a large number of command entries can be greatly reduced. The end-side device also does not need to store a large number of fixed command entries, and only needs to permutate and combine the command words during recognition. On this basis, simply ensuring that the user's spoken command accurately matches the command entries generated by the permutations and combinations can solve the problems in the prior art. Therefore, the embodiments of the present invention perform command element recognition on the collected voice signal to accurately identify the command elements in the voice signal, including but not limited to words, characters, and phrases. The identified command elements and their order are then matched with multiple instructions generated by the permutations and combinations. The instruction with the highest degree of match is then selected as the execution instruction, thereby achieving accurate execution of the instruction. This reduces the storage volume of command entries on the end-side device while ensuring the accuracy of command recognition.

[0096] In addition, in an embodiment of the present invention, the high-dimensional acoustic features of the speech signal are used to identify the command elements through two neural networks with different receptive fields, and then the two recognition results are fused, thereby simultaneously using the global information and local information of the high-dimensional acoustic features, further suppressing the misrecognition between different command elements, thereby improving the accuracy of command element information recognition.

[0097] In addition, in an embodiment of the present invention, the more accurate instruction element information obtained by the above fusion step is input into the instruction decoding graph to match the execution instructions, thereby further improving the accuracy of the execution instruction matching.

[0098] Figure 8 An electronic device according to an embodiment of the present invention is shown, which includes a processor 901 and a memory 902, which can be connected via a bus or other means. Figure 8 The bus connection is taken as an example.

[0099] The processor 901 may be a central processing unit (CPU). The processor 901 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0100] Memory 902, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the program instructions / modules corresponding to the methods in the above-described method embodiments. Processor 901 executes the non-transitory software programs, instructions, and modules stored in memory 902 to perform various processor functions and data processing, thereby implementing the methods in the above-described method embodiments.

[0101] The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created by the processor 901, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 902 may optionally include a memory remotely located relative to the processor 901, and these remote memories may be connected to the processor 901 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0102] One or more modules are stored in the memory 902 and, when executed by the processor 901 , perform the method in the above method embodiment.

[0103] The specific details of the above electronic device can be understood by referring to the corresponding descriptions and effects in the above method embodiments, and will not be repeated here.

[0104] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing related hardware through a computer program. The implemented program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-described embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). The storage medium can also include a combination of the above-mentioned types of memory.

[0105] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for voice command recognition, characterized in that: The method comprises: A target speech signal is collected, and instruction element information is extracted from the target speech signal; the extracting instruction element information from the target speech signal includes: extracting high-dimensional acoustic features of the target speech signal; using two neural network models with different receptive fields to perform instruction element recognition on the high-dimensional acoustic features, respectively, to obtain two recognition results, the two recognition results being used to characterize the probability of each instruction element contained in the high-dimensional acoustic features; and fusing the two recognition results to obtain the instruction element information; Arrange and combine the instruction elements in the pre-stored instruction element table to generate multiple instructions; Filtering instructions that do not conform to preset language convention rules from the multiple instructions to obtain remaining instructions; generating a decoding graph based on the remaining instructions; Filling the instruction element probabilities contained in the instruction element information into the corresponding instruction element nodes in the decoding graph; Find the optimal path from the filled decoding graph and use the instructions corresponding to the optimal path as execution instructions.

2. The method according to claim 1, characterized in that The finding of the optimal path from the padded decoding graph includes: Inverting the instruction element probabilities of the instruction element nodes in the decoding graph to obtain a cost value of each instruction element node; Calculating the cumulative cost value corresponding to each path in the decoding graph based on the pre-generated transfer arc cost value and the cost value of the instruction element node in the decoding graph; The path with the smallest cumulative cost is taken as the optimal path.

3. The method according to claim 1, characterized in that The two neural network models with different receptive fields are used to respectively perform instruction element recognition on the high-dimensional acoustic features to obtain two recognition results, including: Recognizing the instruction element in the high-dimensional acoustic feature by using a first neural network model to obtain a first recognition result; Extracting a portion of a first preset number of frames from the high-dimensional acoustic feature to remove redundant portions of the high-dimensional acoustic feature, thereby obtaining a second high-dimensional acoustic feature; Recognizing the instruction element in the second high-dimensional acoustic feature through a second neural network to obtain a second recognition result; The receptive field of the first neural network is smaller than the receptive field of the second neural network.

4. The method according to claim 3, characterized in that The first recognition result includes the probability of an instruction start frame, the probabilities of several instruction elements and the probability of an instruction end frame, wherein the instruction start frame and the instruction end frame respectively represent the second preset number of speech frames before the instruction starts and after the instruction ends in the high-dimensional acoustic features.

5. The method according to claim 4, characterized in that The fusing of the two recognition results to obtain the instruction element information includes: Extracting sub-recognition results corresponding to the first preset number of frames from the first recognition result; Determining whether the probability of the instruction end frame is greater than a preset threshold; If the probability of the instruction end frame is greater than a preset threshold, the sub-recognition result and the second recognition result are weighted multiplied to obtain the instruction element information.

6. The method according to claim 1, characterized in that The step of generating the pre-stored instruction element table includes: Decompose instruction elements from the collected control instructions and label the decomposed instruction elements with their types; Count the high-frequency instruction elements of each type whose occurrence frequency is greater than a preset frequency threshold; The high-frequency instruction elements in each type are summarized into the pre-stored instruction element table.

7. A voice command recognition device, characterized in that: The device comprises: The instruction element extraction module is configured to collect a target speech signal and extract instruction element information from the target speech signal; the extraction of instruction element information from the target speech signal includes: extracting high-dimensional acoustic features of the target speech signal; performing instruction element recognition on the high-dimensional acoustic features using two neural network models with different receptive fields, respectively, to obtain two recognition results, the two recognition results being used to characterize the probability of each instruction element contained in the high-dimensional acoustic features; and fusing the two recognition results to obtain the instruction element information. An instruction generation module is used to arrange and combine instruction elements in a pre-stored instruction element table to generate multiple instructions; An instruction filtering module, configured to filter instructions that do not conform to preset language convention rules from the plurality of instructions to obtain remaining instructions; An execution instruction matching module is used to generate a decoding graph based on the remaining instructions; fill the instruction element probabilities contained in the instruction element information into the corresponding instruction element nodes in the decoding graph; find the optimal path from the filled decoding graph, and use the instruction corresponding to the optimal path as the execution instruction.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 6 by executing the computer instructions.

Images