Command word recognition method, electronic device, and storage medium

Abstract

The application discloses a command word recognition method, an electronic device and a storage medium, and the method comprises the following steps: acquiring audio input by a user, decoding the audio to obtain a decoding result; if the decoding result matches a certain command word in a command word table, checking whether other command words in the command word table are prefix and parent strings of a string of the certain command word; if the command word table contains other command words which are prefix and parent strings of the string of the certain command word, performing subsequent decoding according to a decoding configuration corresponding to the prefix and parent strings. According to the application, whether the command word table contains prefix and parent strings is judged, different decoding configurations are used in the decoder if the command word table contains the prefix and parent strings, the errors of parent strings and child strings can be reduced, and the calculation resources are not increased.

Description

Technical Field

[0001] This invention belongs to the field of command word recognition technology, and particularly relates to a command word recognition method, electronic device, and storage medium. Background Technology

[0002] As more and more edge devices use voice command recognition, the demands on this technology are also increasing. Due to cost and technological limitations, edge devices often lack sufficient computing resources. Furthermore, the command words used in smart home, control panel, and IoT scenarios contain many easily confused words. Improving the recognition accuracy of these easily confused words is crucial for user experience. Existing technologies often require significant computing resources, but low-cost chips often cannot meet these requirements. Additionally, the unique characteristics of easily confused words prevent the achievement of optimal results.

[0003] Existing techniques for identifying easily confused words often employ a two-stage processing module. The result from the first identification module is then fed into the second identification module, improving the accuracy of command word recognition through this two-stage process. The first identification module generates command word confidence scores based on the audio sequence input. Easily confused words are then identified based on these confidence scores and fed into the second identification module to further differentiate command words and mitigate misidentification. However, adding a second module to the calculations of the first identification module inevitably increases computational power. If the second identification module has a high real-time rate, this can lead to significant computational latency and data congestion, which is unacceptable in a real-time system. Conversely, a low real-time rate requires a chip with high computing power to support the two-stage module, a requirement that is often demanding for low-cost chips.

[0004] The inventors discovered that existing similar technologies require two-stage processing modules, which increases the consumption of computing resources; when applied to real-time systems, the second recognition module is required to have a low real-time rate, which is a demanding condition for low-cost chips, and no optimization for easily confused words was involved when adjusting the modules. Summary of the Invention

[0005] The embodiments of the present invention are intended to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, embodiments of the present invention provide a command word recognition method, comprising: acquiring audio input by a user; decoding the audio to obtain a decoding result; if the decoding result matches a command word in a command word table, searching the command word table for other command words whose strings are prefixes of the command word, wherein strings with the same prefix characters are prefixes of each other; if the command word table contains other command words whose strings are prefixes of the command word, then performing subsequent decoding according to the decoding configuration corresponding to the prefixes of the command word; if the command word table does not contain other command words whose strings are prefixes of the command word, then continuing to perform subsequent decoding using the current decoding configuration.

[0007] Secondly, embodiments of the present invention provide an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the command word recognition methods described above.

[0008] Thirdly, embodiments of the present invention provide a storage medium storing one or more programs including execution instructions, the execution instructions being readable and executable by electronic devices (including but not limited to computers, servers, or network devices, etc.) to perform any of the command word recognition methods described above.

[0009] Fourthly, embodiments of the present invention also provide a computer program product, the computer program product including a computer program stored on a storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to execute any of the above-described command word recognition methods.

[0010] This invention reduces errors in parent-child strings by determining whether the command word lexicon contains a prefix parent string. If the parent string is present, a different decoding configuration is used in the decoder, which does not increase computational resources. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of an embodiment of the command word recognition method of the present invention; Figure 2 This is a flowchart illustrating the process of implementing the command word recognition method of the present invention. Figure 3 This is a schematic diagram of the structure of an embodiment of the electronic device of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0015] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, elements, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0016] In this invention, terms such as "module," "device," and "system" refer to relevant entities applied to a computer, such as hardware, combinations of hardware and software, software, or software in execution. More specifically, for example, an element can be, but is not limited to, a process running on a processor, a processor, an object, an executable element, an execution thread, a program, and / or a computer. Furthermore, an application program or script running on a server, and the server itself, can also be an element. One or more elements may be in an execution process and / or thread, and elements may be localized on a single computer and / or distributed across two or more computers, and may be run on various computer-readable media. Elements can also communicate via local and / or remote processes based on signals having one or more data packets, for example, signals from data interacting with another element in a local system, a distributed system, and / or interacting with other systems via signals over a network on the Internet.

[0017] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0018] This invention provides a command word recognition method that can be applied to electronic devices. The electronic device can be a computer, server, or other electronic product, and this invention does not limit this to any particular device.

[0019] Please refer to Figure 1 This illustrates a command word recognition method provided by an embodiment of the present invention.

[0020] like Figure 1 As shown, in step 101, the audio input by the user is obtained, and the audio is decoded to obtain the decoding result; In step 102, if the decoding result matches a command word in the command word list, the system searches the command word list to see if there are other command words in the list that are prefixed to the string of the command word. Strings with the same prefix characters are prefixed to each other. In step 103, if there are other command words in the command word list that are prefixes of the strings of a certain command word, then subsequent decoding is performed according to the decoding configuration corresponding to the prefixes. In step 104, if there are no other command words in the command word list that are prefixes of the parent and child strings of a certain command word, then the current decoding configuration will continue to be used for subsequent decoding.

[0021] In this embodiment, for step 101, the audio information input by the user is obtained by using a microphone or other device to pick up the audio information input by the user, and the audio information input by the user is decoded by a decoder. For example, if the audio information input by the user is "help me turn on the air conditioner", the decoder decodes each word in the audio input by the user to obtain the decoding result "help me turn on the air conditioner".

[0022] Next, for step 102, based on the decoding result obtained from the user-input audio, the command words in the decoding result are searched, whereby the command words in the decoding result are simply referred to as decoding command words. Then, the command words corresponding to the decoding command words are searched or matched in the prepared command word list. If a command word corresponding to the decoding command word is matched in the command word list, the search continues in the command word list for other command words whose strings are prefixes of each other. Strings with the same prefix characters are prefixes of each other. The search continues in the command word list for other command words whose prefix characters are the same as the corresponding command word. A certain command word in the command word list is the command word in the decoding result that matches the command word in the command word list. For example, if the decoding command word is "turn on the air conditioner", then the command word list is matched for the command word corresponding to "turn on the air conditioner". If the corresponding command word is "turn on the air conditioner", then the search continues in the command word list for other command words with the same prefix characters as the command word "turn on the air conditioner", such as "turn on the TV" or "turn on the light".

[0023] Then, for step 103, if another command word with the same prefix character as a given command word is found in the command word list, decoding is performed according to the decoding configuration corresponding to the string with the same prefix character. For example, if the command word list contains two command words, "open" and "turn on the air conditioner", and the system detects that they are parent and child strings with prefixes to each other, when the decoder decodes to the node "open", it will wait for a longer time than the default decoder setting. If the node "empty" is not encountered, "open" is output; if the node "empty" is encountered, decoding continues until "turn on the air conditioner" is decoded.

[0024] Finally, for step 104, if no other command word with the same prefix character as a given command word is found in the command word list, the decoding configuration of the current command word continues to be used for decoding. If the command word does not have a prefix substring, such as the command word "enable" in the command word list, the decoder waits for the time set by the default decoder after decoding the node "enable" before outputting the decoding result.

[0025] This invention reduces errors in parent-child strings by determining whether the command word lexicon contains a prefix parent string. If the parent string is present, a different decoding configuration is used in the decoder, which does not increase computational resources.

[0026] In some optional embodiments, among multiple strings that are prefixes of each other, the shorter string is the substring, and the longest string among these prefixes is the parent string. For example, in "turn on the air conditioner," "turn on" is the substring, and "turn on the air conditioner" is the parent string. When the command word in the decoding result matches the command word in the command word table, and the parent string is the command word, a short-delay decoding configuration is used; when the command word in the decoding result matches the command word in the command word table, and the substring is the command word, a long-delay decoding configuration is used. Short delay refers to the default decoding time, typically tens of milliseconds, while long delay refers to a longer time, typically hundreds of milliseconds. "Long" and "short" can be understood as slow and fast output decoding results; using "long delay" results in a slower output decoding result, and using "short delay" results in a faster output decoding result.

[0027] In some optional embodiments, the recognition module is trained using contrastive learning. This contrastive learning method compares the decoding results with target and obfuscated sequences. Target and obfuscated sequences are similar command words obtained from the user's command vocabulary that match the user's input. After training, the trained recognition module is used to recognize and decode the user's audio input, obtaining the user's audio decoding result. For easily confused words, the contrastive learning of target and obfuscated sequences continuously reduces the distance between the recognition module's output and the target sequence while increasing the distance with the obfuscated sequences, improving the discriminative power of the recognition module and optimizing the recognition results. The target sequence refers to the expected correct sequence result, while the obfuscated sequence refers to an incorrect sequence result that is similar to the correct result but is not intended for output. For example, in the case of "open" and "click," if "open" is the target sequence, then "click" is the obfuscated sequence. When the expected recognition is of "open," it may easily be recognized as "click," causing an error.

[0028] It should be noted that the recognition module in this application, during training, not only learns to reduce errors compared to the target sequence, but also learns to distinguish easily confused words, increasing the differentiation from easily confused sequences. This comparative learning enhances the recognition module's ability to distinguish easily confused words. In the decoder of the recognition module, different configurations are set according to the characteristics of easily confused words. Different decoding delays are used for some parent and child strings; for example, different delays are used for the commands "open" and "turn on the air conditioner," effectively reducing crosstalk between the two. This method uses a single decoding module with different configurations, resulting in almost no increase in computational resource consumption.

[0029] In some optional embodiments, speech synthesis technology is used to synthesize speech data corresponding to the command vocabulary, and parallel computing technology is used to accelerate the processing of this speech data. To reduce costs, speech synthesis technology is used to synthesize audio of command words and audio of easily confused words. To improve efficiency, parallel computing technology is used to accelerate the audio synthesis process. To improve model robustness, data simulation is performed on the audio to increase data diversity. When the cost of acquiring audio data of easily confused words is high, speech synthesis technology is used to generate corresponding audio data, and it can synthesize not only data within the command vocabulary but also data outside the vocabulary that is similar to the corresponding command words. This method of data preparation can significantly reduce costs and also help improve the accuracy of easily confused words.

[0030] It should be noted that, in the solution provided in this application, a decoder can also be used to decode the audio input by the user to obtain the first command word output by the decoder that matches the command word list. This application does not limit this.

[0031] It should be noted that the technical solution provided in this application uses speech synthesis technology to synthesize command word audio and easily confused word audio for data preparation. Because speech synthesis technology does not require real people to record specific speech data, it reduces the cost of collecting and labeling real data, making it suitable for applications with limited budgets. By comparing the target sequence and the confused sequence, this discriminative learning method helps reduce recognition crosstalk and achieve a higher recognition rate. This method is a modification of the training method and does not involve changes to engineering reasoning; therefore, this optimization will not result in additional computational resource changes. For prefix substrings, no decoding configuration is used because the substring uses a long delay configuration. Therefore, when encountering errors in the parent-substring pair, the correct result will be obtained when the recognition result of the parent string is output, thus reducing crosstalk. Since the same decoder with different configurations is used, no additional computational resources are added.

[0032] It should be noted that this application also provides a single-module, multi-output scheme. This scheme involves a module with two outputs, sharing some parameters, and the two outputs jointly decide the output result. This scheme saves computational resources by sharing some parameters between the two branches, and the joint decision-making of the two outputs performs double verification of easily confused words, reducing errors related to these words. However, both outputs need to be decoded, requiring two decoders, and adjusting one output may affect the other, making modifications inflexible. Furthermore, the joint decision-making strategy is complex and unstable. The single-module, multi-output scheme can also solve the easily confused word error problem to some extent. This scheme's recognition module contains two recognition results, and the final recognition result is determined jointly by the two results. The advantages of this scheme are that the two outputs share some parameters, saving computational resources, and the joint decision-making plays a role in reducing some crosstalk errors. The disadvantages are that the two outputs influence each other, making adjustments inflexible, and the joint decision-making involves many factors, resulting in insufficient robustness.

[0033] Please refer to Figure 2 The document presents a flowchart illustrating the implementation of the command word recognition method of the present invention.

[0034] like Figure 2 As shown, Step 1: Data preparation. To reduce costs, speech synthesis technology is used to synthesize audio of command words and easily confused words. To improve efficiency, parallel computing technology is used to accelerate the audio synthesis process. To enhance model robustness, data simulation is performed on the audio to increase data diversity.

[0035] Step 2: Training the recognition module parameters. For easily confused words, compare and learn the target sequence and the confused sequence. Continuously reduce the distance between the output of the recognition module and the target sequence, while increasing the distance with the confused sequence, thereby improving the discriminativeness of the recognition module and optimizing the recognition results.

[0036] Step 3: Determine whether the command word vocabulary contains a prefix parent string. If it does, use a different decoding configuration in the decoder: use a long delay configuration for the child string and a short delay configuration for the parent string to reduce errors in the parent string. If it does not contain the seed parent string, use the same decoding configuration.

[0037] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of combined actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, as some steps can be performed in other orders or simultaneously according to the present invention. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention. In the above embodiments, the descriptions of each embodiment have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0038] In some embodiments, the present invention provides a non-volatile computer-readable storage medium storing one or more programs including execution instructions, which can be read and executed by an electronic device (including but not limited to a computer, server, or network device, etc.) to perform any of the command word recognition methods described above.

[0039] In some embodiments, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-volatile computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform any of the above-described command word recognition methods.

[0040] In some embodiments, the present invention also provides an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a command word recognition method.

[0041] Figure 3 This is a schematic diagram of the hardware structure of an electronic device that performs a command word recognition method according to another embodiment of this application, as shown below. Figure 3 As shown, the device includes: One or more processors 310 and memory 320, Figure 3 Take the 310 processor as an example.

[0042] The device for performing the command word recognition method may further include an input device 330 and an output device 340.

[0043] The processor 310, memory 320, input device 330, and output device 340 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0044] The memory 320, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the command word recognition method in the embodiments of this application. The processor 310 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 320, thereby implementing the command word recognition method in the above-described method embodiments.

[0045] The memory 320 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the command word recognition device. Furthermore, the memory 320 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 320 may optionally include memory remotely located relative to the processor 310, and this remote memory may be connected to the command word recognition device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0046] Input device 330 can receive input numerical or character information and generate signals related to user settings and function control of the command word recognition device. Output device 340 may include a display device such as a display screen.

[0047] The one or more modules are stored in the memory 320, and when executed by the one or more processors 310, they execute the command word recognition method in any of the above method embodiments.

[0048] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0049] The electronic devices in this application embodiments exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0050] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include: PDAs, MIDs, and UMPCs, etc.

[0051] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players, handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0052] (4) Other airborne electronic devices with data interaction capabilities, such as vehicle-mounted systems installed on vehicles.

[0053] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0054] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A command word recognition method, comprising: Obtain the audio input by the user, and decode the audio to obtain the decoding result; If the decoding result matches a command word in the command word list, check if there are other command words in the command word list that are prefixed to the string of the command word, where strings with the same prefix characters are prefixed to each other. If the command word list contains other command words whose strings are prefixes of the given command word, then subsequent decoding is performed according to the decoding configuration corresponding to the prefixes. Here, the shorter string among the multiple strings that are prefixes is the substring, and the longest string among the multiple strings that are prefixes is the parent string. The subsequent decoding according to the decoding configuration corresponding to the prefixes includes: if the given command word string is the parent string, a short-delay decoding configuration is used; if the given command word string is the substring, a long-delay decoding configuration is used. Using different configurations with a single decoding module ensures that the computational resource consumption is almost unchanged. If there are no other command words in the command word list that are prefixes of the parent and child strings of a certain command word, then the current decoding configuration will continue to be used for subsequent decoding.

2. The method according to claim 1, wherein, The decoding of the audio to obtain the decoding result includes: The decoding result is obtained based on the recognition module after training, wherein the recognition module after training is trained by comparison learning.

3. The method according to claim 2, wherein, The training method using contrastive learning includes: The target sequence and the obfuscated sequence are compared and learned based on the decoding results, wherein the target sequence and the obfuscated sequence are similar command words obtained from the user's command word list.

4. The method according to claim 3, wherein, Before comparing the target sequence and the confused sequence based on the decoding result, the method further includes: Speech data corresponding to the command vocabulary is synthesized using speech synthesis technology.

5. The method according to claim 4, wherein, The method further includes: The speech data corresponding to the command vocabulary is accelerated using parallel computing techniques.

6. The method according to claim 1, wherein, The condition that the decoding result matches a command word in the command word list includes: The audio is decoded using a decoder to obtain the first command word output by the decoder that matches the command word list.

7. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 6.

8. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images