A cascaded wake-up method, device, and storage medium based on keyword recognition technology

Abstract

This specification provides a cascaded wake-up method, apparatus, and storage medium based on keyword recognition technology. The method includes: receiving externally input voice information, the voice information including a voice sequence; identifying keyword nodes in the voice sequence; determining the probability that the voice information is a wake-up signal based on the keyword nodes and the voice sequence; and executing a wake-up system or performing voice recognition on the voice sequence according to the probability. The technical solution provided in this application aims to solve the problem of the mismatch between the performance of the wake-up system and the power consumption and computing power of embedded devices.

Description

Technical Field

[0001] This document relates to the field of artificial intelligence, and in particular to a cascaded wake-up method, device, and storage medium based on keyword recognition technology. Background Technology

[0002] Wake-up is characterized by low power consumption and fast response, so it is used in many smart devices.

[0003] Embedded intelligent devices are characterized by their small size, low power consumption, and limited computing power. Therefore, implementing wake-up functionality in embedded intelligent devices presents the following challenges:

[0004] If the wake-up system has high accuracy in speech recognition, there may be a mismatch between its performance and the power consumption and computing power of the embedded device. Conversely, if the wake-up system's performance is matched to the embedded device's power consumption and computing power, there may be a problem with its accuracy in speech recognition. Summary of the Invention

[0005] Based on the above analysis, this application aims to propose a cascaded wake-up method, device, and storage medium based on keyword recognition technology, so as to ensure the accuracy of the wake-up system in recognizing speech while matching the performance of the wake-up system with the power consumption and computing power of the embedded intelligent device.

[0006] Firstly, one or more embodiments of this specification provide a cascaded wake-up method based on keyword recognition technology, including:

[0007] Receive externally input voice information, the voice information including a voice sequence;

[0008] Identify keyword nodes in the speech sequence;

[0009] Based on the keyword nodes and the speech sequence, determine the probability that the speech information is a wake-up signal;

[0010] Based on the probability, either the wake-up system is activated or speech recognition is performed on the speech sequence.

[0011] Further, determining the probability that the voice information is a wake-up signal based on the keyword node and the voice sequence includes:

[0012] For the keyword nodes, sequence start and end detection and node detection are performed sequentially, and node timing detection is performed on the speech sequence to obtain the probability that the speech information is a wake-up signal.

[0013] Further, the sequence beginning and end detection for the keyword node includes:

[0014] Detect whether the first and last keyword nodes in the speech sequence contain preset keywords;

[0015] The node detection is performed when both the first keyword node and the last keyword node contain the preset keyword.

[0016] Further, the node detection for the keyword node includes:

[0017] Determine the audio data of each keyword node in the speech sequence:

[0018] The similarity between the sound data of each keyword node and the corresponding preset target sound data is determined.

[0019] When the similarity of each keyword node is greater than a preset value, the node timing detection is performed.

[0020] Further, the node timing detection for the speech sequence includes:

[0021] Determine the audio data of the speech sequence;

[0022] The probability of determining that the audio data of the speech sequence is the preset target overall audio data.

[0023] Further, the step of executing the wake-up system or performing speech recognition on the speech sequence based on the probability includes:

[0024] When the probability reaches a first preset value, the wake-up system is activated;

[0025] When the probability does not reach the first preset value, speech recognition is performed on the speech sequence.

[0026] Further, the step of executing the wake-up system or performing speech recognition on the speech sequence based on the probability includes:

[0027] Determine whether the probability reaches a first preset value;

[0028] When the probability reaches a first preset value, it is determined whether the probability has reached a second preset value;

[0029] Upon determining that the second preset value has been reached, the system is activated to wake up.

[0030] If the probability does not reach the second preset value, speech recognition is performed on the speech sequence.

[0031] Secondly, one or more embodiments of this specification provide a cascaded wake-up device based on keyword recognition technology, including: a receiving module, an identification and determination module, a data processing module, and an execution module;

[0032] The receiving module is used to receive externally input voice information, which includes a voice sequence;

[0033] The identification and determination module is used to identify keyword nodes in the speech sequence;

[0034] The data processing module is used to determine the probability that the voice information is a wake-up signal based on the keyword node and the voice sequence;

[0035] The execution module is used to execute the wake-up system or perform speech recognition on the speech sequence according to the probability.

[0036] Furthermore, the data processing module is used to perform sequence beginning and end detection and node detection sequentially on the keyword nodes, and to perform node timing detection on the speech sequence to obtain the probability that the speech information is a wake-up signal.

[0037] Thirdly, one or more embodiments of this specification provide a storage medium, including:

[0038] Used to store computer-executable instructions, which, when executed, implement the method described in the first aspect.

[0039] Compared with the prior art, this application can achieve at least the following technical effects:

[0040] The voice information used to wake up a system is typically a sentence. Each character in the sentence can be considered a keyword node, and these keyword nodes form a speech sequence in a certain order. This application determines the probability that the voice information is a wake-up signal based on the keyword nodes. When the probability is relatively high, the system is woken up directly without triggering speech recognition. Since speech recognition usually requires a large amount of computation, most of the power consumption of embedded devices comes from speech recognition. This application can reduce the frequency of speech recognition triggers through the above method, thereby saving power consumption. At the same time, when it is impossible to determine that the voice information is a wake-up signal, speech recognition can be enabled to ensure accuracy. Thus, under the condition that the performance of the wake-up system matches the power consumption and computing power of the embedded intelligent device, the accuracy of the wake-up system in recognizing speech is guaranteed. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart of a cascaded wake-up method based on keyword recognition technology provided for one or more embodiments of this specification. Specific implementation manners

[0043] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will clearly and completely describe the technical solutions in one or more embodiments of this specification in conjunction with the accompanying drawings in one or more embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this document.

[0044] An embedded device consists of hardware and software and is a device that can operate independently. Its software content only includes the software running environment and its operating system. The hardware content includes various aspects such as a signal processor, a memory, a communication module, etc. Compared with a general computer processing system, there are significant differences in an embedded system. It cannot achieve a large-capacity storage function because there is no matching large-capacity medium. Most intelligent devices in life are embedded devices. For example, intelligent door locks, intelligent sweeping robots, etc.

[0045] These embedded devices receive user instructions through voice recognition. However, the more accurate the voice recognition algorithm, usually the greater its energy consumption, which is not only not conducive to the quick response of the embedded device but also increases the user's usage cost. However, if a voice recognition algorithm with low accuracy is selected, the embedded device may fail to wake up.

[0046] Based on the above scenarios, the embodiments of this application provide a cascaded wake-up method based on keyword recognition technology, including the following steps:

[0047] Step 1: Receive externally input voice information.

[0048] In the embodiments of this application, the voice information includes a voice sequence.

[0049] Step 2: Identify keyword nodes in the voice sequence.

[0050] In the embodiments of this application, the voice sequence includes at least one keyword node. For example, if the wake-up signal is "Hello, Xiaoyi", then "you", "hao", "xiao", and "yi" are all keyword nodes of the voice sequence. Each keyword node is arranged in a certain order according to semantics. That is, only when each keyword node conforms to the order carried in the voice sequence can the system be woken up. For example, "you hao yi xiao" cannot wake up the system.

[0051] Step 3: Determine the probability that the voice information is a wake-up signal based on the keyword nodes and the voice sequence.

[0052] In the embodiment of the present application, determining the probability that the voice information is a wake-up signal requires three steps, namely, performing sequence start and end detection, node detection, and node timing detection in sequence. Among them, for the keyword nodes, sequence start and end detection and node detection are performed in sequence, and for the voice sequence, node timing detection is performed.

[0053] Specifically, the sequence start and end detection includes:

[0054] Detect whether the first keyword node and the last keyword node in the voice sequence contain a preset keyword;

[0055] When both the first keyword node and the last keyword node contain the preset keyword, perform node detection.

[0056] If one of the first keyword node and the last keyword node is not the preset keyword, it means that the probability that this voice information is a wake-up signal is 0, and the current process is directly ended.

[0057] Due to the differences in voice, speech rate, and pronunciation methods of each person, "Hello, Xiaoyi" can have multiple pronunciations. For example, for a person with a faster speech rate, the waveforms corresponding to the four keywords "you", "hao", "xiao", and "yi" are difficult to separate. At this time, no matter at the start or end of the voice sequence, there may be partial waveforms corresponding to "hao" and "xiao". Therefore, in order to prevent misjudgment and filter out noise at the same time, whether both the first keyword node and the last keyword node contain the preset keyword is used as a criterion.

[0058] The node detection includes: <00001​​​​​​​​​​​​​​​​​​

[0065] Determine the audio data of the speech sequence.

[0066] Determine the probability that the audio data of the speech sequence is the overall audio data of the preset target.

[0067] After sequence start and end detection and node detection, most of the noise has been filtered out, leaving only noise like "you're so kind." At this point, the order of the keyword nodes in the speech sequence is checked to see if it conforms to the preset order, in order to improve the accuracy of recognition.

[0068] In summary, sequence beginning and end detection performs an initial screening of the speech signal. Then, following a holistic approach followed by component-by-component analysis, node detection and node timing detection are performed sequentially to accurately determine the probability that the speech information is a wake-up signal. Furthermore, none of these methods rely on speech recognition to identify whether speech information is a wake-up signal. Therefore, these methods can replace some of the speech recognition functions in embedded devices, thus ensuring the accuracy of speech recognition in wake-up systems while matching the performance of the wake-up system with the power consumption and computing power of the embedded intelligent device.

[0069] Step 4: Based on the probability, execute the wake-up system or perform speech recognition on the speech sequence.

[0070] In this embodiment of the application, the wake-up system is executed when the probability reaches a first preset value;

[0071] If the probability does not reach the first preset value, speech recognition is performed on the speech sequence.

[0072] Preferably, the calculation result obtained by combining start / end detection, node detection, and node timing detection is less accurate than that of AI-based speech recognition. Therefore, to improve accuracy, this embodiment sets a second preset value. When the probability reaches the first preset value, it is determined whether the probability reaches the second preset value. If the second preset value is reached, the wake-up system is executed; if the probability does not reach the second preset value, speech recognition of the speech sequence is performed. Speech signals that do not reach the first preset value are directly identified as not being wake-up signals. Simultaneously, by setting the second preset value, the trigger frequency of speech recognition is adjusted. For example, if a user prefers higher accuracy, the second preset value can be set to 99%, which is equivalent to increasing the trigger frequency of speech recognition, thus resulting in higher accuracy. If a user prefers energy saving, the second preset value can be set to 93%, which is equivalent to reducing the trigger frequency of speech recognition, thus reducing energy consumption.

[0073] In summary, this application enables flexible adjustment of the balance between power consumption and accuracy in embedded devices.

[0074] Specifically, the method described in this application is applicable to wake-up methods primarily based on neural networks. With a limited neural network model size, it can effectively balance the wake-up rate and false wake-up rate. For a given set of 2000 positive data points (wake words) and 75 hours of negative data (random recordings excluding wake words), under an appropriate threshold, a wake-up rate of 99% and a false wake-up rate of 0.03 per hour can be achieved. Tested on a 2GHz machine with a 32-core CPU, the real-time rate (RT, the ratio of speech processing time to speech length) can reach 0.02-0.03.

[0075] This application provides a cascaded wake-up device based on keyword recognition technology, including: a receiving module, an identification and determination module, a data processing module, and an execution module;

[0076] The receiving module is used to receive externally input voice information, which includes a voice sequence;

[0077] The identification and determination module is used to identify keyword nodes in the speech sequence;

[0078] The data processing module is used to determine the probability that the voice information is a wake-up signal based on the keyword node and the voice sequence;

[0079] The execution module is used to execute the wake-up system or perform speech recognition on the speech sequence according to the probability.

[0080] In this embodiment of the application, the data processing module is used to perform sequence beginning and end detection and node detection sequentially on the keyword nodes, and to perform node timing detection on the speech sequence to obtain the probability that the speech information is a wake-up signal.

[0081] This application provides a storage medium, including:

[0082] Used to store computer-executable instructions, which, when executed, implement the methods described in the above embodiments.

[0083] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0084] In the 1930s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement to the methodology cannot be implemented using a hardware physical module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0085] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0086] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0087] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0088] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0093] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0094] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0095] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0096] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0097] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0098] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. A cascading wake-up method based on keyword spotting technology, characterized in that, Comprising: receiving external input voice information, the voice information comprising a voice sequence; identifying keyword nodes in the voice sequence; determining whether the voice information is a wake-up signal based on the keyword nodes and the voice sequence through a three-stage cascade determination method, the three-stage cascade determination method comprising sequentially performing the following three stages: S1: sequence head and tail detection stage, detecting whether the first keyword node and the last keyword node in the voice sequence contain a preset keyword; when the first keyword node and the last keyword node both contain a preset keyword, entering the node detection stage; otherwise, ending the process; S2: node detection stage, respectively determining the similarity of the sound data of each keyword node in the voice sequence to the corresponding preset target sound data; when the similarity of each keyword node is greater than a preset value, entering the node timing detection stage; S3: node timing detection stage, determining the probability that the sound data of the voice sequence is a preset target overall sound data; based on the probability, executing a wake-up system or performing voice recognition on the voice sequence, specifically comprising: determining whether the probability reaches a first preset value; when the probability reaches the first preset value, determining whether the probability reaches a second preset value; when it is determined that the second preset value is reached, executing the wake-up system; when the probability does not reach the second preset value, performing voice recognition on the voice sequence; when the probability does not reach the first preset value, directly identifying the voice information as not a wake-up signal.

2. A cascading wake-up device based on keyword spotting technology, characterized in that, Comprising: a receiving module, an identification and determination module, a data processing module, and an execution module; the receiving module is configured to receive external input voice information, the voice information comprising a voice sequence; the identification and determination module is configured to identify keyword nodes in the voice sequence; the data processing module is configured to determine whether the voice information is a wake-up signal based on the keyword nodes and the voice sequence through a three-stage cascade determination method, the three-stage cascade determination method comprising sequentially performing the following three stages: S1: sequence head and tail detection stage, detecting whether the first keyword node and the last keyword node in the voice sequence contain a preset keyword; when the first keyword node and the last keyword node both contain a preset keyword, entering the node detection stage; otherwise, ending the process; S2: node detection stage, respectively determining the similarity of the sound data of each keyword node in the voice sequence to the corresponding preset target sound data; when the similarity of each keyword node is greater than a preset value, entering the node timing detection stage; S3: node timing detection stage, determining the probability that the sound data of the voice sequence is a preset target overall sound data; the execution module is configured to execute a wake-up system or perform voice recognition on the voice sequence based on the probability, specifically configured to: determine whether the probability reaches a first preset value; when the probability reaches the first preset value, determine whether the probability reaches a second preset value; when it is determined that the second preset value is reached, execute the wake-up system; When the probability does not reach the second preset value, performing speech recognition on the speech sequence; When the probability does not reach the first preset value, the speech information is directly identified as not being a wake-up signal.

3. A storage medium, characterized by Comprise: A computer readable medium storing computer executable instructions that, when executed, implement the method recited in claim 1.

Images