Voice wake-up recognition methods, devices and storage media

Abstract

This application relates to a voice wake-up recognition method, device, and storage medium, belonging to the field of computer technology. The method includes: inputting audio data into a pre-trained speech detection model to obtain a speech detection result; when the speech detection result indicates that the audio data is speech data, inputting the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake-up word recognition model into a preset algorithm code segment to obtain a wake-up word recognition result; when the wake-up word recognition result successfully wakes up the electronic device, inputting the algorithm parameters corresponding to the command word recognition model into the preset algorithm code segment to obtain a command word recognition result. This method can solve the problem that voice wake-up recognition methods based on high-performance neural network models cannot run on low-computing-power chips, and that neural network models running on low-computing-power chips have a high wake-up recognition error rate. It can ensure the accuracy of wake-up recognition while reducing the requirements for chip computing power.

Description

Technical Field

[0001] This application relates to a voice wake-up recognition method, device, and storage medium, belonging to the field of computer technology. Background Technology

[0002] With the development of artificial intelligence, many types of electronic devices now possess the ability to recognize audio data. Users can then interact with these devices via voice. During this interaction, users typically need to utter a wake-up word to trigger the device to enter a working state, and they also need to input a command word to trigger the device to execute the corresponding command. For example, in a wake-up-free scenario, a user might say the voice command "Open music," where "Open music" is both the wake-up word and the command word. Accordingly, the electronic device needs to recognize both the wake-up word and the command word to enable voice interaction with the user.

[0003] Traditional voice wake-up recognition methods include: acquiring audio data; performing voice detection on the audio data, such as using a Voice Activity Detection (VAD) algorithm to obtain valid voice data; inputting the valid voice data into a pre-trained wake-up word detection model to detect wake-up words in the valid voice data; and, if the valid voice data contains a wake-up word, inputting the valid voice data into a command word recognition model to obtain the command word.

[0004] However, the aforementioned VAD algorithm, wake-up word detection model, and command word recognition model are typically implemented using more than three neural network models. This requires deploying multiple algorithm models and their corresponding code segments in the flash memory of electronic devices. Consequently, the simultaneous operation of at least three models during voice wake-up and command word recognition consumes significant computing power and memory, making it impossible to implement voice wake-up and recognition methods on electronic devices with limited chip computing power.

[0005] In some implementation schemes, algorithm modules such as VAD algorithm, wake word detection model and command word recognition model can be compressed and trimmed. Although such reduction method can make the model run in electronic devices with limited chip computing power, it will reduce the algorithm performance of individual modules. At this time, the wake-up error rate and the command word recognition error rate are both high, which will lead to the problem of reducing the voice interaction effect. Summary of the Invention

[0006] This application provides a voice wake-up recognition method, apparatus, device, and storage medium, which can solve the problem that voice wake-up recognition methods based on high-performance neural network models cannot run on low-computing-power chips, and that neural network models running on low-computing-power chips have a high wake-up recognition error rate. This application provides the following technical solution:

[0007] Firstly, a voice wake-up recognition method is provided, the method comprising:

[0008] In response to receiving audio data, the audio data is input into a pre-trained speech detection model to obtain speech detection results;

[0009] When the speech detection result indicates that the audio data is speech data, the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake word recognition model are input into a preset algorithm code segment to perform wake word recognition and obtain the wake word recognition result; the wake word is used to trigger the electronic device to enter the working state;

[0010] If the electronic device is successfully woken up by the wake-up word recognition result, the algorithm parameters corresponding to the command word recognition model are input into the preset algorithm code segment to perform command word recognition on the valid voice data to obtain the command word recognition result; the command word is used to instruct the electronic device to execute the command corresponding to the recognized word.

[0011] Optionally, the preset algorithm code segment extracts the feature sequence of the wake word or command word through a character classification algorithm, and decodes the feature sequence into the wake word or command word through a decoding network.

[0012] Optionally, the step of inputting the valid speech frames detected by the speech detection model and the corresponding algorithm parameters of the wake word recognition model into a preset algorithm code segment to perform wake word recognition and obtain the wake word recognition result includes:

[0013] The wake-up difficulty level of the wake-up word is determined by the decoding network.

[0014] The decoding network outputs the wake-up word recognition result according to the wake-up difficulty level and the feature sequence.

[0015] Optionally, determining the wake-up difficulty level of the wake-up word through the decoding network includes:

[0016] Read the wake-up history and historical audio data of the electronic device and the historical voice detection results;

[0017] If the wake-up history indicates that the electronic device has been woken up within a first preset time period, the wake-up difficulty level of the wake-up word is determined to be the first level;

[0018] If the historical voice detection result indicates that no voice data is recognized in the current environment within a second preset time period, the wake-up difficulty level of the wake-up word is determined to be the second level.

[0019] If the historical voice detection result indicates that voice data is continuously recognized in the current environment within a third preset duration, and the wake-up history indicates that the electronic device has not been woken up within a fourth preset duration, then the wake-up difficulty level of the wake-up word is determined to be level three; the fourth preset duration is less than the third preset duration.

[0020] The first level is lower than the second level, which is lower than the third level.

[0021] Optionally, the wake word recognition model and the command word recognition model belong to the same recognition model, which includes a first network sharing layer, a first network branch and a second network branch connected to the first network sharing layer, wherein the first network sharing layer and the first network branch together constitute the wake word recognition model, and the first network sharing layer and the second network branch together constitute the command word recognition model;

[0022] or,

[0023] The wake word recognition model and the command word recognition model are the same recognition model, and the recognition model recognizes both the wake word and the command word simultaneously.

[0024] Optionally, when the wake-up word recognition model and the command word recognition model belong to the same recognition model, the step of inputting the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake-up word recognition model into a preset algorithm code segment to perform wake-up word recognition and obtain the wake-up word recognition result includes:

[0025] The effective voice data and the algorithm parameters corresponding to the first network branch are input into the preset algorithm code segment to obtain the wake word recognition result;

[0026] Accordingly, the step of inputting the algorithm parameters corresponding to the command word recognition model into the preset algorithm code segment to perform command word recognition on the valid speech data to obtain the command word recognition result includes:

[0027] The feature data output by the first network shared layer and the algorithm parameters corresponding to the second network branch are input into the preset algorithm code segment to obtain the command word recognition result.

[0028] Optionally, the speech detection model and the recognition model belong to the same multi-task model, which includes a second network sharing layer and a third network branch and a fourth network branch connected to the second network sharing layer. The second network sharing layer and the third network branch together constitute the speech detection model, and the second network sharing layer and the fourth network branch together constitute the recognition model. The number of network layers in the second network sharing layer is less than the number of network layers in the first network sharing layer.

[0029] Optionally, the step of inputting the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake word recognition model into a preset algorithm code segment to perform wake word recognition and obtain the wake word recognition result includes:

[0030] If the speech detection result indicates that the audio data is speech data, the fourth network branch is triggered to use the valid speech data output by the second network sharing layer to identify the wake word, and obtain the wake word identification result.

[0031] In a second aspect, an electronic device is provided, the device including a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement the voice wake-up recognition method provided in the first aspect.

[0032] Thirdly, a computer-readable storage medium is provided, wherein a program is stored in the storage medium, and the program, when executed by a processor, is used to implement the voice wake-up recognition method provided in the first aspect.

[0033] The beneficial effects of this application include at least the following: In response to received audio data, the audio data is input into a pre-trained speech detection model to obtain a speech detection result; when the speech detection result indicates that the audio data is speech data, the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake-up word recognition model are input into a preset algorithm code segment to perform wake-up word recognition and obtain a wake-up word recognition result; when the wake-up word recognition result successfully wakes up the electronic device, the algorithm parameters corresponding to the command word recognition model are input into a preset algorithm code segment to perform command word recognition on the valid speech data and obtain a command word recognition result; this can solve the problem that speech wake-up recognition methods based on high-performance neural network models cannot run on low-computing-power chips, and that neural network models running on low-computing-power chips have a high wake-up recognition error rate; since the wake-up word recognition model and the command word recognition model share the same set of preset algorithm code segments, only one set of algorithm code segments needs to be deployed on the chip. This saves at least one set of algorithm code segments, thereby reducing the computational power requirements of the chip without model pruning, ensuring both the accuracy of wake-up recognition and reducing the computational power requirements of the chip.

[0034] In addition, since the preset algorithm code segment uses the algorithm code segment corresponding to the command word recognition model, multiple wake words can be recognized when using this preset algorithm code segment for command word recognition.

[0035] In addition, the wake-up difficulty level of the wake-up word is determined by the decoding network of the wake-up word recognition model; the wake-up word recognition result is output by the decoding network according to the wake-up difficulty level and feature sequence; this can solve the problem of high false wake-up rate when recognizing multiple wake-up words and improve wake-up accuracy.

[0036] In addition, by designing the wake word recognition model and the command word recognition model in the same recognition model and sharing the same first network shared layer, the calculation results of the first network shared layer can be reused when recognizing command words, thus saving the computing resources of electronic devices.

[0037] In addition, by designing the speech detection model and the recognition model in the same wake-up recognition model and sharing the same second network shared layer, the calculation results of the second network shared layer can be reused when recognizing wake words and command words, thus saving computing resources of electronic devices.

[0038] In addition, by sharing parameters among multiple neural networks, the goal of multi-task training can be achieved. That is, a single neural network can output multiple different tasks, thus achieving the goals of memory sharing and computing power sharing, and also enabling the completion of different speech tasks.

[0039] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of a voice wake-up recognition process provided in one embodiment of this application;

[0041] Figure 2 This is a flowchart of a voice wake-up recognition method provided in one embodiment of this application;

[0042] Figure 3 This is a schematic diagram of a recognition model provided in one embodiment of this application;

[0043] Figure 4 This is a schematic diagram of a wake-up recognition model provided in one embodiment of this application;

[0044] Figure 5 This is a block diagram of a voice wake-up recognition device provided in one embodiment of this application;

[0045] Figure 6 This is a block diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0046] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0047] refer to Figure 1 Traditional voice wake-up recognition methods typically involve inputting audio data into a VAD algorithm, a wake-up detection model, and a command word recognition model for voice wake-up recognition. The VAD algorithm, wake-up detection model, and command word recognition model can all be implemented using neural networks (NNs). These neural networks can be feedforward sequential memory networks (FSMNs), deep neural networks (DNNs), convolutional neural networks (CNNs), or time-delay neural networks (TDNNs), etc. This embodiment will not describe the traditional implementations of the VAD algorithm, wake-up detection model, and command word recognition model in detail.

[0048] Traditional voice wake-up recognition methods develop the VAD algorithm, wake-up detection model, and command word recognition model as separate modules. This results in the industry using different algorithms for wake-up and recognition, which are essentially maintained and developed separately. Consequently, existing voice wake-up recognition methods often port algorithms from high-resource chips to low-resource chips through trimming and compression. However, this reduces algorithm performance, leading to poor voice interaction on low-resource chips.

[0049] Meanwhile, commonly used wake word detection algorithms are based on neural network models to extract features and then perform streaming real-time wake word detection. This approach typically only supports the detection of one wake word at a time. When multiple wake words are needed, strict formatting requirements become necessary to prevent word mixing. Furthermore, because wake word algorithms perform streaming matching, they struggle to handle substring issues. For example, if two wake words are supported simultaneously, "open music" and "open music control," when the user says "open music control," it might be mixed up to "open music." The probability of false wake-ups is also higher when multiple wake words are supported compared to when a single wake word is used.

[0050] However, the essence of VAD algorithm, wake word detection, and command word recognition is the classification and processing of valid speech data, so the three tasks have similarities. Based on this, wake word detection and command word recognition can essentially be treated as a single recognition task. The post-decoding processing for wake word detection and command word recognition can be unified using a single scheme. This way, wake word detection and command word recognition can share a single decoding algorithm, and only one set of corresponding algorithm code segments needs to be deployed on the chip. This saves at least one set of algorithm code segments, thereby reducing the computational power requirements of the chip.

[0051] Based on the above principles, the voice wake-up recognition method provided in this application will be described in detail below. Optionally, this application uses the voice wake-up recognition method provided in various embodiments in an electronic device as an example for illustration. The electronic device is a terminal or a server. The terminal can be a mobile phone, computer, tablet computer, scanner, electronic eye, surveillance camera, etc. This embodiment does not limit the type of electronic device.

[0052] Figure 2 This is a flowchart of a voice wake-up recognition method provided in one embodiment of this application. The method includes at least the following steps:

[0053] Step 201: In response to receiving audio data, input the audio data into a pre-trained speech detection model to obtain speech detection results.

[0054] The audio data can be collected by an audio acquisition device on an electronic device, or it can be sent to the electronic device by other devices; the audio data can be streaming audio data or offline audio data, and the audio data consists of at least one frame. This embodiment does not limit the source and implementation method of the audio data.

[0055] Speech detection models are used to detect the presence of valid speech data in audio data. Specifically, they use the Voice Analyzer (VAD) algorithm to determine speech in audio data. The VAD algorithm is based on a neural network model. Valid speech data refers to the user-generated speech data that is expected to be recognizable.

[0056] In one example, the speech detection model includes a decision module and a post-processing module. The decision module determines whether a frame of audio data belongs to a human voice and assigns a label. The input to the decision module is the audio data, and the output is the classification result of that audio data. Typically, the classification result includes two categories: speech and non-speech. Optionally, the decision module includes an input layer, multiple feature extraction layers, and a classification output layer connected to the feature extraction layers. The input layer is used to input the audio data into the speech detection model. The feature extraction layers are used to extract features from the audio data input to the speech detection model; each feature extraction layer can be implemented using N layers. The classification output layer outputs the feature data from the feature extraction layers as the classification result of the audio data.

[0057] The post-processing module smooths the label sequence output by the decision module to avoid discontinuous spikes in the classification results. This post-processing module can be implemented using a smoothing window of preset length.

[0058] The speech detection model is trained using sample audio data and the corresponding classification labels for that sample audio data.

[0059] The speech detection results include two scenarios:

[0060] The first scenario: The audio data includes valid speech data. In this case, the speech detection result not only indicates that the audio data includes valid speech data, but also includes valid audio data in the audio data, such as valid speech frames in the audio data.

[0061] The second scenario: The audio data does not include valid speech data. In this case, the speech detection result indicates that the audio data does not include valid speech data.

[0062] Step 202: If the speech detection result indicates that the audio data is speech data, input the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake word recognition model into the preset algorithm code segment to perform wake word recognition and obtain the wake word recognition result.

[0063] Among them, the wake word is used to trigger the electronic device to enter the working state.

[0064] In this embodiment, the wake-up word recognition model and the command word recognition model share the same set of preset algorithm code segments. These preset algorithm code segments are used by the command word recognition model, which can generally recognize multiple command words. Therefore, using this preset algorithm code segment for wake-up word recognition allows for the identification of multiple wake-up words.

[0065] In this embodiment, the preset algorithm code segment extracts the feature sequence of the wake-up word or command word through a character classification algorithm, and decodes the feature sequence into the wake-up word or command word through a decoding network. In other words, the preset algorithm code segment includes the algorithm code segment corresponding to the character classification algorithm and the algorithm code segment corresponding to the decoding network. In one embodiment, the character classification algorithm can be the Connectionist Temporal Classification (CTC) algorithm, and correspondingly, the decoding network can be a CTC decoding network; in other embodiments, the character classification algorithm can also be other algorithms, which will not be listed here.

[0066] In one example, the wake word recognition model includes a multi-layer feature extraction layer, a classification result output layer connected to the multi-layer feature extraction layer, and a decoding network connected to the classification result output layer.

[0067] The algorithm parameters for the wake word recognition model are obtained by training the character classification algorithm and the decoding network using sample audio data and the wake word labels corresponding to the sample audio data.

[0068] Optionally, since the probability of false wake-ups is relatively high in scenarios with multiple wake words, the pre-defined algorithm code can also obtain the current wake-up difficulty level of each wake word, thereby reducing the probability of false wake-ups.

[0069] In one example, the wake-up difficulty level of the wake-up word is determined by a decoding network; the wake-up word recognition result is output by the decoding network according to the wake-up difficulty level and the feature sequence output by the character classification algorithm.

[0070] Optionally, the decoding network may determine the wake-up difficulty level of the wake word in the following ways, including but not limited to:

[0071] The first method involves decoding the network to read the wake-up history and historical audio data of electronic devices, and then determining the wake-up difficulty level based on this wake-up history and historical voice detection results.

[0072] Schematic, determining the wake-up difficulty level based on the wake-up history and historical voice detection results includes: determining the wake-up difficulty level of the wake-up word as level one when the wake-up history indicates that the electronic device has been woken up within a first preset duration; determining the wake-up difficulty level of the wake-up word as level two when the historical voice detection results indicate that no voice data has been recognized in the current environment within a second preset duration; and determining the wake-up difficulty level of the wake-up word as level three when the historical voice detection results indicate that voice data has been continuously recognized in the current environment within a third preset duration and the wake-up history indicates that the electronic device has not been woken up within a fourth preset duration; wherein the fourth preset duration is less than the third preset duration.

[0073] The first level is lower than the second level, which is lower than the third level. The higher the awakening difficulty level, the more difficult it is to awaken the user.

[0074] The first preset duration, the second preset duration, the third preset duration, and the fourth preset duration may be the same or different. This embodiment does not limit the values ​​of the first preset duration, the second preset duration, the third preset duration, and the fourth preset duration.

[0075] For example, the initial state of the default decoding network is normal (i.e., level 2). After the electronic device is successfully woken up, if it is woken up again within a first preset duration (e.g., 10 seconds), the decoding network will assign a reward score to the wake word, switching its wake-up difficulty level to level 1. At this point, the electronic device is more easily woken up by the wake word. Simultaneously, the speech detection model determines the current scene information. If the electronic device is in a silent environment, it indicates that no one is speaking or using the device, and it is less likely to be mistakenly woken up. Therefore, the electronic device remains in normal mode, with the wake word's wake-up difficulty level at level 2. If the speech detection model detects a third preset duration (e.g., 10 minutes) of speech consecutively, and the same wake word has not been used for the most recent fourth preset duration (e.g., 5 minutes), it indicates that no one is using the electronic device, but many users are speaking, increasing the risk of mistaken wake-up. Therefore, the electronic device enters a penalty state, where a penalty score is assigned to the wake word to make it less likely to be mistakenly woken up.

[0076] The second method involves decoding the network to read the wake-up history of the wake-up word and determining the wake-up difficulty level based on this wake-up history.

[0077] Schematic illustration: if the electronic device is not woken up within the fifth preset time period indicated by the wake-up history, the wake-up difficulty level of the wake-up word is determined to be level three; if the electronic device is woken up within the fifth preset time period indicated by the wake-up history, the wake-up difficulty level of the wake-up word is determined to be level two; if the electronic device is woken up a preset number of times within the sixth preset time period indicated by the wake-up history, the wake-up difficulty level of the wake-up word is determined to be level one. The sixth preset time period is shorter than the fifth preset time period.

[0078] For example: If a user has not woken up the electronic device within the most recent fifth preset time period (e.g., 10 minutes), the electronic device enters a difficult-to-wake state, and the currently recognized wake word has a penalty score to reduce the probability of the electronic device being falsely woken up. If the electronic device has been woken up, it remains in a normal state, and the wake word has neither a reward score nor a penalty score. If the electronic device has been woken up twice within the sixth preset time period (e.g., 5 seconds) (i.e., reaching the preset number), the currently recognized wake word has a reward score, and the electronic device is in an easy-to-wake state.

[0079] In other embodiments, the decoding network may determine the wake-up difficulty level in other ways, or the wake-up difficulty level may be set to more levels. This embodiment does not limit the setting and determination methods of the wake-up difficulty level.

[0080] The wake word recognition result is output by the decoding network according to the feature sequence output by the wake-up difficulty level and the character classification algorithm. This includes: obtaining the confidence of each wake word obtained by the decoding network according to the feature sequence; determining the confidence threshold corresponding to the wake word according to the wake-up difficulty level, and the wake-up difficulty level and the confidence threshold are positively correlated; and determining the wake word with a confidence greater than the confidence threshold as the wake word recognition result.

[0081] Alternatively, obtain the confidence level of each wake word obtained by the decoding network according to the feature sequence; determine the confidence coefficient corresponding to the wake word according to the wake difficulty level, which is negatively correlated with the confidence threshold; and determine the wake word recognition result as the wake word with a confidence level greater than the preset confidence threshold.

[0082] In actual implementation, the wake word recognition result can be output by the decoding network according to the feature sequence output by the wake-up difficulty level and the character classification algorithm, or other methods can be used, which will not be listed in this embodiment.

[0083] There is a logic judgment module between the speech detection model and the wake word recognition process to judge the speech detection result. If the speech detection result indicates that the audio data is speech data, step 202 is executed; if the speech detection result indicates that the audio data is not speech data, step 201 is executed again for the next audio data.

[0084] Step 203: If the electronic device is successfully woken up by the wake-up word recognition result, the algorithm parameters corresponding to the command word recognition model are input into the preset algorithm code segment to perform command word recognition on the valid voice data to obtain the command word recognition result.

[0085] The command word is used to instruct the electronic device to execute the command corresponding to the recognition word.

[0086] Since the command word recognition model and the wake word recognition model share the same preset algorithm code segment, the command word recognition model also includes a multi-layer feature extraction layer, a classification result output layer connected to the multi-layer feature extraction layer, and a decoding network connected to the classification result output layer.

[0087] In one example, the wake-up word recognition model and the command word recognition model have different algorithm parameters. In this case, the wake-up word recognition model and the command word recognition model can be considered as two cascaded recognition models. Accordingly, the wake-up word recognition model and the command word recognition model are trained through two independent training processes. In other words, the algorithm parameters for the command word recognition model are obtained by training the character classification algorithm and the decoding network using sample audio data and the corresponding command word labels for that sample audio data.

[0088] Accordingly, there is a logic judgment module between the speech detection model and the wake word recognition process to judge the wake word recognition result. If the wake word recognition result indicates that the wake word has been recognized, step 203 is executed; if the wake word recognition result indicates that the wake word has not been recognized, step 202 is executed again for the next valid speech data.

[0089] However, due to space limitations, two separate models cannot be too large, thus limiting their supported capabilities. For example, compared to a 100k-parameter wake-word recognition model and a 100k-parameter command word recognition model, a 200k-parameter integrated model that performs wake-up and recognition tasks through parameter sharing will have better performance. This is because wake-up and recognition essentially involve the discrimination of speech frames, which allows for parameter sharing. Based on this, this embodiment also provides the following model implementation method.

[0090] In another example, the wake-word recognition model and the command word recognition model are the same model, and the model recognizes both wake-words and command words simultaneously. In this case, the algorithm parameters for the wake-word recognition model and the command word recognition model are identical, and the output layer of this model can output both wake-words and command words simultaneously. Accordingly, the algorithm parameters for the recognition model are obtained by training the character classification algorithm and the decoding network using sample audio data, the wake-word labels corresponding to the sample audio data, and the command word labels corresponding to the sample audio data.

[0091] In the example, although parameter sharing is possible between the wake word recognition model and the command word recognition model, the recognition model performs both tasks simultaneously through a single output layer, resulting in excessive coupling. If either wake word recognition or command word recognition encounters a problem, they cannot be optimized independently and will affect each other. Therefore, this embodiment also provides the following model.

[0092] In yet another example, refer to Figure 3 The wake word recognition model and the command word recognition model belong to the same recognition model 31. The recognition model includes a first network sharing layer 311, a first network branch 312 and a second network branch 313 connected to the first network sharing layer 311. The first network sharing layer and the first network branch together form the wake word recognition model, and the first network sharing layer and the second network branch together form the command word recognition model.

[0093] The first network shared layer includes M network layers, where M is a positive integer. The first network branch 312 includes a classification result output layer of the character classification algorithm and a decoding network connected to the classification result output layer; the second network branch 313 also includes a classification result output layer of the character classification algorithm and a decoding network connected to the classification result output layer.

[0094] The training process of the recognition model 31 includes: training the first network shared layer 311 and the first network branch 312 using sample audio data and the wake word labels corresponding to the sample audio data; after training is completed, keeping the algorithm parameters of the first network shared layer 311 unchanged, training the second network branch 313 using sample audio data and the command word labels corresponding to the sample audio data, until the model training is completed and the recognition model is obtained.

[0095] At this point, as an alternative to step 202, the effective speech data and the algorithm parameters corresponding to the first network branch are input into a preset algorithm code segment to obtain the wake-up word recognition result; correspondingly, as an alternative to step 203, the feature data output by the first network shared layer and the algorithm parameters corresponding to the command word recognition model are input into a preset algorithm code segment to obtain the command word recognition result.

[0096] As can be seen from the above process, since the first network branch and the second network branch share the first network shared layer, the output results of the first network shared layer can be directly used in the command word recognition process, saving computing resources.

[0097] Accordingly, there is a logic judgment module between the first network branch and the second network branch to judge the wake word recognition result. If the wake word recognition result indicates that the wake word has been recognized, the second network branch is triggered to run to execute step 203; if the wake word recognition result indicates that the wake word has not been recognized, step 202 is executed again for the next audio data.

[0098] In summary, the voice wake-up recognition method provided in this embodiment, in response to received audio data, inputs the audio data into a pre-trained speech detection model to obtain a speech detection result; when the speech detection result indicates that the audio data is speech data, the effective speech data detected by the speech detection model and the algorithm parameters corresponding to the wake-up word recognition model are input into a preset algorithm code segment to perform wake-up word recognition and obtain a wake-up word recognition result; when the wake-up word recognition result successfully wakes up the electronic device, the algorithm parameters corresponding to the command word recognition model are input into the preset algorithm code segment to perform command word recognition on the effective speech data and obtain a command word recognition result. This can solve the problem that voice wake-up recognition methods based on high-performance neural network models cannot run on low-computing-power chips, and the wake-up recognition error rate of neural network models running on low-computing-power chips is high; since the wake-up word recognition model and the command word recognition model share the same set of preset algorithm code segments, only one set of algorithm code segments needs to be deployed on the chip. This can save at least one set of algorithm code segments, thereby reducing the computing power requirements of the chip without model pruning, ensuring the accuracy of wake-up recognition while reducing the computing power requirements of the chip.

[0099] In addition, since the preset algorithm code segment uses the algorithm code segment corresponding to the command word recognition model, multiple wake words can be recognized when using this preset algorithm code segment for command word recognition.

[0100] In addition, the wake-up difficulty level of the wake-up word is determined by the decoding network of the wake-up word recognition model; the wake-up word recognition result is output by the decoding network according to the wake-up difficulty level and feature sequence; this can solve the problem of high false wake-up rate when recognizing multiple wake-up words and improve wake-up accuracy.

[0101] In addition, by designing the wake word recognition model and the command word recognition model in the same recognition model and sharing the same first network shared layer, the calculation results of the first network shared layer can be reused when recognizing command words, thus saving the computing resources of electronic devices.

[0102] Optionally, based on the above embodiments, in order to further save the computing resources of electronic devices, the speech detection model and the recognition model belong to the same multi-task model. The multi-task model includes a second network sharing layer, and a third network branch and a fourth network branch connected to the second network sharing layer. The second network sharing layer and the third network branch together form the speech detection model, and the second network sharing layer and the fourth network branch together form the recognition model. The number of network layers in the second network sharing layer is less than the number of network layers in the first network sharing layer.

[0103] Based on the recognition model Figure 3 Taking the model structure shown as an example, refer to Figure 4 The multi-task model sequentially includes an input layer 41, a second network sharing layer 42 connected to the input layer 41, a third network branch 43 connected to the second network sharing layer 42, and a fourth network branch 44. The fourth network branch 44 includes a first network sharing layer 311, a first network branch 312 connected to the first network sharing layer 311, and a second network branch 313.

[0104] The second network sharing layer 42 includes N network layers, where N is a positive integer, and since the real-time requirements for speech detection are high, N < M.

[0105] The third network branch 43 includes the output layer of the decision module and a post-processing module. A logic decision module is located between the third network branch 43 and the first network shared layer 311. If the speech detection result output by the post-processing module indicates that the audio data is speech data, the feature data output by the second network shared layer 42 is input as valid speech data into the first network shared layer 311 for wake-up word recognition.

[0106] The training process of the multi-task model includes: training the second network shared layer 42 and the third network branch 43 using sample audio data and the corresponding voice tags; after training, keeping the algorithm parameters of the second network shared layer 42 and the third network branch 43 unchanged, training the first network shared layer 311 and the first network branch 312 using sample audio data and the corresponding wake-up word tags; after training, keeping the algorithm parameters of the first network shared layer 311 and the first network branch 312 unchanged, training the second network branch 313 using sample audio data and the corresponding command word tags, until the model training is completed and the wake-up recognition model is obtained.

[0107] At this point, as an alternative step to step 202, the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake-word recognition model are input into a preset algorithm code segment to perform wake-word recognition and obtain the wake-word recognition result, including:

[0108] If the speech detection result indicates that the audio data is speech data, the fourth network branch is triggered to use the valid speech data output by the second network shared layer to identify the wake word, and the wake word identification result is obtained.

[0109] As can be seen from the above process, the third network branch can be run continuously until it detects valid speech data. The peak computational load during the simultaneous operation of the speech detection model, wake word recognition, and command word recognition is mainly from the input layer to the first shared network layer of the M layer, which greatly reduces the use of memory and computing power. At the same time, due to the sharing of multi-task parameters, each task can be completed effectively.

[0110] In this embodiment, by designing the speech detection model and the recognition model in the same wake-up recognition model and sharing the same second network shared layer, the calculation results of the second network shared layer can be reused when recognizing wake words and command words, thus saving the computing resources of electronic devices.

[0111] In addition, by sharing parameters among multiple neural networks, the goal of multi-task training can be achieved. That is, a single neural network can output multiple different tasks, thus achieving the goals of memory sharing and computing power sharing, and also enabling the completion of different speech tasks.

[0112] Figure 5 This is a block diagram of a voice wake-up recognition device provided in one embodiment of this application. The device includes at least the following modules: a voice detection module 510, a wake-up word recognition module 520, and a command word recognition module 530.

[0113] The speech detection module 510 is used to, in response to receiving audio data, input the audio data into a pre-trained speech detection model to obtain a speech detection result;

[0114] The wake-up word recognition module 520 is used to input the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake-up word recognition model into a preset algorithm code segment when the speech detection result indicates that the audio data is speech data, so as to perform wake-up word recognition and obtain a wake-up word recognition result; the wake-up word is used to trigger the electronic device to enter the working state;

[0115] The command word recognition module 530 is used to input the algorithm parameters corresponding to the command word recognition model into the preset algorithm code segment when the wake-up word recognition result successfully wakes up the electronic device, so as to perform command word recognition on the valid voice data to obtain the command word recognition result; the command word is used to instruct the electronic device to execute the command corresponding to the recognized word.

[0116] For relevant details, please refer to the above method implementation examples.

[0117] It should be noted that the voice wake-up recognition device provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the voice wake-up recognition device can be divided into different functional modules to complete all or part of the functions described above. In addition, the voice wake-up recognition device and the voice wake-up recognition method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0118] Figure 6This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 601 and a memory 602.

[0119] Processor 601 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 601 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0120] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 are used to store at least one instruction, which is executed by the processor 601 to implement the voice wake-up recognition method provided in the method embodiments of this application.

[0121] In some embodiments, the electronic device may also optionally include a peripheral device interface and at least one peripheral device. The processor 601, memory 602, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to, radio frequency circuits, touch displays, audio circuits, and power supplies.

[0122] Of course, electronic devices may also include fewer or more components, and this embodiment does not limit this.

[0123] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the voice wake-up recognition method of the above method embodiments.

[0124] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the voice wake-up recognition method of the above method embodiments.

[0125] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A voice wake-up recognition method, characterized in that, The method includes: In response to receiving audio data, the audio data is input into a pre-trained speech detection model to obtain speech detection results; When the speech detection result indicates that the audio data is speech data, the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake word recognition model are input into a preset algorithm code segment to perform wake word recognition and obtain the wake word recognition result; the wake word is used to trigger the electronic device to enter the working state; If the electronic device is successfully woken up by the wake-up word recognition result, the algorithm parameters corresponding to the command word recognition model are input into the preset algorithm code segment to perform command word recognition on the valid voice data to obtain the command word recognition result; the command word is used to instruct the electronic device to execute the command corresponding to the recognized word; The wake word recognition model and the command word recognition model belong to the same recognition model. The recognition model includes a first network sharing layer, a first network branch and a second network branch connected to the first network sharing layer. The first network sharing layer and the first network branch together constitute the wake word recognition model, and the first network sharing layer and the second network branch together constitute the command word recognition model. or, The wake word recognition model and the command word recognition model are the same recognition model, and the recognition model recognizes both the wake word and the command word simultaneously; When the wake-up word recognition model and the command word recognition model belong to the same recognition model, the step of inputting the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake-up word recognition model into a preset algorithm code segment to perform wake-up word recognition and obtain the wake-up word recognition result includes: The effective voice data and the algorithm parameters corresponding to the first network branch are input into the preset algorithm code segment to obtain the wake word recognition result; Accordingly, the step of inputting the algorithm parameters corresponding to the command word recognition model into the preset algorithm code segment to perform command word recognition on the valid speech data to obtain the command word recognition result includes: The feature data output by the first network sharing layer and the algorithm parameters corresponding to the second network branch are input into the preset algorithm code segment to obtain the command word recognition result; The speech detection model and the recognition model belong to the same multi-task model, which includes a second network sharing layer, and a third network branch and a fourth network branch connected to the second network sharing layer. The second network sharing layer and the third network branch together constitute the speech detection model, and the second network sharing layer and the fourth network branch together constitute the recognition model. The number of network layers in the second network sharing layer is less than the number of network layers in the first network sharing layer.

2. The method according to claim 1, characterized in that, The preset algorithm code segment extracts the feature sequence of the wake word or command word through a character classification algorithm, and decodes the feature sequence into the wake word or command word through a decoding network.

3. The method according to claim 2, characterized in that, The step of inputting the valid speech frames detected by the speech detection model and the corresponding algorithm parameters of the wake word recognition model into a preset algorithm code segment to perform wake word recognition and obtain the wake word recognition result includes: The wake-up difficulty level of the wake-up word is determined by the decoding network. The decoding network outputs the wake-up word recognition result according to the wake-up difficulty level and the feature sequence.

4. The method according to claim 3, characterized in that, The step of determining the wake-up difficulty level of the wake-up word through the decoding network includes: Read the wake-up history and historical audio data of the electronic device and the historical voice detection results; If the wake-up history indicates that the electronic device has been woken up within a first preset time period, the wake-up difficulty level of the wake-up word is determined to be the first level; If the historical voice detection result indicates that no voice data is recognized in the current environment within a second preset time period, the wake-up difficulty level of the wake-up word is determined to be the second level. If the historical voice detection result indicates that voice data is continuously recognized in the current environment within a third preset duration, and the wake-up history indicates that the electronic device has not been woken up within a fourth preset duration, then the wake-up difficulty level of the wake-up word is determined to be level three; the fourth preset duration is less than the third preset duration. The first level is lower than the second level, which is lower than the third level.

5. The method according to claim 4, characterized in that, The step of inputting the valid speech data detected by the speech detection model and the algorithm parameters corresponding to the wake word recognition model into a preset algorithm code segment to perform wake word recognition and obtain the wake word recognition result includes: If the speech detection result indicates that the audio data is speech data, the fourth network branch is triggered to use the valid speech data output by the second network sharing layer to identify the wake word, and obtain the wake word identification result.

6. An electronic device, characterized in that, The device includes a processor and a memory; the memory stores a program that is loaded and executed by the processor to implement the voice wake-up recognition method as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement the voice wake-up recognition method as described in any one of claims 1 to 5.

Images