Voice wake-up method and apparatus, electronic device, and readable storage medium

Abstract

The application provides a voice wake-up method and device, electronic equipment and a readable storage medium. The method comprises: obtaining audio data and a target audio frame in the audio data, the target audio frame comprising an audio frame containing human voice in the audio data; extracting an acoustic feature of the target audio frame; using a pre-trained neural network model to identify the acoustic feature to obtain an identification result of the audio data; obtaining a confidence level of the audio data based on the identification result; and in the case that the confidence level is greater than a preset threshold, waking up a target device. The application can improve the real-time performance of voice wake-up.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a voice wake-up method, device, electronic device, and readable storage medium. Background Technology

[0002] Voice wake-up is primarily used in voice interaction scenarios to address situations where touch is inconvenient but interaction is still required. Devices in operation continuously process received audio information, treating all sounds as valid information, which can lead to incoherent speech. Voice wake-up successfully avoids this problem, working only when the user calls its name and remaining dormant at other times. Therefore, voice wake-up requires real-time detection of specific segments of the speaker within a continuous speech stream. Existing solutions use acoustic models with attention modules for acoustic modeling; however, attention modules are computationally intensive and have high latency, resulting in poor real-time performance of voice wake-up. Summary of the Invention

[0003] This application provides a voice wake-up method, apparatus, electronic device, and readable storage medium to solve the problem of poor real-time performance of voice wake-up.

[0004] In a first aspect, embodiments of this application provide a voice wake-up method, including:

[0005] Acquire audio data and target audio frames in the audio data, wherein the target audio frames include audio frames in the audio data that contain human voices;

[0006] Extract the acoustic features of the target audio frame;

[0007] The acoustic features are identified using a pre-trained neural network model to obtain the identification results of the audio data;

[0008] The confidence level of the audio data is obtained based on the recognition result;

[0009] If the confidence level is greater than a preset threshold, the target device is woken up.

[0010] Secondly, embodiments of this application also provide a voice wake-up device, including:

[0011] The first acquisition module is used to acquire audio data and target audio frames in the audio data, wherein the target audio frames include audio frames in the audio data that contain human voices;

[0012] An extraction module is used to extract the acoustic features of the target audio frame;

[0013] The recognition module is used to recognize the acoustic features using a pre-trained neural network model to obtain the recognition result of the audio data;

[0014] The second acquisition module is used to acquire the confidence level of the audio data based on the recognition result;

[0015] The first wake-up module is used to wake up the target device when the confidence level is greater than or equal to a preset threshold.

[0016] Thirdly, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and running on the processor, wherein the program or instructions, when executed by the processor, implement the steps in the method described in the first aspect above.

[0017] Fourthly, embodiments of this application also provide a computer-readable storage medium for storing a program, wherein the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of the method described in the first aspect above.

[0018] In this embodiment, a pre-trained neural network model is used to identify the acoustic features to obtain the identification result of the audio data, and the confidence level of the audio data is obtained based on the identification result. If the confidence level is greater than a preset threshold, the target device is woken up. By judging the confidence level of the audio data, the wake-up performance is ensured, thereby avoiding the use of attention modules for acoustic modeling and improving the real-time performance of voice wake-up. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of this application, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the voice wake-up method provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the wake-up process of the voice wake-up system provided in the embodiments of this application;

[0022] Figure 3 This is a schematic diagram of the acoustic model provided in the embodiments of this application;

[0023] Figure 4 This is a schematic diagram of the structure of the recurrent neural network converter model provided in the embodiments of this application;

[0024] Figure 5 This is one of the structural schematic diagrams of the voice wake-up device provided in the embodiments of this application;

[0025] Figure 6 This is a second schematic diagram of the structure of the voice wake-up device provided in the embodiments of this application;

[0026] Figure 7 This is the third structural schematic diagram of the voice wake-up device provided in the embodiments of this application;

[0027] Figure 8 This is the fourth structural schematic diagram of the voice wake-up device provided in the embodiments of this application;

[0028] Figure 9 This is the fifth schematic diagram of the structure of the voice wake-up device provided in the embodiments of this application;

[0029] Figure 10 This is the sixth schematic diagram of the structure of the voice wake-up device provided in the embodiments of this application;

[0030] Figure 11 This is the seventh schematic diagram of the structure of the voice wake-up device provided in the embodiments of this application;

[0031] Figure 12 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: including A alone, B alone, C alone, and the presence of both A and B, both B and C, both A and C, and the presence of A, B, and C.

[0034] Please see Figure 1 , Figure 1 This is one of the schematic diagrams of the voice wake-up process provided in the embodiments of this application, such as... Figure 1As shown, it includes the following steps:

[0035] Step 101: Obtain audio data and target audio frames in the audio data, wherein the target audio frames include audio frames in the audio data that contain human voices.

[0036] It is understandable that after segmenting and windowing the acquired audio data, multiple audio frames can be obtained. Performing VAD (Voice Activity Detection) on these multiple audio frames can eliminate non-speech frames and obtain the target audio frames containing human voices. Specifically, segmentation and windowing include two processes: segmentation and windowing. Segmentation involves dividing the acquired audio data into fixed-length audio frames, and windowing involves processing each audio frame using a window function to prevent spectral leakage. In this way, when using an acoustic model to identify and judge the audio data, useless audio can be filtered out.

[0037] Step 102: Extract the acoustic features of the target audio frame.

[0038] Specifically, the aforementioned acoustic features can be obtained using the 80-dimensional FBank (Filter Bank) features extracted from the target audio frame. Compared to MFCC (Mel-Frequency Cepstral Coefficients) features, this method can retain more correlation information between different frequencies.

[0039] Step 103: Use a pre-trained neural network model to identify the acoustic features to obtain the identification results of the audio data.

[0040] It is understandable that the audio data used for training the aforementioned neural network model can also be processed by the aforementioned frame segmentation and windowing before being input into the aforementioned neural network model, thereby improving the training effect of the aforementioned neural network model.

[0041] The aforementioned neural network model can be obtained based on the RNN-T (Recurrent Neural Network-Transducer) structure. The RNN-T structure has a small number of parameters, low latency, and low power consumption, making it suitable for voice wake-up tasks.

[0042] Step 104: Obtain the confidence level of the audio data based on the recognition result.

[0043] The confidence level mentioned above can represent the wake-up effect of the wake-up word in the above recognition result. When using a custom wake-up word to wake up the device, the wake-up effect of the wake-up word can be judged based on the confidence level mentioned above. If the wake-up effect of the wake-up word in the above recognition result is good, the wake-up word can be used directly for wake-up.

[0044] Step 105: If the confidence level is greater than a preset threshold, wake up the target device.

[0045] It is understood that if the confidence level is greater than the preset threshold, the wake-up word corresponding to the recognition result is considered to have a good wake-up effect, that is, the wake-up word corresponding to the recognition result can be used directly to wake up the target device.

[0046] The target device mentioned above can be a device that acquires the audio data and performs the steps mentioned above, or it can be other devices, such as the device corresponding to the wake word in the audio data. This application does not limit this.

[0047] In this embodiment, a pre-trained neural network model is used to identify the acoustic features to obtain the identification result of the audio data, and the confidence level of the audio data is obtained based on the identification result. If the confidence level is greater than a preset threshold, the target device is woken up. By judging the confidence level of the audio data, the wake-up performance is ensured, thereby avoiding the use of attention modules for acoustic modeling and improving the real-time performance of voice wake-up.

[0048] Optionally, the neural network model includes a first recurrent neural network transformer model;

[0049] Step 103, which involves using a pre-trained neural network model to identify the acoustic features in order to obtain the recognition result of the audio data, may specifically include:

[0050] The acoustic features are identified using a pre-trained first recurrent neural network transformer model to obtain the pinyin sequence of the audio data.

[0051] In the process of voice wake-up, more attention is paid to the pronunciation of audio data. If the pronunciation is correct, it can be regarded as voice wake-up of the target device.

[0052] In this embodiment, the acoustic features are identified using a pre-trained first recurrent neural network transformer model to obtain the pinyin sequence of the audio data, and the pronunciation accuracy of the audio data can be judged through the pinyin sequence.

[0053] Optionally, the neural network model may further include a second recurrent neural network transformer model;

[0054] Step 103, which involves using a pre-trained neural network model to identify the acoustic features in order to obtain the recognition result of the audio data, may further include:

[0055] The pre-trained second recurrent neural network transformer model is used to recognize the pinyin sequence to obtain the text sequence of the audio data;

[0056] After step 103, where a pre-trained neural network model is used to identify the acoustic features to obtain the recognition result of the audio data, the method may further include the following steps:

[0057] Output the text sequence.

[0058] The second recurrent neural network converter model and the first recurrent neural network converter model can be trained independently. For example, during training, the input of the second recurrent neural network converter model is a pinyin sequence, but the input pinyin sequence can be the pinyin sequence output by the first recurrent neural network converter model, or it can be other pinyin sequences that can be used for training.

[0059] The output can be the display of the text sequence, which converts the audio data into text and displays it on a screen. This can be applied to speech recognition scenarios such as meeting transcription services.

[0060] In this embodiment, the pre-trained second recurrent neural network transformer model is used to recognize the pinyin sequence and output the obtained text sequence. The text sequence of the audio data can be recognized as the recognition result based on the acoustic features.

[0061] Optionally, the recognition result includes a first wake word;

[0062] After step 103, where a pre-trained neural network model is used to identify the acoustic features to obtain the recognition result of the audio data, the method may further include the following steps:

[0063] Get the set of wake words;

[0064] If the first wake-up word includes the second wake-up word, the target device is woken up, wherein the second wake-up word belongs to the set of wake-up words.

[0065] The aforementioned wake word set may include multiple wake words used to train the aforementioned neural network model, and the confidence level of the wake words in the aforementioned wake word set is greater than the aforementioned preset threshold. The aforementioned first wake word can be understood as a user-defined wake word.

[0066] It is understood that the above-mentioned first wake word including the above-mentioned second wake word may include the following situations: the above-mentioned first wake word is the same as the above-mentioned second wake word, or a part of the above-mentioned first wake word is the above-mentioned second wake word. For example, the above-mentioned wake word set includes the second wake word "Xiaotian Xiaotian". When the above-mentioned first wake word is "Xiaotian Xiaotian", "Xiaotian Xiaotian Xiaotian", "Ah Xiaotian Xiaotian", etc., it can be understood that the above-mentioned first wake word includes the above-mentioned second wake word.

[0067] The first wake-up word in the above recognition result can be in the form of the above pinyin sequence or the above text sequence. By comparing the above first wake-up word with the pinyin sequence or text sequence of the wake-up words in the above wake-up word set, it can be determined whether the above first wake-up word includes the wake-up words in the above wake-up word set.

[0068] In this embodiment, if the first wake-up word includes the second wake-up word, it can be understood that the acquired audio data includes wake-up words from the wake-up word set, and the target device can be woken up directly based on the first wake-up word.

[0069] Optionally, the step 104 of obtaining the confidence level of the audio data based on the recognition result may specifically include:

[0070] If the first wake word does not include the second wake word, the confidence level of the audio data is obtained based on the recognition result;

[0071] After obtaining the confidence level of the audio data based on the recognition result in step 104, the method may further include the following steps:

[0072] If the confidence level is less than or equal to the preset threshold, obtain the third wake word in the wake word set that has the highest similarity to the first wake word;

[0073] Output a first message, the first message including replacing the third wake word with the first wake word to wake up the target device;

[0074] Upon receiving a second message, the device is woken up, the second message including consent to replace the third wake word with the first wake word to wake up the target device.

[0075] The third wake-up word can be the wake-up word with the highest pronunciation similarity to the first wake-up word. The first wake-up word can be understood as a user-defined wake-up word. By recognizing the audio data containing the user-defined wake-up word through the neural network model, if the confidence level of the first wake-up word is less than or equal to a preset threshold, it can be recommended to replace the first wake-up word with the third wake-up word to improve the success rate of waking up the device.

[0076] Specifically, when the confidence level of the aforementioned audio data is less than or equal to the aforementioned preset threshold, a voice prompt can be output asking the user whether they agree to replace the aforementioned first wake-up word with the aforementioned third wake-up word. If the user agrees, the user-defined wake-up word is replaced with the aforementioned third wake-up word, and the aforementioned target device is woken up. During the aforementioned wake-up process, the aforementioned output voice is the aforementioned first message, and the aforementioned user agreement is the aforementioned second message. Furthermore, the aforementioned second message can be preset in various forms, such as: voice containing words like "agree," "can," or "through," touch or press, default agreement to replacement if there is no response after 10 seconds, etc. This application does not impose any restrictions on this.

[0077] In this embodiment, when the confidence level is less than or equal to the preset threshold, the third wake-up word with the highest similarity to the first wake-up word in the wake-up word set is obtained, and the third wake-up word is recommended as a replacement wake-up word for the first wake-up word, thereby improving the wake-up effect of the audio data.

[0078] Optionally, when the confidence level is less than or equal to the preset threshold, obtaining the third wake-up word in the wake-up word set that has the highest similarity to the first wake-up word may specifically include:

[0079] If the confidence level is less than or equal to the preset threshold, obtain the first vector corresponding to the first wake-up word, and the second vector set corresponding to each wake-up word in the wake-up word set;

[0080] Determine the third vector in the second vector set that is closest to the first vector;

[0081] Obtain the third wake word corresponding to the third vector.

[0082] Specifically, the conversion between the wake word and the vector can be achieved through a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model. The vector closest to the first vector can be understood as the wake word whose pronunciation is most similar to the first wake word.

[0083] In this embodiment, by determining the third vector in the second vector set that is closest to the first vector, and obtaining the third wake-up word corresponding to the third vector, the third wake-up word that is most similar in pronunciation to the first wake-up word can be quickly obtained.

[0084] Optionally, the recognition result includes posterior probability;

[0085] Step 104, which involves obtaining the confidence level of the audio data based on the recognition result, may specifically include:

[0086] The confidence level of the audio data is obtained based on the posterior probability.

[0087] The aforementioned posterior probability can be the posterior probability corresponding to each character in the aforementioned recognition result, and the aforementioned confidence level can be obtained based on the posterior probability corresponding to each character.

[0088] In this embodiment, the confidence level of the audio data is obtained based on the posterior probability. The confidence level can be obtained through the recognition result of the above neural network model, thereby determining the wake-up effect of the recognition result.

[0089] The various optional implementation methods described in the embodiments of this application can be combined with each other or implemented individually without conflict. The embodiments of this application do not limit this.

[0090] For ease of understanding, a specific example is as follows:

[0091] This application provides an end-to-end voice wake-up system capable of recommending custom wake words. The voice wake-up system includes a wake word customization stage and an inference stage (i.e., the wake-up stage). For example... Figure 2 As shown, the voice wake-up process of the above end-to-end voice wake-up system can specifically include the following steps:

[0092] The input multi-microphone speech is segmented into fixed-length audio frames, and a Hamming window is used to prevent spectrum leakage, which also facilitates network training.

[0093] After being framed and windowed, the audio is sent to the VAD to determine whether each frame is human voice and discard audio frames that are not human voice.

[0094] The speech frames identified as human voices by VAD are used to extract an 80-dimensional FBank as feature input. The input data is represented by Input... (i,80) This indicates that FBank retains more correlation information between different frequencies compared to MFCC, and this correlation information can be used for neural network modeling.

[0095] Input the above 80-dimensional FBank features (i,80) As input to the acoustic model, the pre-trained acoustic model is used to recognize speech results, which include the recognition sequence and its posterior probability.

[0096] The confidence score is obtained based on the speech results and compared with a preset threshold T.

[0097] If score is less than or equal to T, it indicates that the user-defined wake-up word cannot be well adapted to the voice wake-up system. Therefore, the inference algorithm is used to recommend built-in wake-up words with similar pronunciations based on the pronunciation of the user-defined wake-up word. If the user agrees to use the built-in wake-up word, the device is activated.

[0098] If score is greater than T, it indicates that the user-defined wake-up word can be adapted to the voice wake-up system, and the device can be activated according to the recognized voice result.

[0099] Among them, as Figure 3 shown, the voice wake-up system selects Cascade RNN-T as the acoustic model. Both the "Syllable based RNN-T" (pinyin recognition network) and the "Syllable-to-Character Converter" (pinyin-character converter) in Cascade RNN-T are composed of the RNN-T structure. The RNN-T structure is as Figure 4 shown, where M1, M2, M, and N are parameters representing the number of layers. In the voice wake-up system provided in this application, M1 is 2, M2 is 3, M is 2, and N is 2. After the above Input (i,80) is input into the acoustic model, the pinyin sequence and character sequence recognized by Cascade RNN-T can be obtained.

[0100] Among them, the "Syllable based RNN-T" (pinyin recognition network) and the "Syllable-to-Character Converter" (pinyin-character converter) in Cascade RNN-T are independently trained. That is, the training of the "Syllable-to-Character Converter" is an independent process and can be carried out separately from the "Syllable based RNN-T". The "Syllable based RNN-T" takes Input (i,80) as the input and outputs a Syllable sequence (pinyin sequence). The "Syllable-to-Character Converter" takes the Syllable sequence as the input and outputs a Char sequence (character sequence). Specifically, taking the audio content "Hello, Xiaodi" as an example, the pinyin sequence output by the above acoustic model is Syllable = {ni3, hao3, xiao3, di2}, and the character sequence is Char = {你, 好, 小, 迪}.

[0101] The recommendation process for the aforementioned built-in wake-up words can specifically include: using a pre-trained Chinese BERT model to identify the vector VectorU corresponding to the pinyin sequence of the above speech results, and inputting the pinyin sequences of all built-in wake-up words in the voice wake-up system into the BERT model to obtain the corresponding vector VectorL, and calculating the distance Distance between the vector VectorL and the vector VectorU corresponding to each built-in wake-up word: Find the built-in wake word corresponding to the vector VectorL with the smallest distance Distance to vector VectorU among all vectors VectorL, and return the recommended wake word to the user.

[0102] Furthermore, in the aforementioned voice wake-up process, neither device wake-up nor wake-up word recommendation requires the results of the aforementioned text sequence. As long as the pronunciation is correct, voice wake-up can be performed. Therefore, the "Syllable-to-CharacterConverter" part can be extracted according to the application scenario. That is, in the aforementioned voice wake-up system, the output of the acoustic model is the pinyin sequence output by the "Syllable based RNN-T" part. The output recognition result sequence is a 2*m sequence, where m is the number of pinyin characters included during training, and 2 represents the number of elements (token_id, score) per row. token_id represents the aforementioned pinyin sequence, and score represents the score of each element in the pinyin sequence.

[0103] In this embodiment, the voice wake-up system uses Cascade RNN-T as the acoustic model to recognize audio and obtain the confidence level of the audio. When using a user-defined wake-up word, if the confidence level of the user-defined wake-up word is less than or equal to a preset threshold, the system can recommend a built-in wake-up word with the highest pronunciation similarity to the user based on the pronunciation of the user-defined wake-up word, thereby improving the wake-up rate of the voice wake-up system. Furthermore, Cascade RNN-T has a small number of parameters, meaning that the computational load during audio recognition is small, resulting in low power consumption for the hardware.

[0104] In addition, the pinyin sequence recognition and text sequence recognition parts in the acoustic model can be separated to adapt to different application scenarios. For example, the above-mentioned voice wake-up system can be applied to meeting transcription services in conference rooms, intelligent service robot systems, or voice wake-up activation functions of embedded devices, thereby improving the resource reuse level of the voice wake-up system.

[0105] The voice wake-up system includes a wake-up word customization stage and an inference stage (i.e., the wake-up stage). Taking the system's built-in wake-up words "Xiao Tian Xiao Tian" and "Zhang San Zhang San" as an example, and the user wants to customize the wake-up word to "Hello, Xiao Tian", the wake-up word customization stage includes the following process:

[0106] The voice wake-up system collects the user's audio x, and the content of audio x is "Hello, Xiaotian". Then the pinyin sequence Syllable = {ni3, hao3, xiao3, tian1}, and the character sequence Char = {你, 好, 小, 天}. Set the confidence level T = 0.6.

[0107] Next, perform VAD judgment on audio x to remove the interference of non-human voices.

[0108] Extract 80-dimensional Fbank features from the processed audio x above. Here, use Input_i to represent the Fbank features, where i represents the i-th frame.

[0109] Input Input_i into "Syllable based RNN-T" in Cascade RNN-T as acoustic features. Cascade RNN-T will return the Syllable sequence as the result and the score score (assuming score = 0.4). Since this wake-up word is user-defined, it can be judged that score < T. Therefore, the system believes that the performance of the user-defined wake-up word is not good and is not suitable for voice wake-up. Next, enter the wake-up word recommendation process.

[0110] Put the pinyin sequences of the three wake-up words "Hello Xiaotian", "Xiaotian Xiaotian", and "Zhang San Zhang San", which are {ni3, hao3, xiao3, tian1}, {xiao3, tian1, xiao3, tian1}, and {zhang1, san1, zhang1, san1} respectively, into BERT to calculate the word vectors of each Syllable sequence.

[0111] After calculating the distances using the above vector distance calculation formula, the voice wake-up system finds that the built-in wake-up word "Xiaotian Xiaotian" is the closest in pronunciation to the user-defined wake-up word "Hello Xiaotian". Therefore, the voice wake-up system returns "Xiaotian Xiaotian" as the wake-up word.

[0112] The user decides whether the "Xiaotian Xiaotian" returned by the voice wake-up system can be used as an alternative wake-up word to complete the subsequent wake-up work. If the user accepts it, then directly activate the voice system to complete the wake-up process. Otherwise, return to the audio acceptance stage to perform the next round of wake-up word customization reception work.

[0113] The inference stage includes the following process:

[0114] Since the user has completed the customization of the wake-up word and selected the final wake-up word, here assume the wake-up word is "Xiaotian Xiaotian".

[0115] The voice wake-up system collects the user's audio x, and the audio content is "Xiaotian Xiaotian". Then the pinyin sequence Syllable = {xiao3, tian1, xiao3, tian1}, and Char = {小, 天, 小, 天}. The confidence level T is set to 0.6.

[0116] Next, perform VAD judgment on the audio x to remove the interference of non-human voices.

[0117] Extract the 80-dimensional Fbank features of the processed x above, which is represented by Input_i here, where i represents the i-th frame.

[0118] Input Input_i into the "Syllable based RNN-T" in the Cascade RNN-T as the AcousticFeature (acoustic feature), which will return the Syllable sequence as the result and the score score (assuming score = 0.8). Since score > T, the device is activated and the voice wake-up task ends.

[0119] Please refer to Figure 5 , Figure 5 which is one of the structural schematic diagrams of the voice wake-up device provided by the embodiments of the present application. As Figure 5 shown, the voice wake-up device 500 includes:

[0120] The first acquisition module 501 is used to acquire audio data and the target audio frames in the audio data, and the target audio frames include the audio frames containing human voices in the audio data;

[0121] The extraction module 502 is used to extract the acoustic features of the target audio frames;

[0122] The recognition module 503 is used to recognize the acoustic features using a pre-trained neural network model to obtain the recognition result of the audio data;

[0123] The second acquisition module 504 is used to obtain the confidence level of the audio data based on the recognition result;

[0124] The first wake-up module 505 is used to wake up the target device when the confidence level is greater than a preset threshold.

[0125] Optionally, as Figure 6 shown, the neural network model includes a first recurrent neural network transducer model;

[0126] Specifically, the recognition module 503 may include:

[0127] The first recognition unit 5031 is used to recognize the acoustic features using a pre-trained first recurrent neural network transformer model to obtain the pinyin sequence of the audio data.

[0128] Optional, such as Figure 7 As shown, the neural network model also includes a second recurrent neural network converter model;

[0129] The identification module 503 may specifically include:

[0130] The second recognition unit 5032 is used to recognize the pinyin sequence using a pre-trained second recurrent neural network transformer model to obtain the text sequence of the audio data;

[0131] The voice wake-up device 500 may further include:

[0132] The first output module 506 is used to output the text sequence.

[0133] Optional, such as Figure 8 As shown, the recognition result includes a first wake word;

[0134] The voice wake-up device 500 may further include:

[0135] The third acquisition module 507 is used to acquire the set of wake words;

[0136] The second wake-up module 508 is used to wake up the target device if the first wake-up word includes the second wake-up word, wherein the second wake-up word belongs to the set of wake-up words.

[0137] Optional, such as Figure 9 As shown, the second acquisition module 504 may specifically include:

[0138] The first acquisition unit 5041 is used to acquire the confidence level of the audio data based on the recognition result if the first wake-up word does not include the second wake-up word.

[0139] The voice wake-up device 500 may further include:

[0140] The fourth acquisition module 509 is used to acquire the third wake word in the wake word set that has the highest similarity to the first wake word when the confidence level is less than or equal to the preset threshold.

[0141] The second output module 510 is used to output a first message, the first message including replacing the third wake-up word with the first wake-up word to wake up the device;

[0142] The third wake-up module 511 is used to wake up the device upon receiving a second message, the second message including an agreement to replace the third wake-up word with the first wake-up word to wake up the device.

[0143] Optional, such as Figure 10 As shown, the fourth acquisition module 509 may specifically include:

[0144] The second acquisition unit 5091 is used to acquire, when the confidence level is less than or equal to the preset threshold, the first vector corresponding to the first wake-up word and the second vector set corresponding to each wake-up word in the wake-up word set;

[0145] Determining unit 5092 is used to determine the third vector in the second vector set that is closest to the first vector;

[0146] The third acquisition unit 5093 is used to acquire the third wake-up word corresponding to the third vector.

[0147] Optional, such as Figure 11 As shown, the recognition result includes the posterior probability;

[0148] The second acquisition module 504 may specifically include:

[0149] The fourth acquisition unit 5042 is used to acquire the confidence level of the audio data based on the posterior probability.

[0150] The voice wake-up device 500 can achieve the functions described in the embodiments of this application. Figure 1 The various processes in the method embodiments, and the ways to achieve the same beneficial effects, will not be repeated here to avoid repetition.

[0151] This application also provides an electronic device. Please refer to [link to relevant documentation]. Figure 12 The electronic device 1200 includes a processor 1201, a memory 1202, and a program or instructions stored in the memory 1202 and executable on the processor 1201. When the program or instructions are executed by the processor 1201, they implement the various processes of the above-described image recognition method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0152] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware related to program instructions, and the program can be stored in a readable medium.

[0153] This application also provides a computer-readable storage medium for storing a program. The computer-readable storage medium stores a computer program, which, when executed by a processor, can perform the above-described functions. Figure 1Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0154] The storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0155] The above description represents the preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A voice wake-up method, characterized in that, include: Acquire audio data and target audio frames in the audio data, wherein the target audio frames include audio frames in the audio data that contain human voices; Extract the acoustic features of the target audio frame; The acoustic features are identified using a pre-trained neural network model to obtain the identification results of the audio data; The confidence level of the audio data is obtained based on the recognition result; If the confidence level is greater than a preset threshold, wake up the target device; The recognition result includes a first wake word; After using a pre-trained neural network model to identify the acoustic features to obtain the recognition result of the audio data, the method further includes: Get the set of wake words; If the first wake-up word includes the second wake-up word, the target device is woken up, wherein the second wake-up word belongs to the set of wake-up words; The step of obtaining the confidence level of the audio data based on the recognition result includes: If the first wake word does not include the second wake word, the confidence level of the audio data is obtained based on the recognition result; After obtaining the confidence level of the audio data based on the recognition result, the method further includes: If the confidence level is less than or equal to the preset threshold, obtain the third wake word in the wake word set that has the highest similarity to the first wake word; Output a first message, the first message including replacing the third wake word with the first wake word to wake up the target device; Upon receiving a second message, the device is woken up, the second message including consent to replace the third wake word with the first wake word to wake up the target device.

2. The method as described in claim 1, characterized in that, The neural network model includes a first recurrent neural network converter model; The step of using a pre-trained neural network model to identify the acoustic features to obtain the identification results of the audio data includes: The acoustic features are identified using a pre-trained first recurrent neural network transformer model to obtain the pinyin sequence of the audio data.

3. The method as described in claim 2, characterized in that, The neural network model also includes a second recurrent neural network converter model; The step of using a pre-trained neural network model to identify the acoustic features in order to obtain the identification result of the audio data further includes: The pre-trained second recurrent neural network transformer model is used to recognize the pinyin sequence to obtain the text sequence of the audio data; After using a pre-trained neural network model to identify the acoustic features to obtain the recognition result of the audio data, the method further includes: Output the text sequence.

4. The method as described in claim 1, characterized in that, When the confidence level is less than or equal to the preset threshold, obtaining the third wake-up word in the wake-up word set that has the highest similarity to the first wake-up word includes: If the confidence level is less than or equal to the preset threshold, obtain the first vector corresponding to the first wake-up word, and the second vector set corresponding to each wake-up word in the wake-up word set; Determine the third vector in the second vector set that is closest to the first vector; Obtain the third wake word corresponding to the third vector.

5. The method as described in claim 1, characterized in that, The identification result includes the posterior probability; The step of obtaining the confidence level of the audio data based on the recognition result includes: The confidence level of the audio data is obtained based on the posterior probability.

6. A voice wake-up device, characterized in that, include: The first acquisition module is used to acquire audio data and target audio frames in the audio data, wherein the target audio frames include audio frames in the audio data that contain human voices; An extraction module is used to extract the acoustic features of the target audio frame; The recognition module is used to recognize the acoustic features using a pre-trained neural network model to obtain the recognition result of the audio data; The second acquisition module is used to acquire the confidence level of the audio data based on the recognition result; The first wake-up module is used to wake up the target device when the confidence level is greater than a preset threshold. The recognition result includes a first wake word; The third acquisition module is used to acquire the set of wake words; The second wake-up module is used to wake up the target device if the first wake-up word includes the second wake-up word, wherein the second wake-up word belongs to the wake-up word set; The second acquisition module specifically includes: The first acquisition unit is configured to acquire the confidence level of the audio data based on the recognition result if the first wake-up word does not include the second wake-up word. The voice wake-up device also includes: The fourth acquisition module is used to acquire the third wake word in the wake word set that has the highest similarity to the first wake word when the confidence level is less than or equal to the preset threshold. The second output module is used to output a first message, the first message including replacing the third wake-up word with the first wake-up word to wake up the device; A third wake-up module is used to wake up the device upon receiving a second message, the second message including consent to replace the third wake-up word with the first wake-up word to wake up the device.

7. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and running on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of the voice wake-up method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the voice wake-up method as described in any one of claims 1 to 5.

Images