Voice wake-up model training, wake-up method, device, equipment and storage medium

Abstract

The present disclosure provides a training and wake-up method, apparatus, device and storage medium for a voice wake-up model, which relates to the field of artificial intelligence, and in particular to the fields of deep learning and voice technology. The specific implementation scheme is as follows: obtaining created speech recognition training data and voice wake-up training data, first training the basic model according to the speech recognition training data, and obtaining the model parameters of the basic model when the model loss function converges; then updating the configuration parameters of the decoding module in the basic model based on the model configuration instructions to obtain a first model; then training the first model according to the voice wake-up training data, and obtaining a trained voice wake-up model when the model loss function converges. The above scheme can improve the convergence speed of the voice wake-up model training, and processing and analyzing the audio data based on the above voice wake-up model can improve the recognition accuracy and reduce the false alarm rate.

Description

Technical Field

[0001] The present disclosure relates to the fields of deep learning and speech technology in the field of artificial intelligence, and in particular to a training, wake-up method, apparatus, device, and storage medium for a speech wake-up model. Background Art

[0002] With the development of artificial intelligence, more and more electronic devices are beginning to support voice interaction. Voice wake-up, as the switch for voice interaction, plays a crucial role. Currently, voice wake-up functionality has the problem of waking up multiple devices simultaneously. For example, devices of the same brand often support the same wake-up word, resulting in the awkward situation of multiple devices responding to a single wake-up. Summary of the Invention

[0003] The present disclosure provides a training and wake-up method, apparatus, device, and storage medium for a voice wake-up model.

[0004] According to a first aspect of the present disclosure, a method for training a voice wake-up model is provided, comprising:

[0005] Acquiring speech recognition training data, performing speech recognition training on a basic model according to the speech recognition training data, and obtaining model parameters of the basic model when the model loss function converges; the basic model includes an encoding module and a decoding module;

[0006] In response to a model configuration instruction initiated by a user, updating configuration parameters of a decoding module in the base model based on model parameters of the base model to obtain a first model;

[0007] Acquire voice wake-up training data, and perform voice wake-up training on the first model according to the voice wake-up training data to obtain the first model when the model loss function converges;

[0008] The first model when the model loss function converges is used as the voice wake-up model.

[0009] According to a second aspect of the present disclosure, a voice wake-up method is provided, comprising:

[0010] Receive audio data input by the user;

[0011] Perform feature extraction on the audio data to obtain filter bank FBank features corresponding to the audio data;

[0012] The encoding module based on the voice wake-up model encodes the FBank features corresponding to the audio data to obtain a feature coding sequence corresponding to the audio data; the connection time series classification (CTC) decoding is used to determine the target feature coding sequence in the feature coding sequence with a score greater than or equal to a preset value;

[0013] Get the semantic label sequence corresponding to the user-defined wake-up word;

[0014] Based on the decoding module of the voice wake-up model, the target feature coding sequence and the semantic label sequence are decoded and analyzed to determine whether to wake up the terminal device.

[0015] According to a third aspect of the present disclosure, a training device for a voice wake-up model is provided, comprising:

[0016] A first training module is used to perform speech recognition training on a basic model according to the speech recognition training data to obtain model parameters of the basic model when the model loss function converges; the basic model includes an encoding module and a decoding module;

[0017] a model configuration module, configured to update configuration parameters of a decoding module in the base model based on model parameters of the base model in response to a model configuration instruction initiated by a user, to obtain a first model;

[0018] The second acquisition module is used to obtain voice wake-up training data;

[0019] A second training module is configured to perform voice wake-up training on the first model according to the voice wake-up training data to obtain a first model when a model loss function converges;

[0020] The model generation module is used to use the first model when the model loss function converges as the voice wake-up model.

[0021] According to a fourth aspect of the present disclosure, a voice wake-up device is provided, comprising:

[0022] A receiving module, configured to receive audio data input by a user;

[0023] A feature extraction module is used to extract features from the audio data to obtain FBank features corresponding to the audio data;

[0024] A first processing module is configured to encode the FBank features corresponding to the audio data based on the encoding module of the voice wake-up model to obtain a feature encoding sequence corresponding to the audio data;

[0025] The second processing module is used to use CTC decoding to determine a target feature code sequence with a score greater than or equal to a preset value in the feature code sequence;

[0026] The acquisition module is used to obtain the semantic label sequence corresponding to the user-defined wake-up word;

[0027] The third processing module is used to decode and analyze the target feature coding sequence and the semantic label sequence based on the decoding module of the voice wake-up model to determine whether to wake up the terminal device.

[0028] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0029] at least one processor; and

[0030] a memory communicatively connected to the at least one processor; wherein,

[0031] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect or the second aspect.

[0032] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect or the second aspect.

[0033] According to the seventh aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect or the second aspect.

[0034] The technology disclosed herein improves the accuracy of customized voice wake-up.

[0035] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0037] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0038] Figure 2 A schematic diagram of the structure of a voice wake-up model provided in an embodiment of the present disclosure;

[0039] Figure 3 A flowchart of a method for training a voice wake-up model provided in an embodiment of the present disclosure;

[0040] Figure 4A schematic diagram of the process of creating speech recognition training data provided in an embodiment of the present disclosure;

[0041] Figure 5 A schematic diagram of the process of creating voice wake-up training data provided by an embodiment of the present disclosure;

[0042] Figure 6 A flowchart of the voice wake-up method provided in an embodiment of the present disclosure;

[0043] Figure 7 A schematic diagram of the structure of a voice wake-up model provided in an embodiment of the present disclosure;

[0044] Figure 8 A schematic diagram of the structure of a training device for a voice wake-up model provided in an embodiment of the present disclosure;

[0045] Figure 9 A schematic diagram of the structure of a voice wake-up device provided in an embodiment of the present disclosure;

[0046] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0048] Custom wake-up words can solve the problem of multiple devices waking up simultaneously. For example, users can set different custom wake-up words for different devices based on their needs. However, voice wake-up based on custom wake-up words is not very accurate.

[0049] The present disclosure provides a training method for a voice wake-up model and a voice wake-up method, which are applied to the fields of deep learning and voice technology in the field of artificial intelligence to improve the accuracy of customized voice wake-up.

[0050] In order to facilitate the understanding of the technical solutions provided by the present disclosure, first Figure 1 The application scenarios of the embodiments of the present disclosure are described.

[0051] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present disclosure. Figure 1 As shown, this application scenario involves the model training phase and the model application phase.

[0052] During the model training phase, the model training device trains a first model based on a first training dataset. The first training dataset includes training data for speech recognition, and the first model has speech recognition capabilities. A second model is obtained by adjusting model configuration parameters of the first model. The model training device trains a second model based on a second training dataset. The second training dataset includes training data for voice wake-up, and the second model has voice wake-up capabilities. The trained second model is used as the final voice wake-up model.

[0053] It should be noted that the model structures of the first model and the second model are consistent. The difference between the two is the different parameter configurations of the model output part, such as different dimension parameters.

[0054] It can be understood that the first model has a speech recognition function, and its output is the syllable information corresponding to the audio data, such as a sequence of untoned syllables. The second model has a voice wake-up function, and its output is a binary classification result of whether to wake up or not. Therefore, the dimensional parameters of the model output of the first and second models are different.

[0055] In the model application stage, the voice wake-up device is pre-installed with the trained second model, i.e. Figure 1 The voice wake-up model in the voice wake-up device receives the audio data input by the user, and finally obtains the result of whether to wake up after the audio data is preprocessed and processed and analyzed by the voice wake-up model.

[0056] In the disclosed embodiments, the voice wake-up model may adopt an encoder-decoder architecture, or other model structures including an encoder and a decoder, wherein the encoder may also be referred to as an encoding module, and the decoder may also be referred to as a decoding module.

[0057] The following combination Figure 2 A detailed description of the internal structure of the voice wake-up model.

[0058] Figure 2 This is a structural diagram of a voice wake-up model provided by an embodiment of the present disclosure. Figure 2 As shown, the voice wake-up model provided in this embodiment includes an encoding module and a decoding module.

[0059] Among them, the encoding module includes a convolutional neural network (CNN) module and a recurrent neural network (RNN) module. Figure 2The diagram shows two CNN modules and two RNN modules. The encoding module processes data through two CNN modules and then two RNN modules. The encoding module encodes audio features to generate encoded feature data.

[0060] Among them, the decoding module includes an attention mechanism module, an RNN module, a full connection module and a normalization (softmax) module. Figure 2 The figure shows two RNN modules and the data processing process of the decoding module: one path of data is input into the attention mechanism module through the RNN module, and the other path of data is directly input into the attention mechanism module. After being processed by the attention mechanism module, it passes through the RNN module, the fully connected module and the normalization module, and finally outputs the result of whether to wake up.

[0061] Optionally, in some embodiments, the RNN module may be replaced by a Long Short-Term Memory (LSTM) module.

[0062] It should be noted that the voice wake-up model includes two input data sources: the first input data is the feature sequence corresponding to the audio data, such as filter bank (FBank) features, Mel-frequency cepstral coefficients (MFCC) features, etc. The second input data is the semantic label sequence corresponding to the audio data.

[0063] It should be noted that the model training device or voice wake-up device in the embodiments of the present disclosure can be a terminal device, a server or a virtual machine, etc., or a distributed computer system composed of one or more servers and / or computers.

[0064] The terminal device includes, but is not limited to, smartphones, laptops, desktop computers, platform computers, in-vehicle devices, smart wearable devices, and the like, and is not limited in the present embodiment. The server can be a regular server or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product within a cloud computing service system. The server can also be a server for a distributed system or a server integrated with blockchain.

[0065] In the disclosed embodiment, a training data set for two rounds of model training is first constructed, that is, a training data set for speech recognition and a training data set for speech wake-up are constructed. Based on the basic model and the training data set for speech recognition, speech recognition is pre-trained first. After the model configuration parameters are adjusted, speech wake-up is secondary trained based on the model obtained from the pre-training and the training data set for speech wake-up, and finally a speech wake-up model that can be used to recognize user-defined wake-up words is generated. The speech wake-up model can adopt an encoder-decoder structure, and an attention mechanism is used in the decoder part. The speech wake-up model obtained by the above training process improves the recognition accuracy of the customized speech wake-up model compared to the existing wake-up scheme. While reducing the false alarm rate, the recognition accuracy is close to the level of customized wake-up, which can meet the wake-up needs of scenarios including in-vehicle and smart home.

[0066] Next, combined with the above Figure 1 The application scenarios shown are used to describe the technical solutions of the present disclosure in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0067] Figure 3 A flow chart of a method for training a voice wake-up model provided in an embodiment of the present disclosure. Figure 1 The model training device in the example is used as the execution subject to explain. Figure 3 As shown, the training method of the voice wake-up model may include the following steps:

[0068] Step 301: Acquire speech recognition training data.

[0069] In this embodiment, the speech recognition training data includes FBank features, semantic label sequences, and syllable sequences corresponding to the first audio data. The first audio data is any audio data input by the user that contains a custom wake-up word.

[0070] The FBank features corresponding to the first audio data are obtained by extracting features from the first audio data. The semantic tag sequence corresponding to the first audio data is used to indicate semantic information of the first audio data. The syllable sequence corresponding to the first audio data is a frame-level aligned atonal syllable sequence.

[0071] Step 302: Perform speech recognition training on the basic model according to the speech recognition training data to obtain model parameters of the basic model when the model loss function converges.

[0072] In this embodiment, the FBank features corresponding to the first audio data in the speech recognition training data and the semantic label sequence corresponding to the first audio data are used as the input of the basic model, and the syllable sequence corresponding to the first audio data in the speech recognition training data is used as the output of the basic model to perform model training. When the basic model loss function converges, the model parameters at the time of convergence of the basic model loss function are obtained. Among them, the internal structure of the basic model can be referred to Figure 2 , I will not go into details here.

[0073] Step 303: In response to the model configuration instruction initiated by the user, the configuration parameters of the decoding module in the basic model are updated based on the model parameters of the basic model to obtain a first model.

[0074] The configuration parameters include the output dimension of the model. It should be noted that other parameters of the updated model can be set according to actual needs, and this embodiment does not impose any restrictions on this.

[0075] As an example, updating the configuration parameters of the decoding module in the base model includes modifying the output dimension of the decoding module at the output end of the base model. Because the output results of the voice wake-up task include both wake-up and non-wake-up results, the output dimension of the decoding module at the output end of the base model needs to be modified to two dimensions.

[0076] The above modification logic can be written into the configuration information of the device. Based on the device configuration information, the configuration parameters of the decoding module in the trained basic model are updated on the basis of the trained basic model.

[0077] Step 304: Acquire voice wake-up training data.

[0078] In this embodiment, the voice wake-up training data includes positive training data and negative training data, wherein the positive training data includes the FBank features corresponding to the first audio data, the semantic label sequence and the wake-up label, and the negative training data includes the FBank features corresponding to the first audio data, the randomly generated semantic label sequence and the non-wake-up label.

[0079] The positive training data for voice wake-up training data is constructed based on the speech recognition training data. Unlike speech recognition training data, the positive training data for voice wake-up training data replaces the syllable sequence corresponding to the first audio data with the wake-up label. During the voice wake-up training phase, the first audio data is also called the positive audio data.

[0080] The audio data corresponding to the negative training data for voice wake-up training data is also user-entered audio data (i.e., negative audio data), except that this audio data does not contain the user-defined wake-up word. It should be noted that a semantic label sequence can be randomly generated for the negative audio data, as long as the semantic label sequence is different from the semantic label sequence corresponding to the negative audio data.

[0081] Step 305: Perform voice wake-up training on the first model according to the voice wake-up training data to obtain the first model when the model loss function converges.

[0082] In this embodiment, the FBank features corresponding to the first audio data in the voice wake-up training data and the semantic label sequence corresponding to the first audio data are used as the input of the first model, the wake-up label is used as the output of the first model, and the FBank features corresponding to the first audio data in the voice wake-up training data and the randomly generated semantic label sequence are used as the input of the first model, and the non-wake-up label is used as the output of the first model to perform model training. When the first model loss function converges, the model parameters when the first model loss function converges are obtained. Among them, the internal structure of the first model can refer to Figure 2 , I will not go into details here.

[0083] It should be noted that the same loss function can be used when training the base model and the first model. In terms of model structure, both the base model and the first model include an encoding module and a decoding module. The encoding module corresponds to one loss function, and the decoding module corresponds to one loss function. Therefore, the base model and the first model each include two loss functions.

[0084] As an example, the loss function corresponding to the encoding module of the basic model is the same as the loss function corresponding to the encoding module of the first model, for example, both use the connectionist temporal classification (ctc) loss function; the loss function corresponding to the decoding module of the basic model is the same as the loss function corresponding to the decoding module of the first model, for example, both use the cross-entropy error (ce) loss function.

[0085] Optionally, in some embodiments, the loss function corresponding to the encoding module of the base model may be different from the loss function corresponding to the encoding module of the first model, and the loss function corresponding to the decoding module of the base model may be different from the loss function corresponding to the decoding module of the first model.

[0086] Step 306: Use the first model when the model loss function converges as the voice wake-up model.

[0087] The training method of the voice wake-up model shown in this embodiment obtains the created voice recognition training data and voice wake-up training data, first performs voice recognition training on the basic model according to the voice recognition training data, and obtains the model parameters of the basic model when the model loss function converges; then, based on the model configuration instructions initiated by the user, updates the configuration parameters of the decoding module in the basic model to obtain the first model; finally, performs voice wake-up training on the first model according to the voice wake-up training data, and obtains the first model when the model loss function converges, which is used as the final voice wake-up model. The above-mentioned model training scheme is based on the model parameters of the voice recognition training. By adjusting the configuration parameters of the model decoding module and performing voice wake-up training, the convergence speed of the voice wake-up model training can be improved, the recognition accuracy of the voice wake-up model can be improved, and the false alarm rate can be reduced.

[0088] Optionally, in some embodiments, in response to a user-initiated model configuration instruction, updating configuration parameters of a decoding module in the base model based on model parameters of the base model to obtain the first model includes: in response to the user-initiated model configuration instruction, updating configuration parameters of a fully connected submodule and a normalization submodule of the decoding module in the base model based on model parameters of the base model to obtain the first model. As an example, updating the configuration parameters of the fully connected submodule and the normalization submodule of the decoding module in the base model includes: updating the output dimension of the fully connected submodule and the normalization submodule of the decoding module in the base model to two dimensions.

[0089] This embodiment adjusts the model parameters of the basic model to adjust the model function from the speech recognition function to the speech wake-up function, and then undergoes a second round of model training, namely speech wake-up training, to achieve the speech wake-up purpose.

[0090] Optionally, in some embodiments, the loss function of the base model and the loss function of the first model both include a ctc loss function and a ce loss function.

[0091] The ctc loss function is used to train the encoding module of the base model or the first model.

[0092] ce loss function, used to train the base model or the decoding module of the first model.

[0093] In an optional embodiment of this embodiment, speech recognition training is performed on the basic model according to speech recognition training data to obtain model parameters of the basic model when the model loss function converges, including: jointly training the encoding module and decoding module of the basic model according to the speech recognition training data; when the CTC loss function corresponding to the encoding module and the CE loss function corresponding to the decoding module both converge, the model parameters of the basic model are obtained.

[0094] In an optional embodiment of this embodiment, based on the voice wake-up training data, the first model is trained for voice wake-up to obtain the first model when the model loss function converges, including: jointly training the encoding module and the decoding module of the first model according to the voice wake-up training data; when the ctc loss function corresponding to the encoding module and the ce loss function corresponding to the decoding module both converge, the first model is obtained.

[0095] In this embodiment, based on the above two loss functions, the basic model and the first model are respectively trained for model training, which can accelerate the convergence speed of model training.

[0096] Figure 4 It is a schematic flow diagram of creating speech recognition training data provided by an embodiment of the present disclosure. This process can take the Figure 1 model training device in as the execution subject, or can take other devices independent of the model training device as the execution subject. As Figure 4 shown, the process of creating speech recognition training data can include the following steps:

[0097] Step 401, receive the first audio data input by the user, where the first audio data is audio data containing a custom wake-up word.

[0098] Step 402, perform feature extraction on the first audio data to obtain the FBank features corresponding to the first audio data.

[0099] In this embodiment, performing feature extraction on the first audio data means performing feature extraction on each frame of the first audio data. The first audio data obtained after frame division is a time-domain signal. In order to extract FBank features, it is first necessary to convert the time-domain signal into a frequency-domain signal, and the signal can be converted from the time domain to the frequency domain through Fourier transform. After completing the Fourier transform, the obtained is a frequency-domain signal, and the energy sizes of each frequency band range are different, and the energy spectra of different phonemes are different. Then, through Mel filtering and logarithmic operation, the FBank features corresponding to the first audio data are obtained.

[0100] Step 403, obtain the semantic label sequence and syllable sequence corresponding to the first audio data.

[0101] In this embodiment, by performing semantic recognition on the first audio data, semantic information is obtained, and based on a preset label library, the semantic label sequence corresponding to the semantic information is obtained, that is, the semantic label sequence corresponding to the first audio data.

[0102] The label library includes the numbers corresponding to each Chinese character, that is, the correspondence between each Chinese character and the number. For example, "you" and "hao" respectively correspond to 8 and 9.

[0103] Exemplarily, after determining the semantic information corresponding to the audio data, a numerical sequence corresponding to the semantic information can be generated based on the tag library. This numerical sequence is the semantic tag sequence. For example, the semantic information obtained through semantic recognition is "Xiaoman, how is the weather today?" In the tag library, "Xiao" corresponds to 0, "Man" corresponds to 7, "Jin" corresponds to 2, "Tian" corresponds to 1, "Qi" corresponds to 3, "Ru" corresponds to 5, and "He" corresponds to 6. The corresponding semantic tag sequence that can be generated is {0,7,2,1,1,3,5,6}. In this example, the user-defined wake-up word is "Xiaoman".

[0104] In this embodiment, by performing frame-level analysis on the first audio data, the syllable corresponding to each frame in the first audio data is obtained, and the syllable sequence corresponding to the first audio data is obtained based on a preset method.

[0105] The tag library includes the numbers corresponding to the syllables of each Chinese character, that is, the correspondence between the syllables of each Chinese character and the numbers, for example, the syllables "ni" and "hao" correspond to 8 and 9 respectively.

[0106] For example, after determining the syllable corresponding to each frame in the audio data, a syllable sequence corresponding to the audio data can be generated based on the label library. For example, the syllable corresponding to "xiaoman" in the audio data is xiaoxiaoxiaomanman, and the syllables "xiao" and "man" in the label library correspond to 0 and 7 respectively. The corresponding syllable sequence that can be generated is {0,0,0,7,7}.

[0107] It should be noted that step 402 and step 403 may be performed sequentially or simultaneously, and this embodiment does not impose any limitation on this.

[0108] Step 404: Use the FBank features, semantic label sequence, and syllable sequence corresponding to the first audio data as a set of training data for speech recognition training.

[0109] In this embodiment, the FBank features corresponding to the first audio data serve as the input of the encoding module of the basic model, the semantic label sequence corresponding to the first audio data serves as the input of the decoding module of the basic model, and the syllable sequence corresponding to the first audio data serves as the output of the decoding module of the basic model.

[0110] Optionally, in some embodiments, MFCC features corresponding to the first audio data may be extracted, and the MFCC features, semantic label sequence, and syllable sequence corresponding to the first audio data may be used as a set of training data for speech recognition training.

[0111] Among them, MFCC features are discrete cosine transform (DCT) performed on the basis of FBank features. Compared with FBank features, MFCC features have better discrimination but require more calculation.

[0112] The speech recognition training data shown in this embodiment includes multiple sets of training data, each of which includes user-defined audio data, a sequence of semantic tags corresponding to the audio data, and a sequence of syllables. By training the basic model, the model can accurately recognize user-defined audio data, improving the recognition effect of customized audio.

[0113] Figure 5 This is a flow chart of creating voice wake-up training data provided by the embodiment of the present disclosure. Figure 1 The model training device in the example can be used as the execution subject, or other devices independent of the model training device can be used as the execution subject. Figure 5 As shown, the process of creating voice wake-up training data may include the following steps:

[0114] Step 501: Use the FBank features, semantic label sequence, and wake-up label corresponding to the first audio data input by the user as a set of positive data for voice wake-up training. The first audio data is audio data containing a custom wake-up word.

[0115] In this embodiment, based on the existing data in the speech recognition training data, positive example data of the speech wake-up training data is created. It is only necessary to replace the syllable sequence corresponding to the first audio data with the wake-up label to obtain a set of positive example data.

[0116] For any set of positive example data, the FBank feature corresponding to the first audio data serves as the input of the encoding module of the first model, the semantic label sequence corresponding to the first audio data serves as the input of the decoding module of the first model, and the wake-up label serves as the output of the decoding module of the first model.

[0117] Step 502: Receive second audio data input by the user.

[0118] In this embodiment, the second audio data is audio data that does not include a user-defined wake-up word.

[0119] Step 503: Extract features from the second audio data to obtain FBank features corresponding to the second audio data. The method for extracting features from the second audio data can refer to step 402 of the above embodiment and will not be repeated here.

[0120] Step 504: Use the FBank features corresponding to the second audio data, the randomly generated semantic label sequence, and the no-wake-up label as a set of negative example data for speech wake-up training.

[0121] For any set of negative example data, the FBank features corresponding to the second audio data are used as the input of the encoding module of the first model, the randomly generated semantic label sequence is used as the input of the decoding module of the first model, and the non-awakening label is used as the output of the decoding module of the first model.

[0122] The randomly generated semantic label sequence is different from the semantic label sequence corresponding to the second audio data. For example, based on the example of step 403, negative training data is constructed. For example, the semantic information of the negative audio data is "How is the weather today?", and it does not include the user-defined wake-up word "Xiaoman". The semantic label sequence corresponding to the negative audio data is {2, 1, 1, 3, 5, 6}. The randomly generated semantic label sequence only needs to be different from {2, 1, 1, 3, 5, 6}.

[0123] It should be understood that the semantic label sequence corresponding to the semantic information of the positive audio data, such as {0, 7, 2, 1, 1, 3, 5, 6} in the example of step 403, can also be used as the semantic label sequence in a set of negative training data constructed based on the negative audio data.

[0124] Optionally, in some embodiments, MFCC features corresponding to the first audio data / second audio data may also be extracted. The MFCC features, semantic label sequence, and wake-up label corresponding to the first audio data are used as a set of positive data for voice wake-up training, and the MFCC features, randomly generated semantic label sequence, and non-wake-up label corresponding to the second audio data are used as a set of negative data for voice wake-up training.

[0125] The voice wake-up training data shown in this embodiment includes multiple sets of positive data and multiple sets of negative data. Each set of positive data includes user-defined audio data, a corresponding semantic tag sequence, and a wake-up tag. Each set of negative data includes audio data without the user-defined wake-up word, a randomly generated semantic tag sequence, and a no-wake-up tag. Based on the voice wake-up training data, the first model is trained to accurately determine whether to wake the device, improving the wake-up recognition effect for customized audio.

[0126] Based on the above embodiments, the trained voice wake-up model can be preset in the voice wake-up device, such as smart speakers, TVs, mobile phones and other devices, so that the device has the function of customized voice wake-up. Figure 6 Describes the data processing process of the voice wake-up device.

[0127] Figure 6 This is a flow chart of the voice wake-up method provided by the embodiment of the present disclosure. Figure 1 The voice wake-up device in the system is the execution subject. Figure 6As shown, the voice wake-up method may include the following steps:

[0128] Step 601: Receive audio data input by a user.

[0129] Step 602: Extract features from the audio data to obtain FBank features corresponding to the audio data. The method for extracting features from the audio data can refer to step 402 of the above embodiment and will not be repeated here.

[0130] Step 603: The encoding module based on the voice wake-up model encodes the FBank features corresponding to the audio data to obtain a feature encoding sequence corresponding to the audio data.

[0131] In this embodiment, the FBank features corresponding to the audio data are used as the input of the encoding module of the voice wake-up model. Figure 2 , the Fbank features corresponding to the audio data are input into the CNN module at the bottom layer of the encoding module. After being processed by two CNN modules and two RNN modules, the feature coding sequence corresponding to the audio data is output.

[0132] Step 604: Use CTC decoding to determine a target feature code sequence in the feature code sequence whose score is greater than or equal to a preset value.

[0133] In this embodiment, CTC decoding is based on a sliding window of a preset length, for example, a 2-second sliding window. Decoding begins at the feature code sequence corresponding to the starting position of the sliding window. Audio segments with decoding scores greater than or equal to a preset value are obtained. The feature code sequence corresponding to this audio segment, i.e., the target feature code sequence, is used as input to the decoding module of the voice wake-up model. The preset value can be appropriately set based on actual application and is not specifically limited in this embodiment.

[0134] It should be noted that the training process of the voice wake-up model does not include this step, and CTC decoding can be used as a separate processing module. Figure 7 This is a structural diagram of a voice wake-up model provided by an embodiment of the present disclosure. Figure 7 As shown, the processing module for CTC decoding is placed between the encoding module and the decoding module of the voice wake-up model. After filtering by the processing module, the target feature encoding sequence with a score greater than or equal to the preset value is output to the attention mechanism module of the decoding module of the voice wake-up model.

[0135] Step 605: Obtain the semantic tag sequence corresponding to the user-defined wake-up word.

[0136] The semantic tag sequence corresponding to the user-defined wake-up word is pre-stored in the voice wake-up device, and its acquisition method can refer to step 403 of the above embodiment.

[0137] Step 606: The decoding module based on the voice wake-up model decodes and analyzes the target feature coding sequence and the semantic label sequence to determine whether to wake up the terminal device.

[0138] In this embodiment, the target feature coding sequence and the semantic label sequence corresponding to the user-defined wake-up word are used as the input of the decoding module of the voice wake-up model. Figure 2 , the semantic label sequence corresponding to the user-defined wake-up word is input into the RNN module of the decoding module (the RNN module on the right side of the attention mechanism module), and the target feature encoding sequence is input into the attention mechanism module of the decoding module. After processing by the attention mechanism module, RNN module, fully connected module and normalization module, the result of whether to wake up is output.

[0139] The voice wake-up method shown in this embodiment is based on a trained voice wake-up model. The voice wake-up model adopts an encoder-decoder architecture, in which the model decoding part includes an attention mechanism module, which greatly improves the performance of custom voice wake-up.

[0140] The voice wake-up solution shown in the embodiment of the present disclosure is based on a voice wake-up model that includes an attention mechanism module to determine whether to wake up the device. Through experimental testing, Tables 1 and 2 are obtained, wherein Table 1 shows the test statistics of the custom wake-up of the convolutional neural network CNN-deep neural network (Deep Neural Networks, DNN) and the custom wake-up including the attention mechanism, and Table 2 shows the test statistics of the custom wake-up and the custom wake-up including the attention mechanism. It should be pointed out that custom wake-up means that the manufacturer presets the device wake-up word before the device leaves the factory, and there is no function to change the wake-up word. Usually, the performance of custom wake-up is better than that of custom wake-up.

[0141] Table 1

[0142]

[0143] Table 2

[0144]

[0145] As shown in Table 1, compared with the CNN-DNN solution, the custom wake-up solution in this case improves the internal noise accuracy by 13.6% and reduces false alarms by more than 70%.

[0146] As shown in Table 2, compared to the customized wakeup solution, the customized wakeup solution in this case reduced the internal noise accuracy by 0.1% and the external noise accuracy by 2.9%, while maintaining the same false alarm rate. The accuracy of the customized wakeup solution in this case is close to the customized wakeup level.

[0147] It's important to note that internal noise refers to the noise generated by the device itself, while external noise refers to the sum of noise generated by the device's environment. External noise includes both background noise and point noise. Background noise, such as air conditioning noise and traffic noise, is stationary noise; point noise is noise with a clear direction and is non-stationary noise.

[0148] Figure 8 A schematic diagram of the structure of the training device for the voice wake-up model provided in the embodiment of the present disclosure. The training device for the voice wake-up model provided in this embodiment can be an electronic device or a device in an electronic device. Figure 8 As shown, the voice wake-up model training device 800 provided in the embodiment of the present disclosure may include:

[0149] A first acquisition module 801 is used to acquire speech recognition training data;

[0150] A first training module 802 is configured to perform speech recognition training on a basic model based on the speech recognition training data to obtain model parameters of the basic model when the model loss function converges; the basic model includes an encoding module and a decoding module;

[0151] A model configuration module 803 is configured to update configuration parameters of a decoding module in the base model based on model parameters of the base model in response to a model configuration instruction initiated by a user, to obtain a first model;

[0152] The second acquisition module 804 is used to acquire voice wake-up training data;

[0153] A second training module 805 is configured to perform voice wake-up training on the first model according to the voice wake-up training data to obtain a first model when the model loss function converges;

[0154] The model generation module 806 is configured to use the first model when the model loss function converges as the voice wake-up model.

[0155] In an optional embodiment of this embodiment, the model configuration module 803 includes: a model parameter update submodule, which is used to respond to a model configuration instruction initiated by the user, and based on the model parameters of the basic model, update the configuration parameters of the fully connected submodule and the normalization submodule of the decoding module in the basic model to obtain the first model.

[0156] In an optional embodiment of this embodiment, the model configuration module 803 includes: a model parameter update submodule, which is used to update the output dimensions of the fully connected submodule and the normalization submodule of the decoding module in the basic model to two dimensions based on the model parameters of the basic model in response to a model configuration instruction initiated by the user.

[0157] In an optional embodiment of this embodiment, the model loss function includes a CTC loss function and a CE loss function; the CTC loss function is used to train the encoding module of the basic model or the first model; the CE loss function is used to train the decoding module of the basic model or the first model.

[0158] In an optional embodiment of this embodiment, the first training module 802 includes:

[0159] The first joint training submodule is used to jointly train the encoding module and decoding module of the basic model according to the speech recognition training data; when the CTC loss function corresponding to the encoding module and the CE loss function corresponding to the decoding module converge, the model parameters of the basic model are obtained.

[0160] In an optional embodiment of this embodiment, the second training module 805 includes:

[0161] The second joint training submodule is used to jointly train the encoding module and decoding module of the first model according to the voice wake-up training data; when the CTC loss function corresponding to the encoding module and the CE loss function corresponding to the decoding module converge, the first model is obtained.

[0162] In an optional embodiment of this embodiment, the first obtaining module 801 includes:

[0163] A first receiving submodule is configured to receive first audio data input by a user, where the first audio data is audio data containing a custom wake-up word;

[0164] A first feature extraction submodule is used to extract features from the first audio data to obtain FBank features corresponding to the first audio data;

[0165] A first acquisition submodule, configured to acquire a semantic tag sequence and a syllable sequence corresponding to the first audio data;

[0166] The first creation submodule is used to use the FBank features, semantic label sequence and syllable sequence corresponding to the first audio data as a set of training data for the speech recognition training.

[0167] In an optional embodiment of this embodiment, the second obtaining module 804 includes:

[0168] The second creation submodule is used to use the FBank features, semantic label sequence and wake-up label corresponding to the first audio data input by the user as a set of positive data for the voice wake-up training; the first audio data is audio data containing a custom wake-up word;

[0169] A second receiving submodule is configured to receive second audio data input by a user, perform feature extraction on the second audio data, and obtain an FBank feature corresponding to the second audio data; the second audio data is audio data that does not contain the custom wake-up word;

[0170] The third sub-module is used to use the FBank features corresponding to the second audio data, the randomly generated semantic label sequence and the non-wake-up label as a set of negative example data for the voice wake-up training; the randomly generated semantic label sequence is different from the semantic label sequence corresponding to the second audio data.

[0171] The training device for the voice wake-up model provided in this embodiment can be used to execute the model training method in the aforementioned method embodiment. Its implementation principle and technical effects are similar and will not be elaborated here.

[0172] Figure 9 This is a structural diagram of the voice wake-up device provided in the embodiment of the present disclosure. The voice wake-up device provided in this embodiment can be an electronic device or a device in an electronic device. Figure 9 As shown, the voice wake-up device 900 provided in the embodiment of the present disclosure may include:

[0173] Receiving module 901, used to receive audio data input by the user;

[0174] A feature extraction module 902 is used to extract features from the audio data to obtain FBank features corresponding to the audio data;

[0175] The first processing module 903 is configured to encode the FBank features corresponding to the audio data based on the encoding module of the voice wake-up model to obtain a feature encoding sequence corresponding to the audio data;

[0176] The second processing module 904 is configured to use CTC decoding to determine a target feature code sequence in the feature code sequence whose score is greater than or equal to a preset value;

[0177] An acquisition module 905 is used to acquire a semantic tag sequence corresponding to a user-defined wake-up word;

[0178] The third processing module 906 is configured to perform decoding analysis on the target feature coding sequence and the semantic label sequence based on the decoding module of the voice wake-up model to determine whether to wake up the terminal device.

[0179] The voice wake-up device provided in this embodiment can be used to execute the voice wake-up method in the aforementioned method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0180] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0181] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0182] Figure 10 Schematic diagram of an electronic device provided for an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided for example only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0183] like Figure 10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0184] Various components in device 1000 are connected to I / O interface 1005, including an input unit 1006, such as a keyboard, mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, optical disk, etc.; and a communication unit 1009, such as a network card, modem, wireless communication transceiver, etc. The communication unit 1009 allows device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0185] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as the training method of the voice wake-up model or the voice wake-up method. For example, in some embodiments, the training method of the voice wake-up model or the voice wake-up method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the training method of the voice wake-up model or the voice wake-up method described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the training method of the voice wake-up model or the voice wake-up method in any other appropriate manner (for example, by means of firmware).

[0186] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0187] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0188] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0189] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0190] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0191] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0192] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0193] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for training a voice wake-up model, comprising: Acquiring speech recognition training data, performing speech recognition training on a basic model according to the speech recognition training data, and obtaining model parameters of the basic model when the model loss function converges; the basic model includes an encoding module and a decoding module; In response to a model configuration instruction initiated by a user, updating configuration parameters of a decoding module in the base model based on model parameters of the base model to obtain a first model; Acquire voice wake-up training data, and perform voice wake-up training on the first model according to the voice wake-up training data to obtain the first model when the model loss function converges; The first model when the model loss function converges is used as the voice wake-up model; The step of updating, in response to a model configuration instruction initiated by a user, configuration parameters of a decoding module in the base model based on model parameters of the base model to obtain a first model includes: In response to a model configuration instruction initiated by a user, based on the model parameters of the base model, updating the output dimensions of the fully connected submodule and the normalization submodule of the decoding module in the base model to two dimensions, so that the output result of the first model includes both awakening and non-awakening results; The acquiring of voice wake-up training data includes: The FBank features, semantic label sequences, and wake-up labels corresponding to the first audio data input by the user are used as a set of positive data for the voice wake-up training; the first audio data is audio data containing a custom wake-up word; Receive second audio data input by the user, perform feature extraction on the second audio data, and obtain FBank features corresponding to the second audio data; the second audio data is audio data that does not contain the custom wake-up word; The FBank features corresponding to the second audio data, the randomly generated semantic label sequence and the non-wake-up label are used as a set of negative example data for the voice wake-up training; the randomly generated semantic label sequence is different from the semantic label sequence corresponding to the second audio data.

2. The method according to claim 1, wherein The step of updating, in response to a model configuration instruction initiated by a user, configuration parameters of a decoding module in the base model based on model parameters of the base model to obtain a first model includes: In response to a model configuration instruction initiated by a user, configuration parameters of the fully connected submodule and the normalization submodule of the decoding module in the base model are updated based on the model parameters of the base model to obtain the first model.

3. The method according to claim 1 or 2, wherein The model loss function includes a connection time series classification ctc loss function and a cross entropy error ce loss function; The CTC loss function is used to train the encoding module of the basic model or the first model; The ce loss function is used to train the decoding module of the basic model or the first model.

4. The method according to claim 1 or 2, wherein: The performing speech recognition training on the basic model according to the speech recognition training data to obtain model parameters of the basic model when the model loss function converges includes: The encoding module and decoding module of the basic model are jointly trained according to the speech recognition training data; when the CTC loss function corresponding to the encoding module and the CE loss function corresponding to the decoding module converge, the model parameters of the basic model are obtained.

5. The method according to claim 1 or 2, wherein: The performing voice wake-up training on the first model according to the voice wake-up training data to obtain the first model when the model loss function converges includes: According to the voice wake-up training data, the encoding module and the decoding module of the first model are jointly trained; when the CTC loss function corresponding to the encoding module and the CE loss function corresponding to the decoding module converge, the first model is obtained.

6. The method according to claim 1 or 2, wherein: The acquiring of speech recognition training data includes: receiving first audio data input by a user; Performing feature extraction on the first audio data to obtain filter bank FBank features corresponding to the first audio data; Obtaining a semantic tag sequence and a syllable sequence corresponding to the first audio data; The FBank features, semantic label sequence and syllable sequence corresponding to the first audio data are used as a set of training data for the speech recognition training.

7. A voice wake-up method, applied to a terminal device, comprising: Receive audio data input by the user; Perform feature extraction on the audio data to obtain filter bank FBank features corresponding to the audio data; The encoding module based on the voice wake-up model encodes the FBank features corresponding to the audio data to obtain a feature coding sequence corresponding to the audio data; the connection time series classification (CTC) decoding is used to determine the target feature coding sequence in the feature coding sequence with a score greater than or equal to a preset value; Get the semantic label sequence corresponding to the user-defined wake-up word; Based on the decoding module of the voice wake-up model, the target feature coding sequence and the semantic label sequence are decoded and analyzed to determine whether to wake up the terminal device; The voice wake-up model is trained using the method according to any one of claims 1 to 6.

8. A training device for a voice wake-up model, comprising: A first acquisition module is used to acquire speech recognition training data; A first training module is used to perform speech recognition training on a basic model according to the speech recognition training data to obtain model parameters of the basic model when the model loss function converges; the basic model includes an encoding module and a decoding module; a model configuration module, configured to update configuration parameters of a decoding module in the base model based on model parameters of the base model in response to a model configuration instruction initiated by a user, to obtain a first model; The second acquisition module is used to obtain voice wake-up training data; A second training module is configured to perform voice wake-up training on the first model according to the voice wake-up training data to obtain a first model when a model loss function converges; A model generation module, configured to use the first model when the model loss function converges as the voice wake-up model; The model configuration module includes: a model parameter updating submodule, configured to update the output dimensions of the fully connected submodule and the normalization submodule of the decoding module in the base model to two dimensions based on the model parameters of the base model in response to a model configuration instruction initiated by a user, so that the output result of the first model includes two results: awakening and not awakening; The second acquisition module includes: The second creation submodule is used to use the FBank features, semantic label sequence and wake-up label corresponding to the first audio data input by the user as a set of positive data for the voice wake-up training; the first audio data is audio data containing a custom wake-up word; A second receiving submodule is configured to receive second audio data input by a user, perform feature extraction on the second audio data, and obtain an FBank feature corresponding to the second audio data; the second audio data is audio data that does not contain the custom wake-up word; The third sub-module is used to use the FBank features corresponding to the second audio data, the randomly generated semantic label sequence and the non-wake-up label as a set of negative example data for the voice wake-up training; the randomly generated semantic label sequence is different from the semantic label sequence corresponding to the second audio data.

9. The device according to claim 8, wherein The model configuration module includes: a model parameter updating submodule, which is used to respond to a model configuration instruction initiated by a user and, based on the model parameters of the basic model, update the configuration parameters of the fully connected submodule and the normalization submodule of the decoding module in the basic model to obtain the first model.

10. The device according to claim 8 or 9, wherein The model loss function includes a connection time series classification ctc loss function and a cross entropy error ce loss function; The CTC loss function is used to train the encoding module of the basic model or the first model; The ce loss function is used to train the decoding module of the basic model or the first model.

11. The device according to claim 8 or 9, wherein The first training module includes: The first joint training submodule is used to jointly train the encoding module and decoding module of the basic model according to the speech recognition training data; when the CTC loss function corresponding to the encoding module and the CE loss function corresponding to the decoding module converge, the model parameters of the basic model are obtained.

12. The device according to claim 8 or 9, wherein The second training module includes: The second joint training submodule is used to jointly train the encoding module and decoding module of the first model according to the voice wake-up training data; when the CTC loss function corresponding to the encoding module and the CE loss function corresponding to the decoding module converge, the first model is obtained.

13. The device according to claim 8 or 9, wherein The first acquisition module includes: A first receiving submodule, configured to receive first audio data input by a user; A first feature extraction submodule is used to extract features from the first audio data to obtain filter bank FBank features corresponding to the first audio data; A first acquisition submodule, configured to acquire a semantic tag sequence and a syllable sequence corresponding to the first audio data; The first creation submodule is used to use the FBank features, semantic label sequence and syllable sequence corresponding to the first audio data as a set of training data for the speech recognition training.

14. A voice wake-up device, comprising: A receiving module, configured to receive audio data input by a user; A feature extraction module is used to extract features from the audio data to obtain filter bank FBank features corresponding to the audio data; A first processing module is configured to encode the FBank features corresponding to the audio data based on the encoding module of the voice wake-up model to obtain a feature encoding sequence corresponding to the audio data; The second processing module is used to determine a target feature coding sequence with a score greater than or equal to a preset value in the feature coding sequence by using connection temporal classification (CTC) decoding; The acquisition module is used to obtain the semantic label sequence corresponding to the user-defined wake-up word; A third processing module is configured to decode and analyze the target feature coding sequence and the semantic label sequence based on the decoding module of the voice wake-up model to determine whether to wake up the terminal device; The voice wake-up model is trained using the device according to any one of claims 8 to 13.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6, or the method according to claim 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 6, or to execute the method according to claim 7.

17. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 or the method according to claim 7 is implemented.

Images