A voice wake-up method and device

Abstract

The present application provides a voice wake-up method and device, including: converting an audio signal into an acoustic feature sequence, wherein the audio signal is a voice audio signal; inputting the acoustic feature sequence into a filter network, outputting filtered audio features, wherein the filter network includes a filter generation network and a dynamic filter network; inputting the filtered audio features into a voice wake-up network, outputting a voice wake-up probability; and performing a wake-up operation according to the voice wake-up probability. By adding a dynamic filter network, the problem that the voice wake-up method is difficult to recognize the wake-up audio of unknown speakers and unknown noises due to the fixed parameters of the convolutional network is avoided, thereby improving the robustness of the voice wake-up method. At the same time, this method does not perform artificial sub-problem division, but is completely handed over to the deep learning model, so that the entire system can achieve optimal performance and avoid error accumulation caused by different sub-problems.

Description

Technical Field

[0001] The present application relates to the technical field of speech recognition, and in particular to a speech wake-up method and device. Background Art

[0002] Speech recognition technology allows machines to convert speech signals into corresponding text or commands through the process of recognition and understanding. Therefore, speech recognition technology can also be used when waking up terminal devices, so that in complex scenarios, speech recognition can be used to control the wake-up task of terminal devices. Because most speech wake-up technologies need to be used in complex sound environments for a long time, there are certain requirements for the robustness and power consumption of sound recognition.

[0003] In existing voice wake-up technology, convolutional neural networks are usually used for model compression to improve power consumption performance. Although this method can reduce power consumption, it has fixed convolutional network parameters, which is not conducive to detecting unknown speakers and unknown noise wake-up audio, so it cannot meet the robustness requirements of the product.

[0004] Therefore, how to provide a voice wake-up method with high robustness and without increasing power consumption is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a voice wake-up method and device, aiming to improve the robustness of voice wake-up without increasing power consumption.

[0006] In a first aspect, an embodiment of the present application provides a voice wake-up method, comprising:

[0007] Converting an audio signal into an acoustic feature sequence, wherein the audio signal is a speech audio signal;

[0008] Inputting the acoustic feature sequence into a filter network and outputting filtered audio features, the filter network comprising a filter generation network and a dynamic filter network;

[0009] Inputting the filtered audio features into a speech wake-up network and outputting a speech wake-up probability;

[0010] A wake-up operation is performed according to the voice wake-up probability.

[0011] Optionally, the filter generation network includes a first filter generation network and a second filter generation network, and the step of inputting the acoustic feature sequence into the filter network and outputting the filtered audio feature includes:

[0012] Inputting the acoustic feature sequence into the first filter generation network, and outputting first convolution kernel parameters;

[0013] Inputting the acoustic feature sequence into the second filter generation network, and outputting second convolution kernel parameters;

[0014] The acoustic feature sequence, the first convolution kernel parameter and the second convolution kernel parameter are input into the dynamic filtering network, and the filtered audio feature is output.

[0015] Optionally, the first filtering generation network includes:

[0016] w S =Sigmoid(Norm(Conv(x,w)));

[0017] Among them, w S is the first convolution kernel parameter, Sigmoid represents the Sigmoid activation function, Norm represents mean variance normalization, Conv(x,w) represents a convolution network with x as input, w as the convolution kernel, and x as the acoustic feature sequence.

[0018] Optionally, the second filtering generation network includes:

[0019] w N =Transpose(Linear(max(0,b)));

[0020] Among them, w N is the second convolution kernel parameter, Transpose means transposition, Linear means a fully connected network, max means taking the maximum value, b=Norm(Linear(a)), x i is the ith eigenvalue in the acoustic feature sequence, and n is the length of the acoustic feature sequence.

[0021] Optionally, the dynamic filtering network includes:

[0022] x o =Norm(Conv(x,w A ))+x;

[0023] Among them, x o For the filtered audio features, Conv(x,w A ) means taking x as input, w A is the convolutional network with convolution kernel, w A is the dynamic convolution parameter, w A =w S ⊙w N , ⊙ represents the element-wise product operator.

[0024] Optionally, the voice wake-up network includes:

[0025] P(wakeup|x)=WakeUpNet(x o );

[0026] Wherein, P(wake-up|x) is the voice wake-up probability, x o For the filtering audio features, WakeUpNet stands for End-to-End Voice Wake-Up Network.

[0027] Optionally, performing a wake-up operation according to the voice wake-up probability includes:

[0028] In response to the voice wake-up probability being greater than a preset threshold, a wake-up operation is performed.

[0029] In a second aspect, an embodiment of the present application provides a voice wake-up device, including:

[0030] A conversion module, used to convert an audio signal into an acoustic feature sequence, wherein the audio signal is a user's speech audio signal;

[0031] A filtering module, used for inputting the acoustic feature sequence into a filtering network and outputting filtered audio features, wherein the filtering network includes a filtering generation network and a dynamic filtering network;

[0032] A probability module, used for inputting the filtered audio features into a speech wake-up network and outputting a speech wake-up probability;

[0033] The wake-up module is used to perform a wake-up operation according to the voice wake-up probability.

[0034] In a third aspect, an embodiment of the present application provides a device, comprising a memory and a processor, wherein the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the voice wake-up method described in any one of the first aspects above.

[0035] In a fourth aspect, an embodiment of the present application provides a computer storage medium, in which a code is stored. When the code is executed, the device executing the code implements the voice wake-up method described in any one of the first aspects above.

[0036] The embodiment of the present application provides a voice wake-up method and device. When executing the method, the audio signal is first converted into an acoustic feature sequence, and the audio signal is a voice audio signal; then the acoustic feature sequence is input into a filter network to output a filtered audio feature, and the filter network includes a filter generation network and a dynamic filter network; then the filtered audio feature is input into a voice wake-up network to output a voice wake-up probability; finally, the wake-up operation is performed according to the voice wake-up probability. By adding a dynamic filter network, the problem that the voice wake-up method is difficult to recognize the wake-up audio of unknown speakers and unknown noises due to the fixed parameters of the convolutional network is avoided, and the robustness of the voice wake-up method is improved. At the same time, this method does not perform artificial sub-problem division, but is completely handed over to the deep learning model, so that the entire system can achieve optimal performance and avoid error accumulation caused by different sub-problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A method flow chart of a voice wake-up method provided in an embodiment of the present application;

[0039] Figure 2 Another method flow chart of the voice wake-up method provided in the embodiment of the present application;

[0040] Figure 3 A schematic diagram of the structure of a voice wake-up device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] Speech recognition technology, also known as Automatic Speech Recognition (ASR), aims to convert the vocabulary content in human speech into computer-readable input, such as keystrokes, binary codes, or character sequences. It is different from speaker recognition and speaker verification, which attempt to identify or verify the speaker of the speech rather than the vocabulary content contained in it.

[0042] Since speech recognition technology allows machines to convert speech signals into corresponding text or commands through the process of recognition and understanding, speech recognition technology can also be used when waking up terminal devices, so that in complex scenarios, speech recognition can be used to control the wake-up task of terminal devices. Because most speech wake-up technologies need to be used in complex sound environments for a long time, there are certain requirements for the robustness and power consumption of sound recognition.

[0043] In existing voice wake-up technology, convolutional neural networks are usually used for model compression to improve power consumption performance. Although this method can reduce power consumption, it has fixed convolutional network parameters, which is not conducive to detecting unknown speakers and unknown noise wake-up audio, so it cannot meet the robustness requirements of the product.

[0044] The method provided in the embodiment of the present application is executed by a computer device and is used to improve the robustness of voice wake-up without increasing power consumption.

[0045] Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0046] See also Figure 1 , Figure 1 A method flow chart of a voice wake-up method provided in an embodiment of the present application includes:

[0047] Step S101: converting the audio signal into an acoustic feature sequence.

[0048] The audio signal may include a voice audio signal around the terminal device, and the acoustic feature sequence may be a multidimensional array. As a possible implementation, the acoustic feature sequence may be an N×n two-dimensional real number tensor, where n is the length of the acoustic feature sequence and N is the dimension of the acoustic feature.

[0049] Step S102: input the acoustic feature sequence into a filter network, and output filtered audio features.

[0050] The filtering network includes a filtering generation network and a dynamic filtering network. The dynamic filtering network may include dynamic convolution parameters, which avoids the problem that the voice wake-up method is difficult to recognize unknown speakers and unknown noise wake-up audio due to fixed convolution network parameters, thereby improving the robustness of the voice wake-up method.

[0051] Step S103: input the filtered audio features into a speech wake-up network, and output a speech wake-up probability.

[0052] The voice wake-up network can conduct in-depth analysis of audio features and output the probability of successfully waking up the terminal device. Deciding whether to wake up the device based on the probability can make the decision more objective and accurate.

[0053] Step S104: performing a wake-up operation according to the voice wake-up probability.

[0054] As a possible implementation, a threshold may be preset so that the wake-up operation is performed when the voice wake-up probability is greater than the preset threshold.

[0055] In summary, this embodiment avoids the problem that the voice wake-up method is difficult to recognize unknown speakers and unknown noise wake-up audio due to the fixed parameters of the convolutional network by adding a dynamic filtering network, thereby improving the robustness of the voice wake-up method. At the same time, this method does not perform artificial sub-problem division, but completely hands it over to the deep learning model, so that the entire system can achieve optimal performance and avoid error accumulation caused by different sub-problems.

[0056] In the embodiment of the present application, the above Figure 1 There are many possible implementations of the steps, which are introduced below. It should be noted that the implementations given in the following introduction are only for illustrative purposes and do not represent all implementations of the embodiments of the present application.

[0057] See also Figure 2 , which is another method flow chart of the voice wake-up method provided in an embodiment of the present application, including:

[0058] Step S201: converting the audio signal into an acoustic feature sequence.

[0059] As a possible implementation, the audio signal can be converted into an acoustic feature sequence x, where x = [x1, x2...x i ...x n ], the acoustic feature sequence is a two-dimensional real number tensor of N×n, n is the length of the audio frame sequence, and N is the dimension of the acoustic feature.

[0060] Step S202: input the acoustic feature sequence into the first filter generation network, and output the first convolution kernel parameters.

[0061] As a possible implementation, the acoustic feature sequence may be input into the following network:

[0062] w S =Sigmoid(Norm(Conv(x,w)));

[0063] Among them, w Sis the first convolution kernel parameter, Sigmoid represents the Sigmoid activation function, Norm represents mean variance normalization, Conv(x,w) represents a convolution network with x as input, w as the convolution kernel, and x as the acoustic feature sequence.

[0064] Step S203: input the acoustic feature sequence into the second filter generation network, and output second convolution kernel parameters.

[0065] As a possible implementation, the acoustic feature sequence may be input into the following network:

[0066] w N =Transpose(Linear(max(0,b)));

[0067] Among them, w N is the second convolution kernel parameter, Transpose means transposition, Linear means a fully connected network, max means taking the maximum value, b=Norm(Linear(a)), x i is the ith eigenvalue in the acoustic feature sequence, and n is the length of the acoustic feature sequence.

[0068] Step S204: input the acoustic feature sequence, the first convolution kernel parameter and the second convolution kernel parameter into the dynamic filtering network, and output the filtered audio feature.

[0069] As a possible implementation, the dynamic convolution parameter w can be calculated by the following formula: A :

[0070] w A =w S ⊙w N ;

[0071] Among them, w S is the result of step S202, w N is the result of step S203, and ⊙ represents the element-wise product operator.

[0072] The dynamic convolution parameters and acoustic feature sequence are then input into the following dynamic filter network:

[0073] x o =Norm(Conv(x,w A ))+x;

[0074] Among them, x o For the filtered audio features, Conv(x,w A ) means taking x as input, w A A convolutional network with a convolution kernel.

[0075] Step S205: input the filtered audio features into a speech wake-up network, and output a speech wake-up probability.

[0076] As a possible implementation, the filtered audio features can be input into the following speech wake-up network to output the speech wake-up probability:

[0077] P(wakeup|x)=WakeUpNet(x o );

[0078] Wherein, P(wake-up|x) is the voice wake-up probability, x o For the filtering audio features, WakeUpNet stands for End-to-End Voice Wake-Up Network.

[0079] Step S206: performing a wake-up operation according to the voice wake-up probability.

[0080] The above steps are the same as those in the first embodiment and will not be described in detail here.

[0081] In summary, this embodiment completely hands over the problem to the deep learning model through a specific convolutional network, so that the entire system can achieve optimal performance and avoid error accumulation caused by different sub-problems. At the same time, by adding a dynamic filtering network, the problem that the voice wake-up method is difficult to recognize unknown speakers and unknown noise wake-up audio due to fixed convolutional network parameters is avoided, thereby improving the robustness of the voice wake-up method.

[0082] The above are some specific implementations of the default prediction model generation method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding device. The device provided in the embodiment of the present application will be introduced from the perspective of functional modularization.

[0083] See also Figure 3 The structure diagram of the voice awakening device 300 shown in the figure includes a conversion module 301, a filtering module 302, a probability module 303 and a awakening module 304.

[0084] A conversion module 301 is used to convert an audio signal into an acoustic feature sequence, wherein the audio signal is a user's speech audio signal;

[0085] A filtering module 302, configured to input the acoustic feature sequence into a filtering network and output filtered audio features, wherein the filtering network includes a filtering generation network and a dynamic filtering network;

[0086] A probability module 303, used for inputting the filtered audio features into a speech wake-up network and outputting a speech wake-up probability;

[0087] The wake-up module 304 is configured to perform a wake-up operation according to the voice wake-up probability.

[0088] As a possible implementation, the filtering module 302 includes:

[0089] A first filtering unit, configured to input the acoustic feature sequence into the first filtering generation network and output first convolution kernel parameters;

[0090] A second filtering unit, used for inputting the acoustic feature sequence into the second filter generation network, and outputting second convolution kernel parameters;

[0091] The dynamic filtering unit is used to input the acoustic feature sequence, the first convolution kernel parameter and the second convolution kernel parameter into the dynamic filtering network, and output the filtered audio feature.

[0092] As a possible implementation, the first filtering generation network includes:

[0093] w S =Sigmoid(Norm(Conv(x,w)));

[0094] Among them, w S is the first convolution kernel parameter, Sigmoid represents the Sigmoid activation function, Norm represents mean variance normalization, Conv(x,w) represents a convolution network with x as input, w as the convolution kernel, and x as the acoustic feature sequence.

[0095] As a possible implementation, the second filtering generation network includes:

[0096] w N =Transpose(Linear(max(0,b)));

[0097] Among them, w N is the second convolution kernel parameter, Transpose means transposition, Linear means a fully connected network, max means taking the maximum value, b=Norm(Linear(a)), x i is the ith eigenvalue in the acoustic feature sequence, and n is the length of the acoustic feature sequence.

[0098] As a possible implementation, the dynamic filtering network includes:

[0099] x o =Norm(Conv(x,w A ))+x;

[0100] Among them, x o For the filtered audio features, Conv(x,w A) means taking x as input, w A is the convolutional network with convolution kernel, w A is the dynamic convolution parameter, w A =w S ⊙w N , ⊙ represents the element-wise product operator.

[0101] As a possible implementation manner, the voice wake-up network includes:

[0102] P(wakeup|x)=WakeUpNet(x o );

[0103] Wherein, P(wake-up|x) is the voice wake-up probability, x o For the filtering audio features, WakeUpNet stands for End-to-End Voice Wake-Up Network.

[0104] As a possible implementation, the wake-up module 304 includes:

[0105] The wake-up unit is configured to perform a wake-up operation in response to the voice wake-up probability being greater than a preset threshold.

[0106] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the voice wake-up method of the embodiment of the present application is implemented.

[0107] In practical applications, the computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0108] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0109] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0110] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0111] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0112] The above description is merely an exemplary embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A voice wake-up method, characterized in that: The method comprises: Converting an audio signal into an acoustic feature sequence, wherein the audio signal is a speech audio signal; Input the acoustic feature sequence into a filter network, and output filtered audio features, wherein the filter network includes a filter generation network and a dynamic filter network; the filter generation network includes a first filter generation network and a second filter generation network; Inputting the filtered audio features into a speech wake-up network and outputting a speech wake-up probability; Performing a wake-up operation according to the voice wake-up probability; Wherein, the first filtering generation network comprises: w S =Sigmoid(Norm(Conv(x,w))); Among them, w S is the first convolution kernel parameter, Sigmoid represents the Sigmoid activation function, Norm represents mean variance normalization, Conv(x,w) represents a convolution network with x as input, w as the convolution kernel, and x as the acoustic feature sequence; The second filtering generation network comprises: w N =Transpose(Linear(max(0,b))); Among them, w N is the second convolution kernel parameter, Transpose means transposition, Linear means fully connected network, max means taking the maximum value, b = Norm (Linear (a)), x i is the i-th eigenvalue in the acoustic feature sequence, and n is the length of the acoustic feature sequence; The dynamic filtering network comprises: x o =Norm(Conv(x,w A ))+x; Among them, x o For the filtered audio features, Conv(x,w A ) means taking x as input, w A is the convolutional network with convolution kernel, w A is the dynamic convolution parameter, w A =w S ⊙w N , ⊙ represents the element-wise product operator.

2. The method according to claim 1, characterized in that: The step of inputting the acoustic feature sequence into a filter network and outputting filtered audio features comprises: Inputting the acoustic feature sequence into the first filter generation network, and outputting first convolution kernel parameters; Inputting the acoustic feature sequence into the second filter generation network, and outputting second convolution kernel parameters; The acoustic feature sequence, the first convolution kernel parameter and the second convolution kernel parameter are input into the dynamic filtering network, and the filtered audio feature is output.

3. The method according to claim 2, characterized in that The voice wake-up network includes: P(wakeup|x)=WakeUpNet(x o ); Wherein, P(wake-up|x) is the voice wake-up probability, x o For the filtering audio features, WakeUpNet stands for End-to-End Voice Wake-Up Network.

4. The method according to claim 1, characterized in that: The performing a wake-up operation according to the voice wake-up probability includes: In response to the voice wake-up probability being greater than a preset threshold, a wake-up operation is performed.

5. A voice wake-up device, characterized in that: The device comprises: A conversion module, used to convert an audio signal into an acoustic feature sequence, wherein the audio signal is a user's speech audio signal; A filtering module, used for inputting the acoustic feature sequence into a filtering network and outputting filtered audio features, wherein the filtering network includes a filtering generation network and a dynamic filtering network; the filtering generation network includes a first filtering generation network and a second filtering generation network; A probability module, used for inputting the filtered audio features into a speech wake-up network and outputting a speech wake-up probability; A wake-up module, used to perform a wake-up operation according to the voice wake-up probability; Wherein, the first filtering generation network comprises: w S =Sigmoid(Norm(Conv(x,w))); Among them, w S is the first convolution kernel parameter, Sigmoid represents the Sigmoid activation function, Norm represents mean variance normalization, Conv(x,w) represents a convolution network with x as input, w as the convolution kernel, and x as the acoustic feature sequence; The second filtering generation network comprises: w N =Transpose(Linear(max(0,b))); Among them, w N is the second convolution kernel parameter, Transpose means transposition, Linear means fully connected network, max means taking the maximum value, b = Norm (Linear (a)), x i is the i-th eigenvalue in the acoustic feature sequence, and n is the length of the acoustic feature sequence; The dynamic filtering network comprises: x o =Norm(Conv(x,w A ))+x; Among them, x o For the filtered audio features, Conv(x,w A ) means taking x as input, w A is the convolutional network with convolution kernel, w A is the dynamic convolution parameter, w A =w S ⊙w N , ⊙ represents the element-wise product operator.

6. A device, characterized in that: The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the voice wake-up method according to any one of claims 1 to 4.

7. A computer storage medium, wherein a code is stored in the computer storage medium. When the code is executed, a computer storage device executing the code implements the voice wake-up method according to any one of claims 1 to 4.

Images