Voice activation detection method, device, terminal equipment and computer storage medium

Abstract

The present invention discloses a voice activation detection method, device, terminal equipment and computer-readable storage medium. When a sound signal is collected through a microphone, the method extracts features from the sound signal to obtain a feature sequence of the sound signal; inputs the feature sequence into a preset recursive neural network model for cyclic network training to obtain a model output signal, wherein the recursive neural network model is constructed based on a cyclic neural network model with a gate structure; and confirms whether voice information is detected from the sound signal based on the model output signal and a preset fully connected network. By adopting the technical solution of the present invention, a simple model structure with fewer required parameters can be constructed to detect voice activation, thereby effectively reducing the complexity of the entire voice activation detection process.

Description

Technical Field

[0001] The present invention belongs to the field of earphone technology, and in particular relates to a voice activation detection method, device, terminal equipment and computer-readable storage medium. Background Art

[0002] VAD (Voice Activity Detection) technology can reliably detect speech in acoustic signals, and is therefore widely used in voice-enabled devices such as TWS (True Wireless Stereo) headphones and smart speakers. The core of VAD technology is to distinguish whether there is a speech signal from a noisy speech signal, which is essentially a classification problem. Therefore, neural networks that simulate human brain recognition models can effectively assist in solving this problem.

[0003] The U-net model (an improved fully convolutional network FCN structure) is an end-to-end symmetrical structure. This model has been proven to have good performance in speech enhancement processing tasks, but due to the vanishing gradient, the model training is not stable enough. Therefore, when the U-net model is used for speech activation detection, a large number of parameters have to be used for model training, which makes the implementation of the entire detection process very complicated. Summary of the invention

[0004] The main purpose of the present invention is to provide a voice activation detection method, apparatus, terminal device and computer-readable storage medium, aiming to construct a simple model structure with fewer parameters to detect voice activation, thereby effectively reducing the complexity of the entire voice activation detection process.

[0005] In order to achieve the above object, the present invention provides a voice activation detection method, the voice activation detection method comprising:

[0006] When a sound signal is collected by a microphone, feature extraction is performed on the sound signal to obtain a feature sequence of the sound signal;

[0007] Inputting the feature sequence into a preset recursive neural network model to perform recursive network training to obtain a model output signal, wherein the recursive neural network model is constructed based on a recursive neural network model with a gate structure;

[0008] Determine whether speech information is detected from the sound signal based on the model output signal and a preset fully connected network.

[0009] Optionally, the step of inputting the feature sequence into a preset recursive neural network model for recurrent network training to obtain a model output signal comprises:

[0010] Inputting the feature sequence into a preset recursive neural network model to combine the feature sequence with a previous model output signal of the recursive neural network model, and performing a first recurrent network training to obtain an intermediate calculation result;

[0011] The intermediate calculation result is used as the new previous model output signal, and the feature sequence and the new previous model output signal are combined to perform a second cyclic network training to obtain the current model output signal.

[0012] Optionally, the step of confirming whether voice information is detected from the sound signal according to the model output signal and a preset fully connected network includes:

[0013] Passing the model output signal to the preset fully connected network to confirm whether a VAD flag is identified from the model output signal;

[0014] If the VAD flag is confirmed to be recognized, it is confirmed that voice information is detected from the sound signal;

[0015] or,

[0016] If it is confirmed that the VAD flag is not recognized, it is confirmed that no voice information is detected from the sound signal.

[0017] Optionally, the step of transmitting the model output signal to the preset fully connected network includes:

[0018] Performing a decoding operation on the model output signal to obtain a decoded signal;

[0019] The decoded signal is transmitted to the preset fully connected network.

[0020] Optionally, the decoding operation includes: data feature bridging and upsampling;

[0021] The step of performing a decoding operation on the model output signal to obtain a decoded signal comprises:

[0022] The model output signal is input into a preset decoder to perform the data feature crossover and up-sampling on the model output signal to obtain a decoded signal.

[0023] Optionally, the sound signal includes: a current frame sound signal collected by the microphone at the current moment, and a plurality of frames of sound signals collected by the microphone within a preset time period ending at the current moment;

[0024] The step of extracting features from the sound signal to obtain a feature sequence of the sound signal comprises:

[0025] The current frame sound signal or the several frames of sound signals are input into a preset encoder to extract data features for the current frame sound signal or the several frames of sound signals, and then the extracted data features are downsampled to obtain a feature sequence of the current frame sound signal or the several frames of sound signals.

[0026] Optionally, the step of inputting the current frame sound signal or the plurality of frames sound signal into a preset encoder includes:

[0027] Performing time-frequency conversion processing on the current frame sound signal or the plurality of frames sound signals;

[0028] The current frame sound signal or several frames of sound signals after the time-frequency conversion process are input to a preset encoder.

[0029] In addition, to achieve the above object, the present invention further provides a voice activation detection device, the voice activation detection device comprising:

[0030] A feature extraction module is used to extract features from a sound signal to obtain a feature sequence of the sound signal when the sound signal is collected by a microphone;

[0031] A model calculation module, used for inputting the feature sequence into a preset recursive neural network model to perform recursive network training to obtain a model output signal, wherein the recursive neural network model is constructed based on a recursive neural network model with a gate structure;

[0032] The detection module is used to confirm whether voice information is detected from the sound signal based on the model output signal and a preset fully connected network.

[0033] Each functional module of the voice activation detection device of the present invention implements the steps of the motion monitoring motion control method of the wireless headset as described above when running.

[0034] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal device, which includes: a memory, a processor, and a voice activation detection program stored in the memory and executable on the processor, and the control program for motion monitoring of the wireless headset is executed by the processor to implement the steps of the motion monitoring control method of the wireless headset as described above.

[0035] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a voice activation detection program is stored, and when the voice activation detection program is executed by a processor, the steps of the voice activation detection method as described above are implemented.

[0036] The embodiments of the present invention propose a voice activation detection method, apparatus, terminal device and computer-readable storage medium. When a sound signal is collected by a microphone, feature extraction is performed on the sound signal to obtain a feature sequence of the sound signal; the feature sequence is input into a preset recursive neural network model for recurrent network training to obtain a model output signal, wherein the recursive neural network model is constructed based on a recurrent neural network model with a gate structure; and whether voice information is detected from the sound signal is confirmed based on the model output signal and a preset fully connected network.

[0037] In this way, the embodiment of the present invention preliminarily constructs a recursive neural network model through a recursive neural network model with a gate structure, so that when a sound signal is collected through a microphone, the feature sequence of the sound signal is extracted, and the feature sequence is input into the recursive neural network model for recursive network training, and then the model output signal and the fully connected network are combined to confirm whether a voice signal is detected from the currently collected sound signal. In this way, voice activation detection can be achieved with only a few model parameters, thereby effectively reducing the complexity of voice activation detection.

[0038] In addition, since the technical solution of the present invention can easily perform voice activation detection, when the TWS headset device performs voice activation detection based on the technical solution of the present invention, it can improve the user's experience of quickly making voice calls and other operations under ambient noise. Moreover, when the technical solution of the present invention is applied to a security system, it can efficiently detect whether voice is detected in the monitored area, thereby timely triggering the recording or transmission of the audio / video source when voice is detected, effectively improving the monitoring efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the device structure of the hardware operating environment of the terminal device involved in the embodiment of the present invention;

[0040] Figure 2 A schematic diagram of the steps of the first embodiment of the voice activation detection method of the present invention;

[0041] Figure 3 A schematic diagram of the model structure of a recursive neural network model involved in an embodiment of a voice activation detection method of the present invention;

[0042] Figure 4 A schematic diagram of a fully connected network involved in an embodiment of a voice activation detection method of the present invention;

[0043] Figure 5 A schematic diagram of a usage scenario of a terminal device involved in an embodiment of a voice activation detection method of the present invention;

[0044] Figure 6 FIG. 4 is a schematic diagram of functional modules of a voice activation detection device according to an embodiment of the present invention.

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0047] Reference Figure 1 , Figure 1 The present invention is a schematic diagram of the device structure of the hardware operating environment of the terminal device involved in the embodiment of the present invention.

[0048] The terminal device in the embodiment of the present invention is a device that supports voice and is equipped with a microphone, and the terminal device may include: TWS headphones, monitoring equipment in a security system, etc. In addition, in other feasible embodiments, the terminal device may also be a mobile or fixed terminal device such as a smart phone, a tablet, a personal computer, etc.

[0049] like Figure 1 As shown, the terminal device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or it may be a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0050] Those skilled in the art will understand that Figure 1 The terminal device structure shown in the figure does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0051] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a voice activation detection program.

[0052] exist Figure 1In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and perform data communication with the backend server; the user interface 1003 is mainly used to connect to the client and perform data communication with the client; and the processor 1001 can be used to call the voice activation detection program stored in the memory 1005 and perform the following operations:

[0053] When a sound signal is collected by a microphone, feature extraction is performed on the sound signal to obtain a feature sequence of the sound signal;

[0054] Inputting the feature sequence into a preset recursive neural network model to perform recursive network training to obtain a model output signal, wherein the recursive neural network model is constructed based on a recursive neural network model with a gate structure;

[0055] Determine whether speech information is detected from the sound signal based on the model output signal and a preset fully connected network.

[0056] Optionally, the processor 1001 may also be configured to call a voice activation detection program stored in the memory 1005 and perform the following operations:

[0057] Inputting the feature sequence into a preset recursive neural network model to combine the feature sequence with a previous model output signal of the recursive neural network model, and performing a first recurrent network training to obtain an intermediate calculation result;

[0058] The intermediate calculation result is used as the new previous model output signal, and the feature sequence and the new previous model output signal are combined to perform a second cyclic network training to obtain the current model output signal.

[0059] Optionally, the processor 1001 may also be configured to call a voice activation detection program stored in the memory 1005 and perform the following operations:

[0060] Passing the model output signal to the preset fully connected network to confirm whether a VAD flag is identified from the model output signal;

[0061] If the VAD flag is confirmed to be recognized, it is confirmed that voice information is detected from the sound signal;

[0062] or,

[0063] If it is confirmed that the VAD flag is not recognized, it is confirmed that no voice information is detected from the sound signal.

[0064] Optionally, the processor 1001 may also be configured to call a voice activation detection program stored in the memory 1005 and perform the following operations:

[0065] Performing a decoding operation on the model output signal to obtain a decoded signal;

[0066] The decoded signal is transmitted to the preset fully connected network.

[0067] Optionally, the decoding operation includes: data feature bridging and upsampling; the processor 1001 may also be used to call a voice activation detection program stored in the memory 1005 and perform the following operations:

[0068] The model output signal is input into a preset decoder to perform the data feature crossover and up-sampling on the model output signal to obtain a decoded signal.

[0069] Optionally, the sound signal includes: a current frame sound signal collected by the microphone at the current moment, and a plurality of frames of sound signals collected by the microphone within a preset time period ending at the current moment; the processor 1001 may also be used to call a voice activation detection program stored in the memory 1005, and perform the following operations:

[0070] The current frame sound signal or the several frames of sound signals are input into a preset encoder to extract data features for the current frame sound signal or the several frames of sound signals, and then the extracted data features are downsampled to obtain a feature sequence of the current frame sound signal or the several frames of sound signals.

[0071] Optionally, the processor 1001 may also be configured to call a voice activation detection program stored in the memory 1005 and perform the following operations:

[0072] Performing time-frequency conversion processing on the current frame sound signal or the plurality of frames sound signals;

[0073] The current frame sound signal or several frames of sound signals after the time-frequency conversion process are input to a preset encoder.

[0074] Based on the above terminal device, various embodiments of the voice activation detection method of the present invention are proposed.

[0075] In various embodiments of the voice activation detection method of the present invention, the voice activation detection method of the present invention is applied to the above-mentioned terminal device equipped with a microphone.

[0076] Please refer to Figure 2 , Figure 2 It is a flowchart of the first embodiment of the voice activation detection method of the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the voice activation detection method of the present invention can certainly perform the steps shown or described in a different order than here.

[0077] In a first embodiment of the voice activation detection method of the present invention, the voice activation detection method of the present invention includes:

[0078] Step S10: when a sound signal is collected by a microphone, feature extraction is performed on the sound signal to obtain a feature sequence of the sound signal;

[0079] In this embodiment, when the terminal device collects a sound signal from the environment through a microphone, it immediately extracts features from the sound signal to obtain a feature sequence of the sound signal.

[0080] It should be noted that, in the present embodiment, the terminal device can specifically continuously monitor and collect sound signals in the noisy environment through the configured microphone.

[0081] In addition, in this embodiment and other feasible implementation modes described below, the sound signal collected by the terminal device through the microphone can be: the current frame sound signal collected by the microphone only at the current moment, or the sound signal collected by the terminal device through the microphone can also be: several frames of sound signals continuously collected by the microphone during a preset time period ending at the current moment.

[0082] Further, in this embodiment, the step of “extracting features from the sound signal to obtain a feature sequence of the sound signal” in the above step S10 may include:

[0083] The current frame sound signal or the several frames of sound signals are input into a preset encoder to extract data features for the current frame sound signal or the several frames of sound signals, and then the extracted data features are downsampled to obtain a feature sequence of the current frame sound signal or the several frames of sound signals.

[0084] In this embodiment, the terminal device is pre-configured or externally connected with an encoder. After the terminal device acquires the current frame sound signal or several frames of sound signals including the current frame sound signal through a microphone, the current sound signal or the several frames of sound signals are immediately input into the encoder, so that the encoder first extracts data features from the current sound signal or the several frames of sound signals, and then performs down-sampling processing on the extracted data features, thereby extracting the feature sequence.

[0085] Exemplarily, as a feasible implementation, the encoder may specifically use one-dimensional convolution at the encoder layer convolution to extract data features, and then further use a 2*2 pooling layer to downsample the extracted data features, and the activation function of the encoder may specifically use LeakyRelu, and the definition of the activation function LeakyRelu is as follows:

[0086]

[0087] Furthermore, in this embodiment, the above step of “inputting the current frame sound signal or the plurality of frames sound signal into a preset encoder” may further include:

[0088] Performing time-frequency conversion processing on the current frame sound signal or the plurality of frames sound signals;

[0089] The current frame sound signal or several frames of sound signals after the time-frequency conversion process are input to a preset encoder.

[0090] In this embodiment, when the terminal device inputs the current frame sound signal or several frames of sound signals collected by the microphone into the above-mentioned encoder for feature extraction, since the current frame sound signal or several frames of sound signals are time domain signals, the terminal device first needs to perform time-frequency conversion processing on the current frame sound signal or several frames of sound signals, and then input the current frame sound signal or several frames of sound signals after the time-frequency conversion processing into the encoder.

[0091] Exemplarily, as a feasible implementation mode, the terminal device may specifically use Fourier transform to perform time-frequency transform processing on a current frame sound signal or several frames of sound signals collected by a microphone.

[0092] Step S20, inputting the feature sequence into a preset recursive neural network model to perform recursive network training to obtain a model output signal, wherein the recursive neural network model is constructed based on a recursive neural network model with a gate structure;

[0093] In this embodiment, after the terminal device extracts features from the sound signal collected by the microphone to obtain a feature sequence, the feature sequence is further input into a recursive neural network model that is pre-constructed based on a recursive neural network model with a gate structure, so that the recursive neural network model performs recursive network training on the feature sequence to obtain a model output signal.

[0094] It should be noted that, in this embodiment, the terminal device can specifically construct a recursive neural network model with a simple structure based on recurrent neural network models such as LSTM (Long Short-Term Memory, a time recurrent neural network, usually called a long short-term memory network) and GRU (gated recurrent neural network) with a gate structure.

[0095] Furthermore, in this embodiment, the above step S20 may include:

[0096] Inputting the feature sequence into a preset recursive neural network model to combine the feature sequence with a previous model output signal of the recursive neural network model, and performing a first recurrent network training to obtain an intermediate calculation result;

[0097] The intermediate calculation result is used as the new previous model output signal, and the feature sequence and the new previous model output signal are combined to perform a second cyclic network training to obtain the current model output signal.

[0098] In this embodiment, the terminal device inputs the feature sequence of the extracted sound signal into a pre-constructed and trained recursive neural network model, whereby the recursive neural network model performs the first recursive network training to obtain an intermediate calculation result in accordance with the "reset gate" expression and the "new memory gate" expression, combined with the feature sequence and the previous model output signal obtained by the previous recursive network training of the recursive neural network model.

[0099] Then, the terminal device further uses the recursive neural network model to take the intermediate calculation result as the new previous model output signal of the recursive neural network model, and then combines the same feature sequence and the new previous model output signal again to perform a second recurrent network training to obtain the model output signal of the recursive neural network model when the feature sequence is trained on the recursive network for the current time.

[0100] For example, Figure 3 The structure of the recursive neural network model shown in FIG. 1 is that after the terminal device inputs the feature sequence X(t) into the recursive neural network model, the recursive neural network model first performs the “reset gate” expression A1(t)=sigmoid(X(t)*W1+Y(t-1)*V1+B1) and the “new memory gate” expression Combining the feature sequence X(t) and the previous model output signal Y(t-1) obtained by the previous recurrent network training of the recurrent neural network model, the first recurrent network training is performed to obtain the intermediate calculation result Y(t)=(1-A1(t))*Y(t-1)+A1(t)*A2(t).

[0101] Afterwards, the recursive neural network model further uses the intermediate calculation result Y(t) as the new previous model output signal Y(t-1), and then, in accordance with the above-mentioned "reset gate" expression and "new memory gate" expression, again combines the feature sequence X(t) and the new previous model output signal Y(t-1) to perform a second recurrent network training, thereby obtaining the model output signal Y(t) of the recursive neural network model when the feature sequence X(t) is currently subjected to recurrent network training.

[0102] It should be noted that, in the above-mentioned "reset gate" expression and "new memory gate" expression, W1 and V1 are respectively the weight values ​​involved in the "reset gate" expression, B1 is the bias involved in the "reset gate" expression, and W2 and V2 are respectively the weight values ​​involved in the "new memory gate" expression, and B2 is the bias involved in the "new memory gate" expression. Among them, the various weight values ​​and biases are obtained by training based on the back propagation calculation method when the terminal device pre-builds the recursive neural network model.

[0103] Exemplarily, each weight value involved in the “new memory gate” expression is obtained through the following training process:

[0104]

[0105] in,

[0106] Afterwards, 2,k Do the gradient calculation:

[0107]

[0108] Similarly, for v 2,k Do the gradient calculation:

[0109]

[0110]

[0111] in,

[0112] The weight values ​​involved in the "reset gate" expression are obtained through the following training process:

[0113]

[0114] in,

[0115] Afterwards, 1,k Do the gradient calculation:

[0116]

[0117] Similarly, for v 1,k Do the gradient calculation:

[0118]

[0119]

[0120] in,

[0121] Step S30, confirming whether speech information is detected from the sound signal based on the model output signal and a preset fully connected network.

[0122] In this embodiment, after the terminal device performs recurrent network training on the feature sequence through the recursive neural network model to obtain the model output signal, the model output signal can be further passed through a preset fully connected network to confirm whether voice information is detected in the sound signal currently collected by the microphone.

[0123] It should be noted that, in this embodiment, the preset fully connected network can be specifically as follows: Figure 4 As shown, the fully connected network indicates that all nodes in the network are connected to data of all dimensions of the input. In addition, the activation function of the fully connected network can be selected from the Tanh function and the RELU function, where:

[0124]

[0125] RELU(x)=max(0,x).

[0126] Furthermore, in this embodiment, the above step S30 may include:

[0127] Passing the model output signal to the preset fully connected network to confirm whether a VAD flag is identified from the model output signal;

[0128] If the VAD flag is confirmed to be recognized, it is confirmed that voice information is detected from the sound signal;

[0129] or,

[0130] If it is confirmed that the VAD flag is not recognized, it is confirmed that no voice information is detected from the sound signal.

[0131] In this embodiment, after the terminal device inputs the model output signal into the fully connected network, if there is voice information in the sound signal corresponding to the model output signal, the fully connected network will output the VAD voice activation detection flag at the fully connected layer. In this way, the terminal device can confirm whether voice information is detected in the sound signal currently collected by the microphone by confirming whether the fully connected network outputs the VAD voice activation detection flag recognized from the model output signal.

[0132] That is, if it is confirmed that the fully connected network outputs the VAD voice activation detection flag identified from the model output signal, it is confirmed that voice information is detected in the sound signal. Otherwise, if it is confirmed that the fully connected network does not output the VAD voice activation detection flag, it is confirmed that no voice information is currently detected in the sound signal.

[0133] Furthermore, in this embodiment, the above-mentioned step of “transmitting the model output signal to the preset fully connected network” may include:

[0134] Performing a decoding operation on the model output signal to obtain a decoded signal;

[0135] The decoded signal is transmitted to the preset fully connected network.

[0136] In this embodiment, when the terminal device inputs the model output signal into the fully connected network, the terminal device needs to first perform a decoding operation on the model output signal to obtain a decoded signal, and then further transmit the decoded signal to the above-mentioned fully connected network, so that the fully connected network can identify whether there is a VAD voice activation detection flag in the decoded signal, and when it is identified, the flag is output at the fully connected layer.

[0137] Furthermore, in this embodiment, the above decoding operation includes: data feature bridging and upsampling. Based on this, the above step of "performing a decoding operation on the model output signal to obtain a decoded signal" may include:

[0138] The model output signal is input into a preset decoder to perform the data feature crossover and up-sampling on the model output signal to obtain a decoded signal.

[0139] In this embodiment, the terminal device can specifically decode the model output signal through a built-in or external decoder. That is, the terminal device inputs the model output signal into the decoder, and the decoder first cross-links the data features of the model output signal, and then up-samples the cross-linked data features, so that the decoded signal corresponding to the model output signal can be obtained.

[0140] For example, as a feasible implementation, a decoder built into or external to the terminal device may specifically implement upsampling of the crossover data features by means of deconvolution or linear difference.

[0141] Further, please refer to Figure 5 In the application scenario shown in the figure (in the figure: enconde represents an encoder, 1D-conv represents a one-dimensional convolution, donwsample represents downsampling, upsample represents upsampling, and R_rnn represents an improved rnn (recurrent neural network) - the above-mentioned recurrent neural network model), the voice activation detection method of the present invention can be implemented according to the following steps in a feasible embodiment:

[0142] 1. The input microphone signal is a time domain signal y;

[0143] 2. Perform time-frequency transformation Y1(K) on the input time domain signal y;

[0144] 3. Input Y1(K) to the encoder layer to realize data feature extraction and downsampling, and finally output Y2(K) to the R_rnn module;

[0145] 4. After two R_rnn operations, the Y2(K) signal outputs the signal Y3((K);

[0146] 5. Y3((K) implements data feature crossover and upsampling at the decoder layer to obtain signal Y4((K);

[0147] 6.Y4((K) outputs the voice activation detection (vad) flag after passing through the fully connected layer.

[0148] In this embodiment, the voice activation detection method of the present invention is to extract features from the sound signal immediately to obtain a feature sequence of the sound signal when the terminal device collects the sound signal from the environment through the microphone. After that, the terminal device further inputs the feature sequence into a recursive neural network model pre-constructed based on a recursive neural network model with a gate structure, so that the recursive neural network model performs recursive network training on the feature sequence, and then obtains a model output signal.

[0149] Finally, after the terminal device performs recurrent network training on the feature sequence through the recursive neural network model to obtain the model output signal, the model output signal can be further passed through a preset fully connected network to confirm whether voice information is detected in the sound signal currently collected by the microphone.

[0150] In this way, the embodiment of the present invention preliminarily constructs a recursive neural network model through a recursive neural network model with a gate structure, so that when a sound signal is collected through a microphone, the feature sequence of the sound signal is extracted, and the feature sequence is input into the recursive neural network model for recursive network training, and then the model output signal and the fully connected network are combined to confirm whether a voice signal is detected from the currently collected sound signal. In this way, voice activation detection can be achieved with only a few model parameters, thereby effectively reducing the complexity of voice activation detection.

[0151] In addition, since the technical solution of the present invention can easily perform voice activation detection, when the TWS headset device performs voice activation detection based on the technical solution of the present invention, it can improve the user's experience of quickly making voice calls and other operations under ambient noise. Moreover, when the technical solution of the present invention is applied to a security system, it can efficiently detect whether voice is detected in the monitored area, thereby timely triggering the recording or transmission of the audio / video source when voice is detected, effectively improving the monitoring efficiency.

[0152] In addition, an embodiment of the present invention further provides a voice activation detection device, which is applied to a terminal device equipped with a microphone.

[0153] Please refer to Figure 6 , Figure 6 FIG. 1 is a schematic diagram of functional modules of an embodiment of a voice activation detection device according to the present invention. Figure 6 As shown, the voice activation detection device of the present invention comprises:

[0154] The feature extraction module 10 is used to extract features from the sound signal to obtain a feature sequence of the sound signal when the sound signal is collected by the microphone;

[0155] A model calculation module 20, used for inputting the feature sequence into a preset recursive neural network model to perform recursive network training to obtain a model output signal, wherein the recursive neural network model is constructed based on a recursive neural network model with a gate structure;

[0156] The detection module 30 is used to confirm whether voice information is detected from the sound signal based on the model output signal and a preset fully connected network.

[0157] Optionally, the model calculation module 20 includes:

[0158] A first model training unit, used for inputting the feature sequence into a preset recursive neural network model, so as to combine the feature sequence with a previous model output signal of the recursive neural network model, perform a first cyclic network training to obtain an intermediate calculation result;

[0159] The second model training unit is used to use the intermediate calculation result as the new previous model output signal to combine the feature sequence and the new previous model output signal to perform a second recurrent network training to obtain the current model output signal.

[0160] Optionally, the detection module 30 includes:

[0161] an identification unit, used for transmitting the model output signal to the preset fully connected network to confirm whether a VAD flag is identified from the model output signal;

[0162] A first detection confirmation unit, configured to confirm that voice information is detected from the sound signal if the VAD flag is confirmed to be identified;

[0163] The first detection confirmation unit is used to confirm that no voice information is detected from the sound signal if it is confirmed that the VAD flag is not recognized.

[0164] Optionally, the identification unit is further used to perform a decoding operation on the model output signal to obtain a decoded signal; and transmit the decoded signal to the preset fully connected network.

[0165] Optionally, the decoding operation includes: data feature crossover and upsampling; the identification unit is also used to input the model output signal into a preset decoder to perform the data feature crossover and upsampling on the model output signal to obtain a decoded signal.

[0166] Optionally, the sound signal includes: a current frame sound signal collected by the microphone at the current moment, and a plurality of frames of sound signals collected by the microphone within a preset time period ending at the current moment;

[0167] The feature extraction module 10 is also used to input the current frame sound signal or the several frames of sound signals into a preset encoder to extract data features for the current frame sound signal or the several frames of sound signals, and then downsample the extracted data features to obtain a feature sequence of the current frame sound signal or the several frames of sound signals.

[0168] Optionally, the feature extraction module 10 is further used to perform time-frequency transformation processing on the current frame sound signal or the several frames of sound signals; and input the current frame sound signal or the several frames of sound signals after the time-frequency transformation processing to a preset encoder.

[0169] The specific embodiments of the various functional modules of the voice activation detection device of the present invention during operation are substantially the same as the embodiments of the voice activation detection method of the present invention described above, and are not described in detail herein.

[0170] The present invention also provides a computer storage medium on which a voice activation detection program is stored. When the voice activation detection program is executed by a processor, the steps of the voice activation detection program method described in any of the above embodiments are implemented.

[0171] The specific embodiments of the computer storage medium of the present invention are basically the same as the above-mentioned embodiments of the voice activation detection program method of the present invention, and will not be described in detail here.

[0172] The present invention also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the voice activation detection method of the present invention as described in any of the above embodiments are implemented, which will not be described in detail here.

[0173] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0174] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0175] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, including a number of instructions for a terminal device (which can be a TWS headset, etc.) to execute the methods described in each embodiment of the present invention.

[0176] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A voice activation detection method, characterized in that: The voice activation detection method comprises: When a sound signal is collected by a microphone, feature extraction is performed on the sound signal to obtain a feature sequence of the sound signal; Inputting the feature sequence into a preset recursive neural network model to perform recursive network training to obtain a model output signal, wherein the recursive neural network model is constructed based on a recursive neural network model with a gate structure; Confirming whether voice information is detected from the sound signal according to the model output signal and the preset fully connected network, the steps include: transmitting the model output signal to the preset fully connected network to confirm whether a VAD flag is recognized from the model output signal; if the VAD flag is recognized, then confirming that voice information is detected from the sound signal; or, if the VAD flag is not recognized, then confirming that voice information is not detected from the sound signal; The step of transmitting the model output signal to the preset fully connected network comprises: Inputting the model output signal into a preset decoder to perform data feature crossover on the model output signal, and upsampling the model output signal after data feature crossover by deconvolution or linear interpolation to obtain a decoded signal; Passing the decoded signal to the preset fully connected network; The sound signal includes: a current frame sound signal collected by the microphone at the current moment, and a plurality of frames of sound signals collected by the microphone in a preset time period ending at the current moment; the step of extracting features from the sound signal to obtain a feature sequence of the sound signal includes: Performing time-frequency transformation processing on the current frame sound signal or the plurality of frames sound signals by means of Fourier transformation; Inputting the current frame sound signal or the plurality of frames sound signal after the time-frequency transform processing into a preset encoder to extract data features for the current frame sound signal or the plurality of frames sound signal, wherein the encoder is a one-dimensional convolution encoder; The extracted data features are downsampled by pooling to obtain a feature sequence of the current frame sound signal or the several frames of sound signals.

2. The voice activation detection method according to claim 1, wherein: The step of inputting the feature sequence into a preset recursive neural network model for recurrent network training to obtain a model output signal comprises: Inputting the feature sequence into a preset recursive neural network model to combine the feature sequence with a previous model output signal of the recursive neural network model, and performing a first recurrent network training to obtain an intermediate calculation result; The intermediate calculation result is used as the new previous model output signal, and the feature sequence and the new previous model output signal are combined to perform a second cyclic network training to obtain the current model output signal.

3. A voice activation detection device, characterized in that: The voice activation detection device comprises: A feature extraction module is used to extract features from a sound signal to obtain a feature sequence of the sound signal when the sound signal is collected by a microphone; A model calculation module, used for inputting the feature sequence into a preset recursive neural network model to perform recursive network training to obtain a model output signal, wherein the recursive neural network model is constructed based on a recursive neural network model with a gate structure; A detection module, used to confirm whether voice information is detected from the sound signal according to the model output signal and a preset fully connected network; The sound signal includes: a current frame sound signal collected by the microphone at the current moment, and a plurality of frames of sound signals collected by the microphone in a preset time period ending at the current moment; the feature extraction module is also used for: Performing time-frequency transformation processing on the current frame sound signal or the plurality of frames sound signals by means of Fourier transformation; Inputting the current frame sound signal or the plurality of frames sound signal after the time-frequency transform processing into a preset encoder to extract data features for the current frame sound signal or the plurality of frames sound signal, wherein the encoder is a one-dimensional convolution encoder; Down-sampling the extracted data features in a pooling manner to obtain a feature sequence of the current frame sound signal or the plurality of frames of sound signals; The detection module is also used for: The model output signal is transmitted to the preset fully connected network to confirm whether a VAD flag is recognized from the model output signal; if the VAD flag is recognized, it is confirmed that voice information is detected from the sound signal; or, if the VAD flag is not recognized, it is confirmed that voice information is not detected from the sound signal; The detection module is also used for: Inputting the model output signal into a preset decoder to perform data feature crossover on the model output signal, and upsampling the model output signal after data feature crossover by deconvolution or linear interpolation to obtain a decoded signal; The decoded signal is transmitted to the preset fully connected network.

4. A terminal device, characterized in that: The terminal device comprises: a memory, a processor, and a voice activation detection program stored in the memory and executable on the processor, wherein the voice activation detection program, when executed by the processor, implements the steps of the voice activation detection method according to any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a voice activation detection program, which, when executed by a processor, implements the steps of the voice activation detection method according to any one of claims 1 to 2.

Images