Voice activity detection method and apparatus, computer device, and storage medium

Abstract

The application relates to a voice activity detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring acoustic features of each audio frame of audio sampling data; inputting the acoustic features into a pre-trained voice activity detection model to obtain intermediate features output by each feature channel of the voice activity detection model; performing information compression processing on the global feature map of each feature channel and each intermediate feature to obtain a first output corresponding to each feature channel; performing nonlinear change on the first output to obtain a second output; multiplying the second output and the intermediate features to obtain weighted features; calculating the probability that the audio frame is a voice according to the weighted features, and performing voice activity detection according to the probability. The method can improve the accuracy of voice activity detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent speech, in particular to a voice activity detection method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the development of intelligent speech technology, more and more intelligent devices begin to use intelligent speech technology to complete voice interaction, such as smart cars, smart phones and smart speakers.

[0003] Voice activity detection technology is one of the key steps of intelligent speech technology, and its purpose is to determine the voice signal from an audio file containing voice. Voice activity detection not only involves digital signal processing, but also involves auditory perception characteristics and human voice features. At the same time, the diversity of noise also increases the difficulty of voice activity detection.

[0004] In the traditional technology, the traditional convolutional recurrent neural network structure is used to model the audio timing information, so as to perform voice activity detection. However, the traditional convolutional recurrent neural network cannot enhance the expression between features, so that the detection result is not accurate enough. SUMMARY

[0005] Therefore, it is necessary to provide a voice activity detection method, device, computer equipment and storage medium capable of improving the accuracy of voice activity detection in view of the above technical problems.

[0006] A voice activity detection method, the method comprising: obtaining acoustic features of each audio frame of audio sampling data; inputting the acoustic features into a pre-trained voice activity detection model to obtain intermediate features corresponding to the output of each feature channel of the voice activity detection model; performing information compression processing on the global feature map of each feature channel and each intermediate feature to obtain a first output corresponding to each feature channel; performing nonlinear transformation on the first output to obtain a second output; multiplying the second output and the intermediate feature to obtain a weighted feature; calculating the probability that the audio frame is a voice according to the weighted feature, and performing voice activity detection according to the probability.

[0007] In some embodiments, obtaining acoustic features of each audio frame of audio sampling data comprises: obtaining audio sampling data; performing frame processing on the audio sampling data to obtain at least one audio frame; performing acoustic feature extraction on each audio frame to obtain acoustic features corresponding to each audio frame.

[0008] In some embodiments, the acoustic features are input into a pre-trained voice activity detection model to obtain intermediate features corresponding to respective feature channels of the voice activity detection model, including: sequentially performing first convolution operation, batch normalization processing, linear rectification processing and maximum pooling processing on each acoustic feature to obtain first convolution output; sequentially performing second convolution operation, batch normalization processing, linear rectification processing and maximum pooling processing on the first convolution output to obtain second convolution output; performing long short-term memory training on the second convolution output to obtain the intermediate features corresponding to respective feature channels.

[0009] In some embodiments, information compression processing is performed on the global feature map of each feature channel and each intermediate feature to obtain a first output corresponding to each feature channel, including: determining the height and width of the global feature map; generating channel statistical information corresponding to each intermediate feature by using global average pooling processing according to the height and width of the feature map, and taking the channel statistical information as the first output.

[0010] In some embodiments, a second output is obtained by performing nonlinear transformation on the first output, including: obtaining first full connection layer weights and second full connection layer weights; performing first nonlinear transformation on the first output according to the first full connection layer weights and a linear rectification activation function; performing second nonlinear transformation on the first output after the first nonlinear transformation according to the second full connection layer weights and a sigmoid growth curve activation function to obtain the second output.

[0011] In some embodiments, a probability that an audio frame is speech is calculated according to the weighted features, and voice activity detection is performed according to the probability, including: performing full connection layer mapping and exponential normalization processing on the weighted features to obtain the probability that the audio frame is speech; comparing the probability with a preset threshold, and if the probability is greater than the preset threshold, determining that the audio frame is a frame containing voice activity.

[0012] In some embodiments, the training method of the voice activity detection model includes: obtaining vehicle noise, the vehicle noise including at least one of noise when a vehicle is driving on a highway with a window open, noise when the vehicle is driving on the highway with the window closed, noise when the vehicle is in a parking lot environment with the window open, and noise when the vehicle is in the parking lot environment with the window closed; generating an audio sample set according to the vehicle noise; performing parameter optimization on the voice activity detection model by using the audio sample set and minimizing a cross-entropy loss function; and performing model training on the voice activity detection model by using the audio sample set and a self-adaptive moment estimation optimization algorithm.

[0013] A voice activity detection device, the device comprising:

[0014] An acoustic feature acquisition module configured to acquire acoustic features of each audio frame of audio sampling data;

[0015] an intermediate feature calculation module configured to input the acoustic feature into a pre-trained voice activity detection model to obtain intermediate features corresponding to respective feature channels of the voice activity detection model;

[0016] a feature compression module configured to perform information compression processing on the global feature map of each feature channel and the intermediate features to obtain first outputs corresponding to the respective feature channels;

[0017] a feature excitation module configured to perform nonlinear transformation on the first outputs to obtain second outputs;

[0018] a feature weighting module configured to multiply the second outputs by the intermediate features to obtain weighted features;

[0019] a voice activity detection module configured to calculate a probability that the audio frame is a voice based on the weighted features and perform voice activity detection based on the probability.

[0020] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the steps of the voice activity detection method of any one of the above embodiments when executing the computer program.

[0021] A computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the voice activity detection method of any one of the above embodiments when executed by a processor.

[0022] The voice activity detection method, device, computer device, and storage medium described above obtain acoustic features of each audio frame of audio sampling data, obtain intermediate features based on a pre-trained voice activity detection model, perform information compression processing and nonlinear transformation on the intermediate features, multiply the processed results by the intermediate features as weights, and obtain weighted features for voice activity detection. By using the compression and excitation method, the modeling with new inter-channel dependency is generated by assigning different weights to different feature channels, and the voice activity detection results under different feature expressions are weighted. This method is applied to voice activity detection, effectively improves the performance of separating voice segments and non-voice segments in the signal, and improves the accuracy of voice activity detection. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A flowchart of a voice activity detection method in an embodiment;

[0024] Figure 2 A structural diagram of a voice activity detection model in an embodiment;

[0025] Figure 3A structural schematic diagram of a channel attention unit in one embodiment;

[0026] Figure 4 A structural block diagram of a voice activity detection device in one embodiment;

[0027] Figure 5 An internal structural diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application.

[0029] The voice activity detection method provided by the present application can be used in a computer device. It should be noted that the execution subject can be a configuration device of a virtual network card resource. The device can be implemented by software, hardware or a combination of software and hardware to become part or all of the computer device. The computer device can be a terminal or a client or a server. The server can be a single server or a server cluster composed of multiple servers. The terminal in the embodiments of the present application can be a vehicle terminal, a smart phone, a smart speaker, a personal computer, a tablet computer, a wearable device and a smart robot or other smart hardware devices. In the following method embodiments, the execution subject is taken as an example of a computer device.

[0030] In one embodiment, as shown in Figure 1 A voice activity detection method is provided. The method is taken as an example of being applied to a vehicle terminal and includes the following steps.

[0031] Step S102: Acquire acoustic features of each audio frame of audio sampling data.

[0032] The audio sampling data refers to data sampled from the audio to be detected. The audio to be detected can be real-time collected or pre-recorded audio. The acoustic features refer to features containing acoustic information. Specifically, the vehicle terminal can perform voice signal processing on the audio sampling data, for example, can perform voice framing and other processing, thereby obtaining at least one audio frame. Then, acoustic features of each audio frame are extracted respectively to obtain acoustic features corresponding to each audio frame.

[0033] In some embodiments, acquiring acoustic features of each audio frame of audio sampling data includes: acquiring audio sampling data; performing framing on the audio sampling data to obtain at least one audio frame; performing feature extraction on each audio frame to obtain acoustic features corresponding to each audio frame.

[0034] Step S104: input the acoustic features into the pre-trained voice activity detection model to obtain intermediate features corresponding to respective feature channels of the voice activity detection model.

[0035] The voice activity detection model can be constructed based on a neural network model, for example, can be constructed based on a basic framework of a convolutional recurrent neural network, and a convolutional recurrent neural network based on a channel attention mechanism.

[0036] Specifically, the extracted acoustic features of each audio frame can be input into the pre-trained voice activity detection model, the output tensor of the single-layer LSTM (Long Short Term Memory) of the voice activity detection model can be obtained, and the output tensor of the single-layer LSTM can be dimensionally arranged to obtain the intermediate features corresponding to respective feature channels.

[0037] Exemplarily, the shape of the intermediate feature matrix U can be [batch_size x seq_len, channel_num, hight, width], where batch_size is the batch size, seq_len is the sequence length, hight is the height, and width is the width. U = [u1, u2, …, u c , c is the number of feature channels.

[0038] Step S106: performing information compression processing on the global feature map of each feature channel and each intermediate feature to obtain a first output corresponding to each feature channel.

[0039] The first output refers to the output corresponding to each feature channel after the intermediate features of each feature channel are subjected to the compression (Squeeze) processing.

[0040] In some embodiments, the information compression processing on the global feature map of each feature channel and each intermediate feature to obtain the first output corresponding to each feature channel comprises: determining the height and width of the global feature map; generating channel statistical information corresponding to each intermediate feature based on the height and width of the feature map and based on global average pooling processing, and taking the channel statistical information as the first output.

[0041] Exemplarily, the information compression processing can be performed according to the following formula:

[0042]

[0043] wherein H represents the height of the global feature map, W represents the width of the global feature map, u c represents the intermediate feature corresponding to the feature channel c. z c represents the feature corresponding to the feature channel c after the feature compression processing, that is, the first output.

[0044] In this step, based on the feature compression processing, the feature channel statistical information is generated by using the global average pool, so as to compress the global spatial information in the global feature map into the channel descriptor, so as to enhance the dependence between the feature channels, filter and emphasize the important information.

[0045] Step S108: performing nonlinear transformation on the first output to obtain a second output.

[0046] Specifically, the first output obtained through the compression processing can be subjected to an excitation processing, for example, two nonlinear transformations can be performed to obtain an excitation weight for activating each layer channel, that is, the second output.

[0047] In some embodiments, performing nonlinear transformation on the first output to obtain a second output comprises: obtaining a first fully connected layer weight and a second fully connected layer weight; performing first nonlinear transformation on the first output according to the first fully connected layer weight and a linear rectifier activation function; performing second nonlinear transformation on the first output subjected to the first nonlinear transformation according to the second fully connected layer weight and a sigmoid growth curve activation function to obtain the second output.

[0048] Exemplarily, the excitation processing can be implemented by utilizing the information aggregated in the compression processing, and is obtained through two layers of nonlinear transformation, and specific reference can be made to the following formula:

[0049] s = σ(W2δ(W1z))

[0050] Wherein, z is the output result of the compression operation, W1 is the first fully connected layer weight, δ is the linear rectifier activation function (ReLU activation function), W2 is the second layer fully connected weight, and σ is the sigmoid growth curve activation function (sigmoid activation function). Wherein, W1 and W2 are trainable parameters, which can be adjusted through model training.

[0051] Step S110: multiplying the second output with the intermediate feature to obtain a weighted feature.

[0052] Specifically, the second output of each feature channel obtained through the excitation processing can be taken as the weight value of the intermediate feature corresponding to each feature channel, and the second output of each feature channel is multiplied by the intermediate feature corresponding to each feature channel, which is equivalent to multiplying the intermediate feature of each channel by a weight value, and the weight value reflects the importance of the information contained in each channel, so as to weight the feature expression of each channel and further emphasize the important information.

[0053] Exemplarily, after the above squeezing processing and excitation processing, the weighted feature of the final output can be expressed as:

[0054]

[0055] in, The output of the excitation processing s c with u c Multiplication is equivalent to multiplying the intermediate features of each feature channel in the intermediate feature matrix U by a weight value, which reflects the importance of the information contained in the feature channel. Furthermore, the units performing the squeezing process and the activation process can be inserted into the convolutional recurrent neural network model after the nonlinear processing following each convolution process, thereby achieving integration into the standard model architecture.

[0056] Step S112: Calculate the probability that the audio frame is speech based on the weighted features, and perform speech activity detection based on the probability.

[0057] Specifically, the weighted features can be mapped through a fully connected layer and normalized, for example, by Softmax, to obtain the probability of judging whether an audio frame is speech or not. Finally, the probability of an audio frame being speech can be compared with a preset threshold to obtain the result of speech activity detection.

[0058] In some embodiments, the probability that an audio frame is speech is calculated based on weighted features, and speech activity detection is performed based on the probability, including: mapping the weighted features to a fully connected layer and exponentially normalizing them to obtain the probability that the audio frame is speech; comparing the probability with a preset threshold; if the probability is greater than the preset threshold, the audio frame is determined to be a frame containing speech activity; if the probability is less than the preset threshold, the audio frame is determined to be a frame that does not contain speech activity.

[0059] The aforementioned speech activity detection method acquires the acoustic features of each audio frame of the audio sampling data, obtains intermediate features based on a pre-trained speech activity detection model, performs information compression and nonlinear transformation on the intermediate features, and multiplies the processed results with the intermediate features as weights to obtain weighted features for speech activity detection. This approach, utilizing compression and excitation, generates a new model of inter-channel dependencies by assigning different weights to different feature channels. This weights the speech activity detection results under different feature representations. When applied to speech activity detection, this method effectively improves the performance of separating speech segments from non-speech segments in the signal, thereby enhancing the accuracy of speech activity detection.

[0060] In some embodiments, acoustic features are input into a pre-trained speech activity detection model to obtain intermediate features output from each feature channel of the speech activity detection model, including: performing a first convolution operation, batch normalization, linear rectification, and max pooling on the acoustic features in sequence to obtain a first convolution output; performing a second convolution operation, batch normalization, linear rectification, and max pooling on the first convolution output in sequence to obtain a second convolution output; and training the second convolution output with long short-term memory to obtain intermediate features output from each feature channel.

[0061] For example, a convolutional neural network model based on the channel attention mechanism can be constructed as a speech activity detection model. The embodiments of this application creatively adjust the basic framework of the convolutional recurrent neural network, combining the channel attention mechanism with the basic framework of the convolutional recurrent neural network. By fusing the CNN (convolutional neural network) and LSTM network structures, more meaningful feature representations are generated, and adjustments are made to each layer of the model. See [link to relevant documentation]. Figure 2 As shown, Figure 2 A schematic diagram of the structure of a speech activity detection model in one embodiment is shown.

[0062] from Figure 2 As can be seen, the model can be composed of a first convolutional layer with a 5×5 kernel and a second convolutional layer with a 3×3 kernel. The first convolutional layer is used for the first convolution operation, and the second convolutional layer is used for the second convolution operation. After each convolutional layer, there is a Batch Normalization layer, a ReLU activation function layer, and a 2×2 max pooling layer. The Batch Normalization layer is used for batch normalization, so that the input of each layer of the neural network maintains the same distribution during the training of the deep neural network. The ReLU activation function layer is used for linear rectification, and the max pooling layer is used for max pooling. Then, a single-layer LSTM layer and a channel attention unit are connected. The single-layer LSTM layer and the channel attention unit perform the step of processing the intermediate features to obtain weighted features as described in any of the above embodiments. Then, the weighted features output by the channel attention unit are processed by two fully connected layers and a tanh activation function layer to obtain the probability that the audio frame is speech.

[0063] In some embodiments, reference Figure 3 As shown, Figure 3 A schematic diagram of the channel attention unit in one embodiment is shown.

[0064] like Figure 3As shown, information compression processing can be performed based on the global pooling layer and the global feature maps of each feature channel and each intermediate feature to obtain the first output corresponding to each feature channel. The first output can be nonlinearly transformed based on the first fully connected layer, the ReLU activation function layer, the second fully connected layer and the sigmoid activation function layer to obtain the second output. The second output is multiplied with the intermediate features to finally obtain the weighted features.

[0065] In some embodiments, the training method for the speech activity detection model includes: acquiring vehicle noise, which includes at least one of the following: noise when the vehicle is driving on a highway with the window open, noise when the vehicle is driving on a highway with the window closed, noise when the vehicle is driving in a parking lot with the window open, and noise when the vehicle is driving in a parking lot with the window closed; generating an audio sample set based on the vehicle noise; optimizing the parameters of the speech activity detection model based on the audio sample set and minimizing the cross-entropy loss function; and training the speech activity detection model based on the audio sample set and an adaptive moment estimation optimization algorithm (ADAM optimization algorithm). Through the above model training method, the model's fit for detecting vehicle speech activity can be improved, thereby further improving detection accuracy.

[0066] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0067] In one embodiment, such as Figure 4 As shown, a speech activity detection device is provided, including: an acoustic feature acquisition module 402, an intermediate feature calculation module 404, a feature compression module 406, a feature excitation module 408, a feature weighting module 410, and a speech activity detection module 412, wherein:

[0068] The acoustic feature acquisition module 402 is used to acquire the acoustic features of each audio frame of the audio sampling data;

[0069] The intermediate feature calculation module 404 is used to input the acoustic features into a pre-trained speech activity detection model to obtain the intermediate features output by each feature channel of the speech activity detection model.

[0070] The feature compression module 406 is used to perform information compression processing based on the global feature map of each feature channel and each intermediate feature to obtain the first output corresponding to each feature channel.

[0071] The feature activation module 408 is used to perform a nonlinear transformation on the first output to obtain a second output;

[0072] The feature weighting module 410 is used to multiply the second output with the intermediate feature to obtain the weighted feature;

[0073] The speech activity detection module 412 is used to calculate the probability that the audio frame is speech based on the weighted features, and to perform speech activity detection based on the probability.

[0074] In some embodiments, the acoustic feature acquisition module 402 acquires audio sampling data; performs frame-segmentation on the audio sampling data to obtain at least one audio frame; and extracts acoustic features from each audio frame to obtain the acoustic features corresponding to each audio frame.

[0075] In some embodiments, the intermediate feature calculation module 404 sequentially performs a first convolution operation, batch normalization, linear rectification, and max pooling on each acoustic feature to obtain a first convolution output; sequentially performs a second convolution operation, batch normalization, linear rectification, and max pooling on the first convolution output to obtain a second convolution output; and trains the second convolution output using long short-term memory to obtain intermediate features corresponding to the output of each feature channel.

[0076] In some embodiments, the feature compression module 406 determines the height and width of the global feature map; based on the height and width of the feature map and using global average pooling, it generates channel statistics corresponding to each intermediate feature, and uses the channel statistics as the first output.

[0077] In some embodiments, the feature activation module 408 obtains the weights of the first fully connected layer and the weights of the second fully connected layer; performs a first nonlinear transformation on the first output according to the weights of the first fully connected layer and the linear rectification activation function; and performs a second nonlinear transformation on the first output after the first nonlinear transformation according to the weights of the second fully connected layer and the S-shaped growth curve activation function to obtain the second output.

[0078] In some embodiments, the feature weighting module 410 performs fully connected layer mapping and exponential normalization on the weighted features to obtain the probability that the audio frame is speech; the probability is compared with a preset threshold, and if the probability is greater than the preset threshold, the audio frame is determined to be a frame containing speech activity.

[0079] In some embodiments, the speech activity detection module 412 is further used for training a speech activity detection model, specifically including: acquiring vehicle noise, which includes at least one of the following: noise when the vehicle is driving on a highway with the window open, noise when the vehicle is driving on a highway with the window closed, noise when the vehicle is driving in a parking lot with the window open, and noise when the vehicle is driving in a parking lot with the window closed; generating an audio sample set based on the vehicle noise; optimizing the parameters of the speech activity detection model using the audio sample set and minimizing the cross-entropy loss function; and training the speech activity detection model using the audio sample set and an adaptive moment estimation optimization algorithm.

[0080] Specific limitations regarding the voice activity detection device can be found in the limitations of the voice activity detection method described above, and will not be repeated here. Each module in the aforementioned voice activity detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0081] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a voice activity detection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0082] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0083] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: inputting acoustic features into a pre-trained speech activity detection model to obtain intermediate features corresponding to the outputs of each feature channel of the speech activity detection model; performing information compression processing based on the global feature maps of each feature channel and each intermediate feature to obtain a first output corresponding to each feature channel; performing a nonlinear transformation on the first output to obtain a second output; multiplying the second output with the intermediate features to obtain weighted features; calculating the probability that an audio frame is speech based on the weighted features; and performing speech activity detection based on the probability.

[0084] In one embodiment, when the processor executes a computer program to acquire the acoustic features of each audio frame of the audio sampling data, the specific implementation includes: acquiring the audio sampling data; performing frame-segmentation on the audio sampling data to obtain at least one audio frame; and extracting acoustic features from each audio frame to obtain the acoustic features corresponding to each audio frame.

[0085] In one embodiment, when the processor executes a computer program to input acoustic features into a pre-trained speech activity detection model and obtain the intermediate features corresponding to the output of each feature channel of the speech activity detection model, the specific implementation is as follows: each acoustic feature is sequentially subjected to a first convolution operation, batch normalization processing, linear rectification processing, and max pooling processing to obtain a first convolution output; the first convolution output is sequentially subjected to a second convolution operation, batch normalization processing, linear rectification processing, and max pooling processing to obtain a second convolution output; the second convolution output is trained using long short-term memory to obtain the intermediate features corresponding to the output of each feature channel.

[0086] In one embodiment, when the processor executes a computer program to perform information compression processing based on the global feature map of each feature channel and each intermediate feature to obtain the first output corresponding to each feature channel, the specific implementation is as follows: determine the height and width of the global feature map; based on the height and width of the feature map and using global average pooling processing, generate channel statistics information corresponding to each intermediate feature, and use the channel statistics information as the first output.

[0087] In one embodiment, when the processor executes a computer program to perform a nonlinear transformation on the first output to obtain a second output, the specific implementation includes: obtaining the weights of the first fully connected layer and the weights of the second fully connected layer; performing a first nonlinear transformation on the first output based on the weights of the first fully connected layer and a linear rectification activation function; and performing a second nonlinear transformation on the first output after the first nonlinear transformation based on the weights of the second fully connected layer and an S-shaped growth curve activation function to obtain the second output.

[0088] In one embodiment, the processor executes a computer program to calculate the probability that an audio frame is speech based on weighted features. When detecting speech activity based on this probability, the specific implementation includes: mapping the weighted features to a fully connected layer and performing exponential normalization to obtain the probability that the audio frame is speech; comparing the probability with a preset threshold; if the probability is greater than the preset threshold, then determining that the audio frame contains speech activity. In another embodiment, the processor executes the computer program to further implement the following steps: acquiring vehicle noise, including at least one of the following: noise when the vehicle is driving on a highway with the window open, noise when the vehicle is driving on a highway with the window closed, noise when the vehicle is driving in a parking lot with the window open, and noise when the vehicle is driving in a parking lot with the window closed; generating an audio sample set based on the vehicle noise; optimizing the parameters of the speech activity detection model using the audio sample set and minimizing the cross-entropy loss function; and training the speech activity detection model using the audio sample set and an adaptive moment estimation optimization algorithm.

[0089] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: inputting acoustic features into a pre-trained speech activity detection model to obtain intermediate features corresponding to the outputs of each feature channel of the speech activity detection model; performing information compression processing based on the global feature maps of each feature channel and each intermediate feature to obtain a first output corresponding to each feature channel; performing a nonlinear transformation on the first output to obtain a second output; multiplying the second output with the intermediate features to obtain weighted features; calculating the probability that an audio frame is speech based on the weighted features; and performing speech activity detection based on the probability.

[0090] In one embodiment, when a computer program is executed by a processor to acquire the acoustic features of each audio frame of audio sampling data, the specific implementation includes: acquiring audio sampling data; performing frame-segmentation processing on the audio sampling data to obtain at least one audio frame; and extracting acoustic features from each audio frame to obtain the acoustic features corresponding to each audio frame.

[0091] In one embodiment, when a computer program is executed by a processor to input acoustic features into a pre-trained speech activity detection model and obtain intermediate features corresponding to the output of each feature channel of the speech activity detection model, the specific implementation is as follows: each acoustic feature is sequentially subjected to a first convolution operation, batch normalization processing, linear rectification processing, and max pooling processing to obtain a first convolution output; the first convolution output is sequentially subjected to a second convolution operation, batch normalization processing, linear rectification processing, and max pooling processing to obtain a second convolution output; the second convolution output is trained using long short-term memory to obtain intermediate features corresponding to the output of each feature channel.

[0092] In one embodiment, when a computer program is executed by a processor to perform information compression processing based on the global feature map of each feature channel and each intermediate feature to obtain the first output corresponding to each feature channel, the specific implementation is as follows: determine the height and width of the global feature map; based on the height and width of the feature map and using global average pooling processing, generate channel statistics information corresponding to each intermediate feature, and use the channel statistics information as the first output.

[0093] In one embodiment, when a computer program is executed by a processor to perform a nonlinear transformation on a first output to obtain a second output, the specific implementation includes: obtaining the weights of a first fully connected layer and a second fully connected layer; performing a first nonlinear transformation on the first output based on the weights of the first fully connected layer and a linear rectification activation function; and performing a second nonlinear transformation on the first output after the first nonlinear transformation based on the weights of the second fully connected layer and an S-shaped growth curve activation function to obtain the second output.

[0094] In one embodiment, the computer program executed by the processor calculates the probability that an audio frame is speech based on weighted features. When detecting speech activity based on the probability, the specific implementation includes: mapping the weighted features through a fully connected layer and performing exponential normalization to obtain the probability that the audio frame is speech; comparing the probability with a preset threshold; if the probability is greater than the preset threshold, then determining that the audio frame contains speech activity. In another embodiment, the computer program executed by the processor further implements the following steps: acquiring vehicle noise, including at least one of the following: noise when the vehicle is driving on a highway with the window open, noise when the vehicle is driving on a highway with the window closed, noise when the vehicle is driving in a parking lot with the window open, and noise when the vehicle is driving in a parking lot with the window closed; generating an audio sample set based on the vehicle noise; optimizing the parameters of the speech activity detection model using the audio sample set and minimizing the cross-entropy loss function; and training the speech activity detection model using the audio sample set and an adaptive moment estimation optimization algorithm.

[0095] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0096] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] Terms such as “first” and “second” used in this application are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual sequential relationship or order between these entities or operations.

[0098] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting speech activity, the method comprising: Acquire the acoustic features of each audio frame of the audio sampling data; The acoustic features are input into a pre-trained speech activity detection model to obtain the intermediate features output by each feature channel of the speech activity detection model. Information compression processing is performed based on the global feature map of each feature channel and each intermediate feature to obtain the first output corresponding to each feature channel; The first output is subjected to a nonlinear transformation to obtain the second output; Multiply the second output by the intermediate feature to obtain the weighted feature; The probability that the audio frame is speech is calculated based on the weighted features, and speech activity detection is performed based on the probability.

2. The method according to claim 1, characterized in that, The acoustic features of each audio frame of the acquired audio sampling data include: Acquire audio sampling data; The audio sampling data is processed into frames to obtain at least one audio frame; Acoustic features are extracted from each of the audio frames to obtain the acoustic features corresponding to each audio frame.

3. The method according to claim 1, characterized in that, The step of inputting the acoustic features into a pre-trained speech activity detection model to obtain intermediate features corresponding to each feature channel of the speech activity detection model includes: Each of the acoustic features is sequentially subjected to a first convolution operation, batch normalization, linear rectification, and max pooling to obtain the first convolution output. The first convolutional output is sequentially subjected to a second convolution operation, batch normalization, linear rectification, and max pooling to obtain the second convolutional output. The output of the second convolution is trained using Long Short-Term Memory to obtain the intermediate features corresponding to the output of each feature channel.

4. The method according to claim 1, characterized in that, The step of performing information compression processing based on the global feature maps of each feature channel and each intermediate feature to obtain the first output corresponding to each feature channel includes: Determine the height and width of the global feature map; Based on the height and width of the feature map, and using global average pooling, channel statistics corresponding to each intermediate feature are generated, and the channel statistics are used as the first output.

5. The method according to claim 1, characterized in that, The step of performing a nonlinear transformation on the first output to obtain the second output includes: Obtain the weights of the first fully connected layer and the weights of the second fully connected layer; The first output is subjected to a first nonlinear transformation based on the weights of the first fully connected layer and the linear rectified activation function. The second output is obtained by applying a second nonlinear transformation to the first output after the first nonlinear transformation based on the weights of the second fully connected layer and the S-shaped growth curve activation function.

6. The method according to claim 1, characterized in that, The step of calculating the probability that the audio frame is speech based on the weighted features, and performing speech activity detection based on the probability, includes: The weighted features are mapped using a fully connected layer and normalized exponentially to obtain the probability that the audio frame is speech. The probability is compared with a preset threshold. If the probability is greater than the preset threshold, the audio frame is determined to be a frame containing voice activity.

7. The method according to claim 1, characterized in that, The training method for the speech activity detection model includes: The vehicle noise is obtained, including at least one of the following: noise when the vehicle is driving on the highway with the windows open, noise when the vehicle is driving on the highway with the windows closed, noise when the vehicle is driving in a parking lot with the windows open, and noise when the vehicle is driving in a parking lot with the windows closed. An audio sample set is generated based on the vehicle noise; The parameters of the speech activity detection model are optimized using the audio sample set and by minimizing the cross-entropy loss function. The speech activity detection model is trained using the audio sample set and the adaptive torque estimation optimization algorithm.

8. A voice activity detection device, characterized in that, The device includes: The acoustic feature acquisition module is used to acquire the acoustic features of each audio frame of the audio sampling data; The intermediate feature calculation module is used to input the acoustic features into a pre-trained speech activity detection model to obtain the intermediate features output by each feature channel of the speech activity detection model. The feature compression module is used to perform information compression processing based on the global feature map of each feature channel and each intermediate feature to obtain the first output corresponding to each feature channel. The feature activation module is used to perform a nonlinear transformation on the first output to obtain the second output; The feature weighting module is used to multiply the second output with the intermediate features to obtain weighted features; The speech activity detection module is used to calculate the probability that the audio frame is speech based on the weighted features, and to perform speech activity detection based on the probability.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images