A voice activity detection method, system, terminal and storage medium

Abstract

The application provides a voice activity detection method, system, terminal and storage medium, relates to a voice data processing technical field, and in particular to a voice activity detection method, which comprises the following steps: obtaining a voice training sample carrying a voice label; performing feature extraction according to the voice training sample to obtain a root amplitude spectrum feature sample and an Fbank feature sample; and constructing a voice activity detection model, wherein the voice activity detection model is used for performing feature fusion on the root amplitude spectrum feature and the Fbank feature obtained by performing feature extraction on voice data to obtain fusion features, and outputting a probability value of each frame of the voice data existing a voice signal based on the fusion features; training the voice activity detection model by using the root amplitude spectrum feature sample and the Fbank feature sample to obtain a trained voice activity detection model; and performing voice activity detection by using the trained voice activity detection model. The application can improve the accuracy of voice activity detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice data processing, in particular to a voice activity detection method and system, a terminal and a storage medium. BACKGROUND

[0002] Voice activity detection (VAD) is an audio signal processing technology for detecting whether voice exists, which is usually used in speaker positioning, voice communication, speech recognition and other scenarios, and can also be used for cleaning data sets to serve voice enhancement models and large voice models. Through voice activity detection technology, the probability of voice existence can be output on the voice frame level data, so that other audio processing technologies can be more effectively applied when voice appears, which not only helps to save computing resources, but also optimizes the performance and efficiency of the entire system.

[0003] At present, common voiceprint features include MFCC (Mel Frequency Cepstral Coefficient) and Fbank (FilterBank). In the voice activity detection method based on deep learning, the current commonly used model construction method is mostly based on a single type of acoustic feature for model construction, and the model detection accuracy needs to be improved.

[0004] Therefore, how to improve the accuracy of voice activity detection is a technical problem to be solved by those skilled in the art. SUMMARY

[0005] To solve the above technical problems, the present application provides a voice activity detection method which can improve the accuracy of voice activity detection. The present application also provides a voice activity detection system, a terminal and a storage medium, which have the same technical effect.

[0006] The first object of the present application is to provide a voice activity detection method.

[0007] The above application object of the present application is achieved by the following technical scheme:

[0008] A voice activity detection method comprises:

[0009] obtaining a voice training sample carrying a voice label, wherein the voice label is used to represent whether each frame of the voice training sample exists a voice signal;

[0010] performing feature extraction according to the voice training sample to obtain a root amplitude spectrum feature sample and an Fbank feature sample;

[0011] The voice activity detection model is constructed, wherein the voice activity detection model is used for performing feature fusion on root amplitude spectrum features and Fbank features obtained by performing feature extraction on voice data, obtaining fused features, and outputting a probability value of each frame of the voice data existing a voice signal based on the fused features.

[0012] The voice activity detection model is trained by using the root amplitude spectrum feature sample and the Fbank feature sample, to obtain a trained voice activity detection model.

[0013] The trained voice activity detection model is used for voice activity detection.

[0014] Preferably, in the voice activity detection method, the voice activity detection model comprises a feature fusion module, N feature processing modules and an output module connected in sequence, N being a positive integer, wherein:

[0015] The feature fusion module is used for performing feature fusion on root amplitude spectrum features and Fbank features obtained by performing feature extraction on voice data, to obtain fused features.

[0016] The N feature processing modules are used for processing the fused features to obtain output features.

[0017] The output module is used for obtaining a probability value of each frame of the voice data existing a voice signal according to the output features.

[0018] Preferably, in the voice activity detection method, the feature fusion module comprises a first linear layer, a gated convolution unit, a batch normalization layer and a first parameterized ReLU layer, and the feature fusion module is specifically used for:

[0019] The first linear layer is used for performing filter processing on root amplitude spectrum features to obtain high-dimensional features.

[0020] The high-dimensional features and Fbank features are spliced to obtain spliced features.

[0021] The gated convolution unit is used for processing the spliced features to obtain first features.

[0022] The batch normalization layer and the first parameterized ReLU layer are used for processing the first features to obtain the fused features.

[0023] Preferably, in the voice activity detection method, each feature processing module comprises two first feature processing sub-modules, one second feature processing sub-module and a first layer normalization layer connected in sequence, wherein:

[0024] Each of the first feature processing sub-modules comprises a first data shaping layer, a grouped linear layer, a second parameterized ReLU layer and a second layer normalization layer, and the first feature processing sub-module is specifically configured to:

[0025] The first data shaping layer is configured to perform data shaping on the input feature to obtain a second feature.

[0026] The grouped linear layer is configured to process the second feature to obtain a third feature.

[0027] The second parameterized ReLU layer and the second layer normalization layer are configured to process the third feature to obtain a first processed feature.

[0028] The second feature processing sub-module comprises a second data shaping layer, a grouped gated recurrent unit and a third layer normalization layer, and the second feature processing sub-module is specifically configured to:

[0029] The second data shaping layer is configured to perform data shaping on the first processed feature output by the second first feature processing sub-module connected in sequence to obtain a fourth feature.

[0030] The grouped gated recurrent unit is configured to process the fourth feature to obtain a fifth feature.

[0031] The third layer normalization layer is configured to process the fifth feature to obtain a second processed feature.

[0032] The first layer normalization layer is configured to process a third processed feature obtained by summing the first processed feature and the second processed feature output by the second first feature processing sub-module connected in sequence to obtain an output feature of the feature processing module.

[0033] Preferably, in the voice activity detection method, the output module comprises a second linear layer, and the output module is specifically configured to:

[0034] The second linear layer and a Sigmoid activation function are configured to process the output feature to obtain a probability value of each frame of the voice data existing a voice signal.

[0035] Preferably, in the voice activity detection method, the voice training sample carrying a voice label is obtained by:

[0036] A clean voice sample is obtained, and a VAD algorithm in WebRTC is used to detect whether a voice signal exists in the clean voice sample to obtain a detection result.

[0037] According to the detection result, the voice label is added to the clean voice sample.

[0038] The clean speech samples are subjected to data enhancement processing to obtain the speech training samples.

[0039] Preferably, in the voice activity detection method, the feature extraction based on the speech training samples to obtain the root amplitude spectrum feature samples and the Fbank feature samples comprises:

[0040] The speech training samples are subjected to framing, windowing and fast Fourier transform, and the amplitude spectrum samples are obtained by taking the modulus;

[0041] The amplitude spectrum samples are squared, and then subjected to a mel filter bank and logarithm to obtain the Fbank feature samples;

[0042] The amplitude spectrum samples are taken to the root to obtain the root amplitude spectrum feature samples.

[0043] The second object of the present application is to provide a voice activity detection system.

[0044] The above-mentioned second object of the present application is achieved by the following technical scheme:

[0045] A voice activity detection system comprises:

[0046] An acquisition unit is configured to acquire speech training samples carrying voice labels, wherein the voice labels are used to represent whether each frame of the speech training samples contains a voice signal;

[0047] An extraction unit is configured to perform feature extraction based on the speech training samples to obtain root amplitude spectrum feature samples and Fbank feature samples;

[0048] A construction unit is configured to construct a voice activity detection model, wherein the voice activity detection model is used to perform feature fusion on the root amplitude spectrum features and the Fbank features obtained by feature extraction on voice data to obtain fusion features, and output a probability value of each frame of the voice data containing a voice signal based on the fusion features;

[0049] A training unit is configured to train the voice activity detection model using the root amplitude spectrum feature samples and the Fbank feature samples to obtain a trained voice activity detection model;

[0050] A detection unit is configured to perform voice activity detection using the trained voice activity detection model.

[0051] The third object of the present application is to provide a voice activity detection terminal.

[0052] The above-mentioned third object of the present application is achieved by the following technical scheme:

[0053] A voice activity detection terminal comprises a storage medium and a processor.

[0054] The storage medium stores computer execution instructions.

[0055] The processor executes the computer execution instructions stored in the storage medium to implement the voice activity detection method as described above.

[0056] A fourth object of the present application is to provide a computer-readable storage medium.

[0057] The fourth object of the present application is achieved by the following technical solution.

[0058] A computer-readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the voice activity detection method as described above.

[0059] The above technical solution obtains a voice training sample carrying a voice label; performs feature extraction according to the voice training sample to obtain a root amplitude spectrum feature sample and an Fbank feature sample; wherein the high-frequency energy of the voice is generally smaller than the low-frequency energy, that is, the voice energy is mainly concentrated in the low frequency, and the root amplitude spectrum feature can reduce the high-low frequency amplitude difference; further, a voice activity detection model is constructed, the root amplitude spectrum feature sample and the Fbank feature sample are used to train the voice activity detection model to obtain a trained voice activity detection model; the trained voice activity detection model is used to perform voice activity detection; wherein the voice activity detection model is used to perform feature fusion on the root amplitude spectrum feature and the Fbank feature obtained by feature extraction on voice data to obtain fusion features, and output a probability value of each frame of the voice data existing a voice signal based on the fusion features; compared with the model construction method based on a single type of acoustic feature, the above technical solution can improve the effectiveness of the extracted features by fusing the root amplitude spectrum feature and the Fbank feature, and further improve the accuracy of voice activity detection. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0061] Figure 1 A flowchart of a voice activity detection method in an embodiment of the present application;

[0062] Figure 2 FIG. 1 is a structural schematic diagram of a voice activity detection model in an embodiment of the present application;

[0063] Figure 3 FIG. 2 is a structural schematic diagram of a feature fusion module in an embodiment of the present application;

[0064] Figure 4 FIG. 3 is a structural schematic diagram of a feature processing module in an embodiment of the present application;

[0065] Figure 5 FIG. 4 is a structural schematic diagram of a voice activity detection system in an embodiment of the present application;

[0066] Figure 6 FIG. 5 is a structural schematic diagram of a voice activity detection terminal in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to enable persons skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work fall within the scope of protection of the present application.

[0068] In the embodiments provided in the present application, it should be understood that the disclosed method and system can be implemented in other ways. The system embodiments described below are only schematic. For example, the division of the modules is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0069] It should be understood that if "system", "device", "unit" and / or "module" are used in the present application, it is only a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0070] In addition, the terms "first", "second", etc. are used only for descriptive purposes and do not connote or imply relative importance or a quantity of the indicated technical features. Thus, a feature defined with "first", "second", etc. can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" or "several" is two or more, unless explicitly specifically defined otherwise.

[0071] If a flowchart is used in the present application, the flowchart is used to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or subsequent operations are not necessarily performed in sequence. Instead, each step can be processed in reverse order or simultaneously. Meanwhile, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0072] It should also be noted that in this document, terms such as "comprise", "comprising", or any other variant thereof are intended to cover non-exclusive inclusions, so that an item or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such item or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the item or device comprising the above element.

[0073] Embodiments of the present application are written in a progressive manner.

[0074] As shown in the Figure 1 The embodiments of the present application provide a voice activity detection method, which comprises:

[0075] S101. Obtain a voice training sample carrying a voice label;

[0076] In S101, specifically, the voice training sample can be a voice data set with a voice label. The voice label is used to represent whether there is a voice signal in each frame of the voice training sample. For example, the voice label can adopt a 0-1 time domain binary label. For single frame data in the voice training sample, if the time domain binary label is 1, it means that there is a voice signal in the single frame data, and if the time domain binary label is 0, it means that there is no voice signal in the single frame data. The present application is not limited to this.

[0077] In some embodiments, one implementation of the present step specifically comprises:

[0078] S1011. Obtain a clean voice sample, and use the VAD algorithm in WebRTC to detect whether there is a voice signal in the clean voice sample, to obtain a detection result;

[0079] In S1011, specifically, the clean speech sample can be the cleaned anechoic chamber recording voice data; the VAD algorithm in WebRTC models voice and noise based on Gaussian Mixed Model (GMM). GMM is an unsupervised learning method, which can effectively distinguish voice and noise by probabilistic modeling of input data. In WebRTC, the VAD algorithm divides the input spectrum into six subbands and calculates the energy of each subband. Then, the probability density function of GMM is used to model the subband energy to obtain a log-likelihood ratio function. Through global and local judgment of the log-likelihood ratio, voice activity detection is realized. The VAD algorithm in WebRTC is used to detect the voice activity of the clean speech sample to obtain a detection result.

[0080] S1012. According to the detection result, a voice label is added to the clean speech sample;

[0081] In S1012, specifically, according to the detection result, a voice label is added to the clean speech sample to represent whether there is a voice signal in each frame of data. For example, the voice label can be a 0-1 time domain binary label. If the detection result of a single frame of data of the clean speech sample is that there is a voice signal, the voice label of the single frame of data of the clean speech sample is set to 1. If the detection result of a single frame of data of the clean speech sample is that there is no voice signal, the voice label of the single frame of data of the clean speech sample is set to 0. The application is not limited thereto.

[0082] S1013. The clean speech sample is subjected to data enhancement processing to obtain a voice training sample.

[0083] In S1013, specifically, existing data enhancement techniques can be used to perform data enhancement processing on the clean speech sample to expand the number of samples. In some embodiments, the data enhancement processing can include one or more of adding reverberation, adding noise, adjusting frequency response, and destroying frequency band information, which can simulate actual complex application scenarios and help improve the robustness of the model in different environments.

[0084] S102. According to the voice training sample, feature extraction is performed to obtain a root amplitude spectrum feature sample and a Fbank feature sample;

[0085] In S102, specifically, in order to capture different characteristics of the speech signal, feature extraction is performed on the speech training sample to obtain the root amplitude spectrum feature sample and the Fbank feature sample. Among them, the response of the human ear to the sound spectrum is nonlinear, and Fbank is a front-end processing algorithm that processes audio in a manner similar to the human ear, which can improve the performance of speech recognition; the high-frequency energy of speech is generally smaller than the low-frequency energy, that is, the speech energy is mainly concentrated in the low frequency, and by extracting the root amplitude spectrum feature, the difference between the high and low frequency amplitudes can be reduced.

[0086] In some embodiments, one implementation of the present step specifically includes: framing, windowing and fast Fourier transform are performed on the speech training sample to obtain the amplitude spectrum sample; the amplitude spectrum sample is squared, and then passed through the Mel filter bank and logarithm to obtain the Fbank feature sample; the amplitude spectrum sample is taken to the root to obtain the root amplitude spectrum feature sample.

[0087] Specifically, the speech training sample is framed, windowed and fast Fourier transformed to obtain frequency domain data, the amplitude spectrum sample is obtained by taking the modulus of the frequency domain data, then the amplitude spectrum sample is squared to obtain the power spectrum, and then the power spectrum is input to the Mel filter bank (Mel Filter Banks) for filtering, the logarithm of the obtained filter output is taken to obtain the Fbank feature sample, and the root operation is performed on the amplitude spectrum sample to obtain the root amplitude spectrum feature sample.

[0088] S103. Construct a voice activity detection model;

[0089] In S103, specifically, a voice activity detection model based on deep learning can be constructed, which is used to perform feature fusion on the root amplitude spectrum feature and the Fbank feature obtained by feature extraction on the speech data, to obtain fusion features, and based on the fusion features, output the probability value of each frame of the speech data existing speech signal; wherein the specific implementation details of obtaining the root amplitude spectrum feature and the Fbank feature by feature extraction on the speech data can be referred to the above S102; compared with the model construction method based on a single type of acoustic feature, the above voice activity detection model can improve the effectiveness of the extracted features by fusing the root amplitude spectrum feature and the Fbank feature, and further improve the accuracy of voice activity detection.

[0090] S104. Train the voice activity detection model using the root amplitude spectrum feature sample and the Fbank feature sample to obtain a trained voice activity detection model;

[0091] In S104, specifically, the existing model training method can be used to train the voice activity detection model using the root amplitude spectrum feature sample and the Fbank feature sample. When the set termination iteration condition is met, the trained voice activity detection model is obtained. Specifically, in the model training process, the AdamW optimizer and the SmoothL1Loss loss function can be used to train the model, the decay_weight is set to 5e-6, and the learning rate is set to 0.0008. The present application is not limited to this.

[0092] S105. Using the trained voice activity detection model, voice activity detection is performed.

[0093] In S105, specifically, the trained voice activity detection model can be landed on the terminal platform to perform voice activity detection on the voice data to be detected in the platform and obtain a detection result.

[0094] In some embodiments, one implementation of this step specifically includes: obtaining voice data to be detected; performing feature extraction based on the voice data to be detected to obtain root amplitude spectrum feature data and Fbank feature data; inputting the root amplitude spectrum feature data and the Fbank feature data into the trained voice activity detection model to obtain a probability value of each frame of the voice data to be detected existing a voice signal.

[0095] Specifically, the specific implementation details of feature extraction can refer to S102 described above. After inputting the root amplitude spectrum feature data and the Fbank feature data into the trained voice activity detection model, the voice activity detection model performs feature fusion based on the root amplitude spectrum feature data and the Fbank feature data to obtain fusion feature data, and outputs a probability value of each frame of the voice data to be detected existing a voice signal based on the fusion feature data.

[0096] The above embodiment, by acquiring a speech training sample carrying a speech label; according to the speech training sample, feature extraction is carried out to obtain a root amplitude spectrum feature sample and an Fbank feature sample; wherein the high frequency energy of the speech is generally smaller than the low frequency energy, that is, the speech energy is mainly concentrated in the low frequency, and by extracting the root amplitude spectrum feature, the high and low frequency amplitude difference can be reduced; further, by constructing a voice activity detection model, the root amplitude spectrum feature sample and the Fbank feature sample are used to train the voice activity detection model to obtain a trained voice activity detection model; the trained voice activity detection model is used for voice activity detection; wherein the voice activity detection model is used for feature fusion of the root amplitude spectrum feature and the Fbank feature obtained by feature extraction of the speech data, to obtain fusion features, and based on the fusion features, a probability value of each frame of the speech data existing speech signal is output, compared with the model construction mode based on a single type of acoustic feature, the above embodiment can improve the effectiveness of the extracted feature by fusing the root amplitude spectrum feature and the Fbank feature, and further improve the accuracy of the voice activity detection.

[0097] In other embodiments of the present application, as shown in Figure 2 The voice activity detection model includes a feature fusion module, N feature processing modules and an output module connected in sequence, wherein:

[0098] The feature fusion module is configured to perform feature fusion on the root amplitude spectrum feature and the Fbank feature obtained by feature extraction of the speech data to obtain fusion features; the N feature processing modules are configured to process the fusion features to obtain output features; and the output module is configured to obtain a probability value of each frame of the speech data existing speech signal according to the output features.

[0099] Specifically, N is a positive integer, the number of feature processing modules can be one or multiple, and the number of feature processing modules can be adjusted according to the size of the calculation amount, that is, the size of the voice activity detection model is adjusted; when the number of feature processing modules is one, the feature processing module is specifically configured to process the fusion features to obtain the output features; when the number of feature processing modules is multiple, the first feature processing module connected in sequence is configured to process the fusion features to obtain the output features of the feature processing module, and the output features are used as the input of the second feature processing module connected in sequence, and so on, and the output of the last feature processing module connected in sequence is used as the final output feature and is transmitted to the output module.

[0100] In other embodiments of the present application, in order to overcome the problem that the FBank feature representation is not complete due to noise and other factors in the actual collected data, and better fuse the root amplitude spectrum feature and the Fbank feature obtained by feature extraction of the speech data, as Figure 3As shown, the feature fusion module includes a first linear layer, a gated convolutional unit, a batch normalization layer, and a first parameterized ReLU layer, and the feature fusion module is specifically configured to:

[0101] The root amplitude spectrum feature is filtered by using the first linear layer to obtain a high-dimensional feature.

[0102] The high-dimensional feature and the Fbank feature are spliced to obtain a spliced feature.

[0103] The spliced feature is processed by using the gated convolutional unit to obtain a first feature.

[0104] The first feature is processed by using the batch normalization layer and the first parameterized ReLU layer to obtain a fused feature.

[0105] Specifically, the linear layer (Linear Layer) is a basic layer in a neural network, also known as a fully connected layer (Fully Connected Layer) or a dense layer (Dense Layer). The linear layer plays a role in linearly transforming input data in the neural network. First, the root amplitude spectrum of all frequency points is adaptively filtered by using the first linear layer to learn more representative high-dimensional features, and the high-dimensional features are obtained. Then, the obtained high-dimensional features and the Fbank features extracted by the feature extraction are spliced to obtain spliced features.

[0106] The gated convolutional unit (Gated Convolutional Unit, GCU) is an improved convolutional neural network structure, and the gated convolution is an advanced convolution operation that introduces a gating mechanism to enhance the ability of the model to capture complex dependencies in the data. After obtaining the spliced features, the spliced features are input into the gated convolutional unit for convolution operation to obtain the first feature, so as to realize the fusion of the high-dimensional features and the Fbank features and avoid the incomplete representation of the Fbank features.

[0107] Batch Normalization is a special layer in neural networks, and its goal is to normalize the features (make the output of each layer of the network pass through activation) to obtain a zero-mean state with a standard deviation of 1. It not only speeds up the convergence of the model, but more importantly, it to some extent alleviates the problem of "gradient disappearance" in deep network, so that it is easier and more stable to train deep network model. Parametric ReLU, also known as Parametric ReLU activation function, is an improved ReLU (Rectified Linear Unit) activation function used in neural networks to increase nonlinearity and help the network learn complex data distributions. Parametric ReLU introduces learnable parameters to increase the flexibility of the ReLU function, allowing it to adaptively adjust the degree of nonlinearity. The first feature, after being normalized by the batch normalization layer and processed by the first parametric ReLU layer, becomes the fusion feature.

[0108] Based on the above feature fusion module, through model training, the linear layer adaptively filters all frequency point information of the root amplitude spectrum, learns more representative high-dimensional features, and fuses the high-dimensional features with Fbank features through subsequent gating mechanisms to avoid the incomplete representation of FBank features, generating more rich and discriminative speech high-dimensional features as fusion features.

[0109] In other embodiments of the present application, as shown in Figure 4 each feature processing module includes two first feature processing submodules, one second feature processing submodule and a first normalization layer connected in turn, wherein:

[0110] Each first feature processing submodule includes a first data shaping layer, a grouped linear layer, a second parametric ReLU layer and a second normalization layer. The first feature processing submodule is specifically used for:

[0111] Using the first data shaping layer, the input feature is data shaped to obtain a second feature;

[0112] Using the grouped linear layer, the second feature is processed to obtain a third feature;

[0113] Using the second parametric ReLU layer and the second normalization layer, the third feature is processed to obtain a first processed feature;

[0114] Specifically, the first first feature processing submodule connected in sequence in the feature processing module takes the output of the previous layer network connected with the feature processing module as the input feature. For example, assuming that the number of feature processing modules in the speech activity detection model is 1, the first first feature processing submodule connected in sequence in the feature processing module takes the fusion feature output by the feature fusion module as the input feature; correspondingly, the second first feature processing submodule connected in sequence in the feature processing module takes the first processing feature output by the first first feature processing submodule as the input feature.

[0115] Wherein, the data shaping can adopt a data shaping method based on sub-band division of full-band information, and the full-band information is reorganized by the data shaping layer in two modes of cross-band overlap and non-overlap to obtain the reorganized high-dimensional features as the output features of the data shaping layer; Specifically, the first data shaping layer is used to perform data shaping on the input features to obtain the second features for subsequent feature processing by the grouping linear layer.

[0116] Wherein, the Grouped Linear Layers is a linear transformation layer in neural networks, which adopts a grouping method to reduce model parameters and improve efficiency when processing input features. Through the Grouped Linear Layers, the second features are further processed to facilitate effective learning of cross-frequency information, and the third features are obtained.

[0117] Layer Normalization normalizes the activation values of each layer of neurons to keep them within a stable range, preventing gradient vanishing or gradient explosion caused by excessively large or small activation values. After the third features are processed by the Parametric ReLU activation function of the first parameterized ReLU layer and the normalization of the second layer normalization layer, the first processing features are obtained.

[0118] The second feature processing submodule includes a second data shaping layer, a grouped gated recurrent unit, and a third layer normalization layer. The second feature processing submodule is specifically used for:

[0119] The second data shaping layer is used to perform data shaping on the first processing features output by the second first feature processing submodule connected in sequence to obtain the fourth features;

[0120] The grouped gated recurrent unit is used to process the fourth features to obtain the fifth features;

[0121] The third layer normalization layer is used to process the fifth features to obtain the second processing features;

[0122] Specifically, the first processing feature output by the second first feature processing submodule is subjected to data shaping by the second data shaping layer to obtain a fourth feature, and then the fourth feature is processed by the grouped gated recurrent unit to obtain a fifth feature, so as to fuse the information between different frequency bands and capture the long-term dependence within the sequence; finally, the fifth feature is subjected to normalization processing by the third normalization layer to obtain the second processing feature.

[0123] The gated recurrent unit (GRU) is a variant of recurrent neural network (RNN) for processing sequence data, aiming to solve the problem of gradient vanishing or gradient explosion encountered by traditional RNN when processing long sequences. The GRU introduces two gating mechanisms, the update gate and the reset gate, to control the flow of information. These two gating mechanisms help the model decide how much past information to retain and how much to update the current state at each time step. The grouped gated recurrent unit (G-GRU) is an improvement of the traditional gated recurrent unit (GRU), aiming to improve the performance and efficiency of the model. The grouped gated recurrent unit inherits the advantages of the GRU, such as effectively controlling the flow of information and capturing long-term dependencies in sequences, while reducing the number of parameters and computational cost through structural optimization.

[0124] The first normalization layer is specifically configured to process a third processing feature obtained by summing the first processing feature and the second processing feature output by the second first feature processing submodule in sequence, to obtain an output feature of the feature processing module.

[0125] Specifically, the first processing feature and the second processing feature output by the second first feature processing submodule in sequence are summed by adding a skip connection to obtain a third processing feature, and then the third processing feature is subjected to normalization processing by the first normalization layer to obtain the output feature of the feature processing module, thereby alleviating the problem of gradient vanishing commonly encountered during training of deep networks.

[0126] Based on the above feature processing module, the full-band information and the sub-band information are combined by data shaping technology to realize structured processing of the fusion features, which can integrate the input features and facilitate subsequent frequency band correlation learning. The grouped linear layer in the feature processing module is used to learn the correlation between sub-bands, and the grouped gated recurrent unit is used to process the information of each frame of speech to construct long-term dependence.

[0127] In other embodiments of the present application, the output module includes a second linear layer, and the output module is specifically configured to: process the output features by using the second linear layer and a Sigmoid activation function to obtain a probability value of the presence of a speech signal for each frame of the speech data.

[0128] Specifically, in the mask output stage, the final output features processed by the N feature processing modules are subjected to the joint action of the second linear layer and the Sigmoid activation function, so that the model can accurately estimate the probability value of the presence of speech for each frame, i.e., the probability value of the presence of a speech signal for each frame of the output speech data.

[0129] The Sigmoid activation function is one of the commonly used nonlinear activation functions in deep learning, which can map the input value to the range of (0, 1), so that the output can be interpreted as a probability. Taking the 0-1 time domain binary label as an example, the frame-level probability value predicted by the speech data is a set of probability values between 0 and 1, which is used as the detection result of the speech data. Other algorithms can be embedded according to actual needs, and a specific threshold can be set as the speech presence reliability, which is not limited in the present application.

[0130] In addition, based on the above feature fusion module, the feature processing module and the output module, the constructed voice activity detection model adopts a lightweight model structure, and the overall model has very low computational overhead and is easy to deploy on a resource-constrained terminal platform such as a mobile device or an embedded system, which can realize accurate voice activity detection and output the probability of the presence of speech to assist other modules in the system.

[0131] As shown in FIG. Figure 5 In another embodiment of the present application, a voice activity detection system is also provided, which includes:

[0132] The acquisition unit 10 is configured to acquire a speech training sample carrying a speech label, wherein the speech label is used to represent whether each frame of the speech training sample contains a speech signal.

[0133] The extraction unit 11 is configured to perform feature extraction on the speech training sample to obtain a root amplitude spectrum feature sample and an Fbank feature sample.

[0134] The construction unit 12 is configured to construct a voice activity detection model, wherein the voice activity detection model is configured to perform feature fusion on the root amplitude spectrum feature and the Fbank feature obtained by feature extraction on the speech data, to obtain fused features, and output a probability value of the presence of a speech signal for each frame of the speech data based on the fused features.

[0135] The training unit 13 is configured to train the voice activity detection model by using the root amplitude spectrum feature sample and the Fbank feature sample to obtain a trained voice activity detection model.

[0136] The detection unit 14 is configured to perform voice activity detection by using the trained voice activity detection model.

[0137] In other embodiments of the present application, the voice activity detection model comprises a feature fusion module, N feature processing modules and an output module connected in sequence, N being a positive integer, wherein:

[0138] The feature fusion module is configured to perform feature fusion on the root amplitude spectrum feature and the Fbank feature obtained by performing feature extraction on the voice data, to obtain fusion features.

[0139] The N feature processing modules are configured to process the fusion features to obtain output features.

[0140] The output module is configured to obtain a probability value of each frame of the voice data containing a voice signal according to the output features.

[0141] In other embodiments of the present application, the feature fusion module comprises a first linear layer, a gated convolution unit, a batch normalization layer and a first parameterized ReLU layer, and the feature fusion module is specifically configured to:

[0142] filter the root amplitude spectrum feature by using the first linear layer to obtain high-dimensional features;

[0143] perform feature concatenation on the high-dimensional features and the Fbank feature to obtain concatenated features;

[0144] process the concatenated features by using the gated convolution unit to obtain first features;

[0145] process the first features by using the batch normalization layer and the first parameterized ReLU layer to obtain the fusion features.

[0146] In other embodiments of the present application, each feature processing module comprises two first feature processing sub-modules, one second feature processing sub-module and a first layer normalization layer connected in sequence, wherein:

[0147] Each first feature processing sub-module comprises a first data shaping layer, a grouped linear layer, a second parameterized ReLU layer and a second layer normalization layer, and the first feature processing sub-module is specifically configured to:

[0148] perform data shaping on the input features by using the first data shaping layer to obtain second features;

[0149] process the second features by using the grouped linear layer to obtain third features;

[0150] process the third features by using the second parameterized ReLU layer and the second layer normalization layer to obtain first processing features;

[0151] The second feature processing submodule comprises a second data shaping layer, a grouping gated recurrent unit and a third layer normalization layer, and the second feature processing submodule is specifically configured to:

[0152] The second data shaping layer is configured to perform data shaping on the first processing feature output by the second first feature processing submodule in sequence to obtain a fourth feature;

[0153] The grouping gated recurrent unit is configured to process the fourth feature to obtain a fifth feature;

[0154] The third layer normalization layer is configured to process the fifth feature to obtain a second processing feature;

[0155] The first layer normalization layer is configured to process a third processing feature obtained by summing the first processing feature output by the second first feature processing submodule in sequence and the second processing feature to obtain an output feature of the feature processing module.

[0156] In other embodiments of the present application, the output module comprises a second linear layer, and the output module is specifically configured to:

[0157] The second linear layer and a Sigmoid activation function are configured to process the output feature to obtain a probability value of the presence of a speech signal for each frame of the speech data.

[0158] In other embodiments of the present application, the obtaining unit 10, when performing obtaining of the speech training sample carrying the speech label, is specifically configured to:

[0159] Obtain a clean speech sample, and use a VAD algorithm in WebRTC to detect whether a speech signal exists in the clean speech sample to obtain a detection result;

[0160] According to the detection result, add a speech label to the clean speech sample;

[0161] Perform data enhancement processing on the clean speech sample to obtain the speech training sample.

[0162] In other embodiments of the present application, the extracting unit 11, when performing feature extraction according to the speech training sample to obtain the root amplitude spectrum feature sample and the Fbank feature sample, is specifically configured to:

[0163] Frame, window and perform fast Fourier transform on the speech training sample to obtain an amplitude spectrum sample;

[0164] Take the square of the amplitude spectrum sample, and then perform MEL filter group processing and logarithm operation to obtain the Fbank feature sample;

[0165] Taking a square root of the amplitude spectrum sample, a square root amplitude spectrum feature sample is obtained.

[0166] As shown in Figure 6 In another embodiment of the present application, a voice activity detection terminal is also provided, comprising a storage medium 20 and a processor 21.

[0167] The storage medium 20 stores computer execution instructions.

[0168] The processor 21 executes the computer execution instructions stored in the storage medium 20 to implement the voice activity detection method as described above.

[0169] The processor 21 can include one or more processing cores. The processor 21 executes various functions and processes data of the present application by running or executing instructions, programs, code sets or instruction sets stored in the storage medium 20, and calling data stored in the storage medium 20. The processor 21 can be at least one of an application specific integrated circuit, a digital signal processor, a digital signal processing device, a programmable logic device, a field programmable gate array, a central processing unit, a controller, a microcontroller and a microprocessor. It can be understood that for different devices, the electronic devices used to implement the functions of the processor 21 described above can also be other.

[0170] The storage medium 20 can be used to store instructions, programs, codes, code sets or instruction sets. The storage medium 20 can include a storage program area and a storage data area, wherein the storage program area can store instructions for implementing an operating system, instructions for at least one function, and instructions for implementing the voice activity detection method described above, etc.; the storage data area can store data involved in the voice activity detection method described above, etc.

[0171] In another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores computer execution instructions, which are executed by the processor to implement the voice activity detection method as described above.

[0172] The computer readable storage medium can be a U disk, a mobile hard disk, a read-only memory, a random access memory or an optical disc, and various media that can store program codes.

[0173] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A voice activity detection method, characterized by, The method comprises the following steps: acquiring a speech training sample carrying a speech label, wherein the speech label is used to represent whether a speech signal exists in each frame of the speech training sample; performing feature extraction on the speech training sample to obtain a root amplitude spectrum feature sample and an Fbank feature sample; constructing a voice activity detection model, wherein the voice activity detection model is used to perform feature fusion on the root amplitude spectrum feature and the Fbank feature obtained by performing feature extraction on speech data, to obtain fused features, and output a probability value of the existence of a speech signal in each frame of the speech data based on the fused features; training the voice activity detection model using the root amplitude spectrum feature sample and the Fbank feature sample to obtain a trained voice activity detection model; performing voice activity detection using the trained voice activity detection model; wherein the feature extraction on the speech training sample to obtain a root amplitude spectrum feature sample comprises the following steps: frame, window and fast Fourier transform are performed on the speech training sample to obtain an amplitude spectrum sample; taking the root of the amplitude spectrum sample to obtain a root amplitude spectrum feature sample.

2. The method as claimed in claim 1, characterized in that, The voice activity detection model comprises a feature fusion module, N feature processing modules and an output module connected in sequence, wherein N is a positive integer, and wherein: the feature fusion module is used to perform feature fusion on the root amplitude spectrum feature and the Fbank feature obtained by performing feature extraction on speech data to obtain fused features; the N feature processing modules are used to process the fused features to obtain output features; the output module is used to obtain a probability value of the existence of a speech signal in each frame of the speech data based on the output features.

3. The method as claimed in claim 2, characterized in that, The feature fusion module comprises a first linear layer, a gated convolution unit, a batch normalization layer and a first parameterized ReLU layer, and is specifically used to: filter the root amplitude spectrum feature using the first linear layer to obtain high-dimensional features; perform feature concatenation on the high-dimensional features and the Fbank features to obtain concatenated features; process the concatenated features using the gated convolution unit to obtain first features; process the first features using the batch normalization layer and the first parameterized ReLU layer to obtain the fused features.

4. The method as claimed in claim 2, wherein, Each feature processing module comprises two first feature processing submodules, one second feature processing submodule and a first layer normalization layer connected in sequence, wherein: each first feature processing submodule comprises a first data shaping layer, a grouped linear layer, a second parameterized ReLU layer and a second layer normalization layer, and is specifically used to: perform data shaping on the input features using the first data shaping layer to obtain second features; process the second features using the grouped linear layer to obtain third features; process the third features using the second parameterized ReLU layer and the second layer normalization layer to obtain first processing features; The second feature processing submodule comprises a second data shaping layer, a grouping gated recurrent unit and a third layer normalization layer, and is specifically configured to: The second data shaping layer is configured to perform data shaping on the first processing feature output by the second first feature processing submodule in sequence to obtain a fourth feature; The grouping gated recurrent unit is configured to process the fourth feature to obtain a fifth feature; The third layer normalization layer is configured to process the fifth feature to obtain a second processing feature; The first layer normalization layer is configured to process a third processing feature obtained by summing the first processing feature output by the second first feature processing submodule in sequence and the second processing feature to obtain an output feature of the feature processing module.

5. The method as claimed in claim 2, wherein, The output module comprises a second linear layer, and is specifically configured to: The second linear layer and a Sigmoid activation function are configured to process the output feature to obtain a probability value of the presence of a speech signal for each frame of speech data.

6. The method as claimed in claim 1, wherein, The speech training sample carrying a speech label is obtained, including: A clean speech sample is obtained, and a VAD algorithm in WebRTC is used to detect whether a speech signal exists in the clean speech sample to obtain a detection result; According to the detection result, the clean speech sample is added with the speech label; The clean speech sample is processed by data enhancement to obtain the speech training sample.

7. The method as claimed in claim 1, wherein, The Fbank feature sample is obtained by performing feature extraction on the speech training sample, including: The speech training sample is framed, windowed and fast Fourier transformed to obtain an amplitude spectrum sample; The amplitude spectrum sample is squared, and then passed through a mel filter bank and logarithm to obtain the Fbank feature sample.

8. A voice activity detection system characterized by, It comprises: An acquisition unit is configured to acquire a speech training sample carrying a speech label, wherein the speech label is used to represent whether a speech signal exists in each frame of the speech training sample; An extraction unit is configured to perform feature extraction on the speech training sample to obtain a root amplitude spectrum feature sample and an Fbank feature sample; A construction unit is configured to construct a voice activity detection model, wherein the voice activity detection model is configured to perform feature fusion on a root amplitude spectrum feature and an Fbank feature obtained by performing feature extraction on speech data to obtain fused features, and output a probability value of the presence of a speech signal for each frame of the speech data based on the fused features; A training unit is configured to train the voice activity detection model using the root amplitude spectrum feature sample and the Fbank feature sample to obtain a trained voice activity detection model; A detection unit is configured to perform voice activity detection using the trained voice activity detection model; When performing the feature extraction on the speech training sample to obtain the root amplitude spectrum feature sample, the extraction unit is specifically configured to: The speech training sample is framed, windowed and fast Fourier transformed to obtain an amplitude spectrum sample; Taking square roots of the amplitude spectrum samples to obtain square root amplitude spectrum feature samples.

9. A voice activity detection terminal, characterized by Comprise: A storage medium and a processor; Computer-executable instructions are stored in the storage medium; The processor executes the computer-executable instructions stored in the storage medium to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the method of any one of claims 1 to 7.

Images