Voice activity detection method and system, terminal and storage medium

Abstract

The invention provides a voice activity detection method and system, a terminal and a storage medium, and relates to the technical field of voice data processing, in particular to a voice activity detection method, which comprises the following steps: acquiring a voice training sample carrying a voice tag; performing feature extraction according to the voice training sample to obtain a root sign amplitude spectrum feature sample and an Fbank feature sample; a voice activity detection model is constructed, and the voice activity detection model is used for carrying out feature fusion on root number amplitude spectrum features and Fbank features obtained by carrying out feature extraction on the voice data to obtain fusion features, and outputting a probability value of a voice signal existing in each frame of the voice data based on the fusion features; training a voice activity detection model by using the root sign amplitude spectrum feature sample and the Fbank feature sample to obtain a trained voice activity detection model; and performing voice activity detection by using the trained voice activity detection model. According to the invention, the accuracy of voice activity detection can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of voice data processing, and in particular to a voice activity detection method, system, terminal and storage medium. Background Art

[0002] Voice Activity Detection (VAD) is an audio signal processing technology that detects whether speech is present. It is usually used in scenarios such as speaker location, voice communication, and speech recognition. It is also used to clean data sets and serve speech enhancement models and large speech models. Through voice activity detection technology, the probability of speech presence can be output at the speech frame level data, so that other audio processing technologies can be applied more effectively when speech occurs. This not only helps save computing resources, but also optimizes the performance and efficiency of the entire system.

[0003] At present, commonly used voiceprint features include MFCC (Mel Frequency Cepstral Coefficient), Fbank (FilterBank), etc. In the voice activity detection method based on deep learning, the current commonly used model construction method is mostly based on a single type of acoustic feature for model construction, and its model detection accuracy needs to be improved.

[0004] Therefore, how to improve the accuracy of voice activity detection is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] To solve the above technical problems, the present application provides a voice activity detection method, which can improve the accuracy of voice activity detection. The present application also provides a voice activity detection system, a terminal and a storage medium, which have the same technical effects.

[0006] The first objective of the present application is to provide a voice activity detection method.

[0007] The above-mentioned application objective 1 of the present application is achieved through the following technical solutions: A voice activity detection method, comprising: Acquire a speech training sample carrying a speech label, wherein the speech label is used to indicate whether a speech signal exists in each frame of the speech training sample; Perform feature extraction based on the speech training sample to obtain a square root amplitude spectrum feature sample and an Fbank feature sample; Constructing a voice activity detection model, wherein the voice activity detection model is used to perform feature fusion on the root amplitude spectrum features and Fbank features obtained by performing feature extraction on the voice data to obtain fused features, and output a probability value of the presence of a voice signal in each frame of the voice data based on the fused features; Using the square root amplitude spectrum feature samples and the Fbank feature samples, the voice activity detection model is trained to obtain a trained voice activity detection model; The trained voice activity detection model is used to perform voice activity detection.

[0008] Preferably, in the voice activity detection method, the voice activity detection model comprises a feature fusion module, N feature processing modules and an output module connected in sequence, where N is a positive integer, wherein: The feature fusion module is used to perform feature fusion on the root magnitude spectrum feature and Fbank feature obtained by feature extraction of the speech data to obtain a fusion feature; N feature processing modules are used to process the fused features to obtain output features; The output module is used to obtain the probability value of the existence of a speech signal in each frame of the speech data according to the output feature.

[0009] Preferably, in the voice activity detection method, the feature fusion module includes a first linear layer, a gated convolution unit, a batch normalization layer and a first parameterized ReLU layer, and the feature fusion module is specifically used for: Using the first linear layer, filtering the square root amplitude spectrum features to obtain high-dimensional features; Performing feature splicing on the high-dimensional features and the Fbank features to obtain spliced ​​features; Using the gated convolution unit, processing the splicing feature to obtain a first feature; The first feature is processed by using the batch normalization layer and the first parameterized ReLU layer to obtain the fused feature.

[0010] Preferably, in the voice activity detection method, each of the feature processing modules comprises two first feature processing submodules, one second feature processing submodule and a first normalization layer connected in sequence, wherein: Each of the first feature processing submodules includes a first data shaping layer, a grouped linear layer, a second parameterized ReLU layer and a second normalization layer. The first feature processing submodule is specifically used to: Using the first data shaping layer, reshaping the input feature to obtain a second feature; Using the grouped linear layer, processing the second feature to obtain a third feature; Using the second parameterized ReLU layer and the second normalization layer, the third feature is processed to obtain a first processed feature; The second feature processing submodule includes a second data shaping layer, a group gated recurrent unit and a third normalization layer, and the second feature processing submodule is specifically used to: Using the second data shaping layer, data shaping is performed on the first processing feature output by the second first feature processing submodule connected in sequence, so as to obtain a fourth feature; Using the group gated recurrent unit, processing the fourth feature to obtain a fifth feature; Using the third normalization layer, processing the fifth feature to obtain a second processed feature; The first normalization layer is specifically used to process a third processing feature obtained by summing the first processing feature output by the second first feature processing submodule connected in sequence and the second processing feature to obtain the output feature of the feature processing module.

[0011] Preferably, in the voice activity detection method, the output module includes a second linear layer, and the output module is specifically used for: The output feature is processed using the second linear layer and the Sigmoid activation function to obtain a probability value of the presence of a speech signal in each frame of the speech data.

[0012] Preferably, in the voice activity detection method, the step of obtaining a voice training sample carrying a voice tag includes: Obtain a clean voice sample, and use the VAD algorithm in WebRTC to detect whether there is a voice signal in the clean voice sample to obtain a detection result; According to the detection result, adding the voice tag to the clean voice sample; The clean speech samples are subjected to data enhancement processing to obtain the speech training samples.

[0013] Preferably, in the voice activity detection method, the step of extracting features based on the voice training samples to obtain root magnitude spectrum feature samples and Fbank feature samples includes: The speech training samples are framed, windowed and fast Fourier transformed, and modulo-sampled to obtain amplitude spectrum samples; The amplitude spectrum samples are squared, passed through a Mel filter bank and logarithm is taken to obtain Fbank feature samples; The square root of the amplitude spectrum sample is taken to obtain a square root amplitude spectrum feature sample.

[0014] The second objective of the present application is to provide a voice activity detection system.

[0015] The second application objective of the present application is achieved through the following technical solutions: A voice activity detection system, comprising: An acquisition unit, configured to acquire a speech training sample carrying a speech label, wherein the speech label is used to indicate whether a speech signal exists in each frame of the speech training sample; An extraction unit, used for performing feature extraction based on the speech training sample to obtain a root magnitude spectrum feature sample and an Fbank feature sample; A construction unit is used to construct a voice activity detection model, wherein the voice activity detection model is used to perform feature fusion on the root amplitude spectrum features and Fbank features obtained by performing feature extraction on the voice data to obtain fused features, and output a probability value of the presence of a voice signal in each frame of the voice data based on the fused features; A training unit, configured to train the voice activity detection model using the square root magnitude spectrum feature samples and the Fbank feature samples to obtain a trained voice activity detection model; The detection unit is used to perform voice activity detection using the trained voice activity detection model.

[0016] The third objective of the present application is to provide a voice activity detection terminal.

[0017] The third application objective of this application is achieved through the following technical solutions: A voice activity detection terminal, comprising: a storage medium and a processor; The storage medium stores computer-executable instructions; The processor executes the computer-executable instructions stored in the storage medium to implement the voice activity detection method as described above.

[0018] The fourth object of this application is to provide a computer-readable storage medium.

[0019] The fourth application objective of this application is achieved through the following technical solutions: A computer-readable storage medium stores computer-executable instructions, which are used to implement the above-mentioned voice activity detection method when executed by a processor.

[0020] The above technical solution obtains speech training samples with speech labels; performs feature extraction based on the speech training samples to obtain square root amplitude spectrum feature samples and Fbank feature samples; wherein, the high-frequency energy of speech is generally smaller than the low-frequency energy, that is, the speech energy is mainly concentrated in the low frequency, and the difference between the high and low frequency amplitudes can be reduced by extracting the square root amplitude spectrum features; further, by constructing a speech activity detection model, the speech activity detection model is trained using the square root amplitude spectrum feature samples and the Fbank feature samples to obtain a trained speech activity detection model; voice activity detection is performed using the trained voice activity detection model; wherein the speech activity detection model is used to perform feature fusion on the square root amplitude spectrum features and Fbank features obtained by feature extraction of speech data to obtain fused features, and based on the fused features, outputs the probability value of the presence of a speech signal in each frame of the speech data. Compared with the method of constructing a model based on a single type of acoustic feature, the above technical solution can improve the effectiveness of feature extraction and further improve the accuracy of voice activity detection by fusing square root amplitude spectrum features and Fbank features. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flow chart of a voice activity detection method in an embodiment of the present application; Figure 2 This is a schematic diagram of the structure of the voice activity detection model in the embodiment of the present application; Figure 3 This is a structural diagram of a feature fusion module in an embodiment of the present application; Figure 4 This is a schematic diagram of the structure of the feature processing module in the embodiment of the present application; Figure 5 This is a structural diagram of a voice activity detection system in an embodiment of the present application; Figure 6 It is a structural diagram of a voice activity detection terminal in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] In the embodiments provided in the present application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described below are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0025] It should be understood that the use of "system", "device", "unit" and / or "module" in this application is only a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0026] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "multiple" and "several" mean two or more, unless otherwise clearly and specifically defined.

[0027] If a flow chart is used in the present application, the flow chart is used to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, each step can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or a certain step or several steps of operations can be removed from these processes.

[0028] It should also be noted that, in this article, terms such as "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that an article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such articles or devices. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the article or device including the above elements.

[0029] The embodiments of the present application are written in a progressive manner.

[0030] like Figure 1 As shown, the embodiment of the present application provides a voice activity detection method, including: S101. Obtain a speech training sample with a speech tag; In S101, specifically, the speech training sample can be a speech data set that has been labeled with speech labels, wherein the speech labels are used to characterize whether there is a speech signal in each frame of the speech training sample. For example, the speech label can adopt a 0-1 time domain binary label. For a single frame of data in the speech training sample, if the time domain binary label is 1, it represents that there is a speech signal in the single frame of data, and if it is 0, it represents that there is no speech signal in the single frame of data. The present application is not limited to this.

[0031] In some embodiments, one implementation of this step specifically includes: S1011 obtains a clean voice sample, and uses the VAD algorithm in WebRTC to detect whether there is a voice signal in the clean voice sample to obtain the detection result; In S1011, specifically, the clean voice sample can be the voice data recorded in the anechoic room after cleaning; the VAD algorithm in WebRTC models voice and noise based on the Gaussian Mixed Model (GMM). GMM is an unsupervised learning method that can effectively distinguish voice and noise by probabilistically modeling the input data. In WebRTC, the VAD algorithm divides the input spectrum into six sub-bands and calculates the energy of each sub-band. Then, the probability density function of GMM is used to model the energies of these sub-bands to obtain a log-likelihood ratio function. The detection of voice activity is achieved through global and local judgment of the log-likelihood ratio. Using the VAD algorithm in WebRTC, voice activity detection is performed on the clean voice sample to obtain a detection result.

[0032] S1012. Add voice tags to clean voice samples according to the detection results; In S1012, specifically, based on the detection result, a voice tag is added to the clean speech sample to characterize whether there is a voice signal in each frame of data. For example, the voice tag can use a 0-1 time domain binary tag. If the detection result of a single frame of data of the clean speech sample is that there is a voice signal, then the single frame of data of the clean speech sample is marked with a voice tag with a value of 1. If the detection result of a single frame of data of the clean speech sample is that there is no voice signal, then the single frame of data of the clean speech sample is marked with a voice tag with a value of 0. The present application is not limited to this.

[0033] S1013. Perform data enhancement processing on the clean speech samples to obtain speech training samples.

[0034] In S1013, specifically, existing data enhancement technology can be used to perform data enhancement processing on clean speech samples to expand the number of samples; in some embodiments, data enhancement processing can include one or more processing methods such as adding reverberation, adding noise, adjusting frequency response, and destroying frequency band information, so as to simulate actual complex application scenarios and help improve the robustness of the model in different environments.

[0035] S102. According to the speech training sample, feature extraction is performed to obtain the square root amplitude spectrum feature sample and the Fbank feature sample; In S102, specifically, in order to capture different characteristics of speech signals, feature extraction is performed on speech training samples to obtain square root amplitude spectrum feature samples and Fbank feature samples. Among them, the response of the human ear to the sound spectrum is nonlinear, and Fbank is a front-end processing algorithm that processes audio in a manner similar to the human ear, which can improve the performance of speech recognition; the high-frequency energy of speech is generally less than the low-frequency energy, that is, the speech energy is mainly concentrated in the low frequency, and by extracting the square root amplitude spectrum features, the difference between the high and low frequency amplitudes can be reduced.

[0036] In some embodiments, one implementation method of this step specifically includes: framing, windowing and fast Fourier transforming the speech training samples, taking the modulus to obtain the amplitude spectrum samples; squaring the amplitude spectrum samples, passing through the Mel filter group and taking the logarithm to obtain the Fbank feature samples; taking the square root of the amplitude spectrum samples to obtain the square root amplitude spectrum feature samples.

[0037] Specifically, the speech training samples are framed, windowed and fast Fourier transformed to obtain frequency domain data, the frequency domain data is modulo-operated to obtain amplitude spectrum samples, and then the amplitude spectrum samples are squared to obtain the power spectrum, which is then input into a Mel filter bank for filtering, the logarithm of the filter output is taken to obtain Fbank feature samples, and the amplitude spectrum samples are squared to obtain square root amplitude spectrum feature samples.

[0038] S103. Construct a voice activity detection model; In S103, specifically, a voice activity detection model based on deep learning can be constructed. The voice activity detection model is used to perform feature fusion on the square root amplitude spectrum features and Fbank features obtained by performing feature extraction on the voice data to obtain fused features, and based on the fused features, output the probability value of the presence of a voice signal in each frame of the voice data; wherein, for the specific implementation details of performing feature extraction on the voice data to obtain the square root amplitude spectrum features and the Fbank features, reference can be made to the above S102; compared with the method of constructing a model based on a single type of acoustic features, the above voice activity detection model, through the fusion of the square root amplitude spectrum features and the Fbank features, can improve the effectiveness of the extracted features and further improve the accuracy of voice activity detection.

[0039] S104. Using the square root amplitude spectrum feature samples and the Fbank feature samples, the voice activity detection model is trained to obtain a trained voice activity detection model; In S104, specifically, the existing model training method can be adopted to use the square root amplitude spectrum feature samples and Fbank feature samples to iteratively train the voice activity detection model. When the set termination iteration condition is met, the trained voice activity detection model is obtained; specifically, in the model training process, the AdamW optimizer and SmoothL1Loss loss function can be used to train the model, the decay_weight is set to 5e-6, and the learning rate is set to 0.0008. The present application is not limited to this.

[0040] S105. Perform voice activity detection using the trained voice activity detection model.

[0041] In S105, specifically, the trained voice activity detection model can be implemented on the terminal platform, and voice activity detection can be performed on the voice data to be detected in the platform to obtain a detection result.

[0042] In some embodiments, one implementation method of this step specifically includes: obtaining speech data to be detected; performing feature extraction based on the speech data to be detected to obtain square root amplitude spectrum feature data and Fbank feature data; inputting the square root amplitude spectrum feature data and the Fbank feature data into a trained speech activity detection model to obtain the probability value of the existence of a speech signal in each frame of the speech data to be detected.

[0043] Specifically, the specific implementation details of feature extraction can refer to the above S102; after the square root amplitude spectrum feature data and the Fbank feature data are input into the trained voice activity detection model, the voice activity detection model performs feature fusion according to the square root amplitude spectrum feature data and the Fbank feature data to obtain fused feature data, and based on the fused feature data, outputs the probability value of the presence of a voice signal in each frame of the voice data to be detected.

[0044] In the above embodiment, speech training samples carrying speech tags are obtained; feature extraction is performed based on the speech training samples to obtain square root amplitude spectrum feature samples and Fbank feature samples; wherein, the high-frequency energy of speech is generally smaller than the low-frequency energy, that is, the speech energy is mainly concentrated in the low frequency, and the difference between the high and low frequency amplitudes can be reduced by extracting the square root amplitude spectrum features; further, by constructing a speech activity detection model, the speech activity detection model is trained using the square root amplitude spectrum feature samples and the Fbank feature samples to obtain a trained speech activity detection model; speech activity detection is performed using the trained speech activity detection model; wherein the speech activity detection model is used to perform feature fusion on the square root amplitude spectrum features and the Fbank features obtained by feature extraction of speech data to obtain fused features, and based on the fused features, output the probability value of the presence of a speech signal in each frame of the speech data. Compared with the model construction method based on a single type of acoustic features, the above embodiment can improve the effectiveness of feature extraction and further improve the accuracy of speech activity detection by fusing the square root amplitude spectrum features and the Fbank features.

[0045] In other embodiments of the present application, Figure 2 As shown, the voice activity detection model includes a feature fusion module, N feature processing modules and an output module connected in sequence, wherein: The feature fusion module is used to perform feature fusion on the square root amplitude spectrum features and Fbank features obtained by feature extraction of speech data to obtain fused features; N feature processing modules are used to process the fused features to obtain output features; the output module is used to obtain the probability value of the existence of speech signals in each frame of speech data based on the output features.

[0046] Specifically, N is a positive integer, and the number of feature processing modules can be one or more. The number of feature processing modules can be adjusted in advance according to the amount of calculation, that is, the size of the voice activity detection model can be adjusted; when the number of feature processing modules is one, the feature processing module is specifically used to process the fused features to obtain output features; when the number of feature processing modules is multiple, the first feature processing module connected in sequence is used to process the fused features to obtain the output features of the feature processing module, and serve as the input of the second feature processing module connected in sequence, and so on, the output of the last feature processing module connected in sequence is used as the final output feature and is passed to the output module.

[0047] In other embodiments of the present application, in order to overcome the problem of incomplete FBank feature representation caused by factors such as noise in the actual collected data, the root amplitude spectrum features and Fbank features obtained by feature extraction of speech data are better integrated, such as Figure 3 As shown, the feature fusion module includes a first linear layer, a gated convolution unit, a batch normalization layer and a first parameterized ReLU layer. The feature fusion module is specifically used for: Using the first linear layer, the square root amplitude spectrum features are filtered to obtain high-dimensional features; Perform feature splicing on high-dimensional features and Fbank features to obtain spliced ​​features; Using the gated convolution unit, the concatenated features are processed to obtain the first feature; The first feature is processed using a batch normalization layer and a first parameterized ReLU layer to obtain a fused feature.

[0048] Specifically, the linear layer is a basic layer in a neural network, also known as a fully connected layer or a dense layer. The role of the linear layer in a neural network is to perform a linear transformation on the input data. First, using the first linear layer, all frequency point information of the square root amplitude spectrum is integrated to perform adaptive filtering operations, learn more representative high-dimensional features, and obtain high-dimensional features; then, the obtained high-dimensional features are concatenated with the Fbank features obtained by feature extraction to obtain concatenated features.

[0049] Among them, the gated convolutional unit (GCU) is an improved convolutional neural network structure. The gated convolution is an advanced convolution operation that introduces a gating mechanism to enhance the model's ability to capture complex dependencies in the data. After obtaining the spliced ​​features, the spliced ​​features are input into the gated convolution unit for convolution operation to obtain the first feature, so as to achieve the fusion of high-dimensional features and Fbank features and avoid the problem of incomplete representation of Fbank features.

[0050] Batch Normalization is a special layer in neural networks. The goal of batch normalization is to normalize the features (so that the output of each layer of the network is activated) to obtain a zero-mean state with a standard deviation of 1. It not only speeds up the convergence of the model, but more importantly, it alleviates the problem of "vanishing gradients" in deep networks to a certain extent, making it easier and more stable to train deep network models. Parameterized ReLU, also known as the Parametric ReLU activation function, is an improved ReLU (Rectified Linear Unit) activation function. It is used in neural networks to increase nonlinear characteristics and help the network learn complex data distributions. Parameterized ReLU adds flexibility to the ReLU function by introducing learnable parameters, enabling it to adaptively adjust the degree of nonlinearity. The first feature is processed by the normalization of the batch normalization layer and the Parametric ReLU activation function of the first parameterized ReLU layer to obtain a fused feature.

[0051] Based on the above feature fusion module, through model training, the linear layer integrates all frequency point information of the square root amplitude spectrum to perform adaptive filtering operations, learn more representative high-dimensional features, and fuse the high-dimensional features with the Fbank features through the subsequent gating mechanism to avoid the problem of incomplete representation of FBank features and generate richer and more discriminative high-dimensional speech features as fusion features.

[0052] In other embodiments of the present application, Figure 4 As shown, each feature processing module includes two first feature processing submodules, one second feature processing submodule and a first normalization layer connected in sequence, wherein: Each first feature processing submodule includes a first data shaping layer, a grouped linear layer, a second parameterized ReLU layer and a second normalization layer. The first feature processing submodule is specifically used for: Using the first data shaping layer, the input feature is data shaped to obtain the second feature; The second feature is processed using a grouped linear layer to obtain the third feature; The third feature is processed by using a second parameterized ReLU layer and a second normalization layer to obtain a first processed feature; Specifically, the first first feature processing submodule connected in sequence in the feature processing module uses the output of the previous layer network connected to the feature processing module as the input feature. For example, assuming that the number of feature processing modules in the voice activity detection model is 1, the first first feature processing submodule connected in sequence in the feature processing module uses the fusion feature output by the feature fusion module as the input feature; correspondingly, the second first feature processing submodule connected in sequence in the feature processing module uses the first processed feature output by the first first feature processing submodule as the input feature; Among them, data shaping can adopt a data shaping method based on dividing the full-band information of data into sub-bands, and use the data shaping layer to reorganize the full-band information in two modes: cross-band overlapping and non-overlapping, to obtain the reorganized high-dimensional features as the output features of the data shaping layer; specifically, use the first data shaping layer to perform data shaping on the input features to obtain the second features for subsequent feature processing through the grouped linear layer.

[0053] Among them, the Grouped Linear Layers is a linear transformation layer in a neural network. It uses a grouping method when processing input features to reduce model parameters and improve efficiency. Through the Grouped Linear Layer, the second feature is further processed to promote the effective learning of cross-band information to obtain the third feature.

[0054] Layer Normalization normalizes the activation values ​​of each layer of neurons so that the activation values ​​remain within a stable range, preventing the gradient from vanishing or exploding due to activation values ​​that are too large or too small. The third feature is processed by the first parametric ReLU activation function of the first parametric ReLU layer and the normalization of the second normalization layer to obtain the first processed feature.

[0055] The second feature processing submodule includes a second data shaping layer, a group gated recurrent unit and a third normalization layer, and the second feature processing submodule is specifically used for: Using the second data shaping layer, data shaping is performed on the first processing feature output by the second first feature processing submodule connected in sequence to obtain a fourth feature; The fourth feature is processed by using a group gated recurrent unit to obtain a fifth feature; The fifth feature is processed using the third normalization layer to obtain the second processed feature; Specifically, the second data shaping layer is used to perform data shaping on the first processing feature output by the second first feature processing submodule connected in sequence to obtain the fourth feature. The fourth feature is then processed by a group gated recurrent unit to obtain the fifth feature, so as to fuse and learn the information between different frequency bands and capture the long-term dependency within the sequence. Finally, the third normalization layer is used to normalize the fifth feature to obtain the second processing feature.

[0056] Among them, the Gated Recurrent Unit (GRU) is a variant of the Recurrent Neural Network (RNN) for processing sequence data. It aims to solve the gradient vanishing or gradient exploding problems encountered by traditional RNNs when processing long sequences. GRU controls the flow of information by introducing two gating mechanisms, the Update Gate and the Reset Gate. These two gating mechanisms help the model decide how much past information to retain and how much current state to update in each time step. The Grouped Gated Recurrent Unit (G-GRU) is an improvement on the traditional Gated Recurrent Unit (GRU) designed to improve the performance and efficiency of the model. The Grouped Gated Recurrent Unit inherits the advantages of the GRU, such as effectively controlling the flow of information and capturing long-term dependencies in the sequence, while reducing parameters and computational costs through structural optimization.

[0057] The first normalization layer is specifically used to process a third processing feature obtained by summing the first processing feature and the second processing feature output by the second first feature processing submodule connected in sequence, so as to obtain the output feature of the feature processing module.

[0058] Specifically, by adding a jump connection, the first processing feature and the second processing feature output by the second first feature processing submodule connected in sequence are summed to obtain a third processing feature, and then the third processing feature is normalized using the first normalization layer to obtain the output feature of the feature processing module, thereby alleviating the common gradient vanishing problem in the deep network training process.

[0059] Based on the above feature processing module, the full-band information is combined with the sub-band information through data shaping technology, which realizes the structured processing of the fusion feature, can integrate the input features, and facilitate the subsequent band correlation learning. In this feature processing module, the grouped linear layer is used to learn the correlation between sub-bands, and the grouped gated recurrent unit is used to process the information of each frame of speech and build long-term dependencies.

[0060] In other embodiments of the present application, the output module includes a second linear layer, and the output module is specifically used to: use the second linear layer and the Sigmoid activation function to process the output features to obtain the probability value of the presence of a speech signal in each frame of the speech data.

[0061] Specifically, in the mask output stage, the final output features obtained by processing N feature processing modules are processed through the second linear layer and the Sigmoid activation function. The model can accurately estimate the probability value of the existence of speech in each frame, that is, the probability value of the existence of speech signals in each frame of the output speech data.

[0062] Among them, the Sigmoid activation function is one of the commonly used nonlinear activation functions in deep learning. It can map the input value to the range of (0,1) so that its output can be interpreted as a probability. Taking the voice label as an example of a 0-1 time domain binary label, the frame-level probability value predicted by the voice data is a set of probability values ​​between 0 and 1, which is used as the detection result of the voice data. It can be embedded in other algorithms according to actual needs, and a specific threshold can be set as the credibility of the voice existence, but this application is not limited to this.

[0063] In addition, the voice activity detection model constructed based on the above-mentioned feature fusion module, feature processing module and output module adopts a lightweight model structure. The overall model has extremely low computational overhead and is easy to deploy on resource-constrained terminal platforms, such as mobile devices or embedded systems. It can realize accurate voice activity detection and output the probability of voice presence, providing assistance to other modules in the system.

[0064] like Figure 5 As shown, in another embodiment of the present application, a voice activity detection system is also provided, including: An acquisition unit 10 is used to acquire a speech training sample carrying a speech label, wherein the speech label is used to indicate whether there is a speech signal in each frame of the speech training sample; An extraction unit 11 is used to perform feature extraction based on the speech training sample to obtain a root magnitude spectrum feature sample and an Fbank feature sample; A construction unit 12 is used to construct a voice activity detection model, wherein the voice activity detection model is used to perform feature fusion on the square root amplitude spectrum features and Fbank features obtained by performing feature extraction on the voice data to obtain fused features, and output a probability value of the presence of a voice signal in each frame of the voice data based on the fused features; A training unit 13 is used to train the voice activity detection model using the square root amplitude spectrum feature samples and the Fbank feature samples to obtain a trained voice activity detection model; The detection unit 14 is used to perform voice activity detection using the trained voice activity detection model.

[0065] In other embodiments of the present application, the voice activity detection model includes a feature fusion module, N feature processing modules and an output module connected in sequence, where N is a positive integer, wherein: A feature fusion module is used to fuse the root magnitude spectrum features and Fbank features obtained by extracting features from speech data to obtain fused features; N feature processing modules, used to process the fused features to obtain output features; The output module is used to obtain the probability value of the existence of a speech signal in each frame of the speech data according to the output characteristics.

[0066] In other embodiments of the present application, the feature fusion module includes a first linear layer, a gated convolution unit, a batch normalization layer, and a first parameterized ReLU layer, and the feature fusion module is specifically used for: Using the first linear layer, the square root amplitude spectrum features are filtered to obtain high-dimensional features; Perform feature splicing on high-dimensional features and Fbank features to obtain spliced ​​features; Using the gated convolution unit, the concatenated features are processed to obtain the first feature; The first feature is processed using a batch normalization layer and a first parameterized ReLU layer to obtain a fused feature.

[0067] In other embodiments of the present application, each feature processing module includes two first feature processing submodules, one second feature processing submodule and a first normalization layer connected in sequence, wherein: Each first feature processing submodule includes a first data shaping layer, a grouped linear layer, a second parameterized ReLU layer and a second normalization layer. The first feature processing submodule is specifically used for: Using the first data shaping layer, the input feature is data shaped to obtain the second feature; The second feature is processed using a grouped linear layer to obtain the third feature; The third feature is processed by using a second parameterized ReLU layer and a second normalization layer to obtain a first processed feature; The second feature processing submodule includes a second data shaping layer, a group gated recurrent unit and a third normalization layer, and the second feature processing submodule is specifically used for: Using the second data shaping layer, data shaping is performed on the first processing feature output by the second first feature processing submodule connected in sequence to obtain a fourth feature; The fourth feature is processed by using a group gated recurrent unit to obtain a fifth feature; Using the third normalization layer, the fifth feature is processed to obtain the second processed feature; The first normalization layer is specifically used to process a third processing feature obtained by summing the first processing feature and the second processing feature output by the second first feature processing submodule connected in sequence, so as to obtain the output feature of the feature processing module.

[0068] In other embodiments of the present application, the output module includes a second linear layer, the output module, specifically configured to: The output features are processed using the second linear layer and the Sigmoid activation function to obtain the probability value of the presence of a speech signal in each frame of the speech data.

[0069] In other embodiments of the present application, when the acquiring unit 10 acquires the speech training sample carrying the speech tag, it is specifically used to: Get clean voice samples, and use the VAD algorithm in WebRTC to detect whether there is a voice signal in the clean voice samples to obtain the detection results; According to the detection results, add voice tags to the clean voice samples; Perform data enhancement processing on clean speech samples to obtain speech training samples.

[0070] In other embodiments of the present application, when the extraction unit 11 performs feature extraction according to the speech training sample to obtain the square root amplitude spectrum feature sample and the Fbank feature sample, it is specifically used to: The speech training samples are framed, windowed, and fast Fourier transformed, and the amplitude spectrum samples are obtained by taking the modulus; The magnitude spectrum samples are squared, passed through the Mel filter bank and logarithm is taken to obtain the Fbank feature samples; Take the square root of the amplitude spectrum samples to obtain the square root amplitude spectrum feature samples.

[0071] like Figure 6 As shown, in another embodiment of the present application, a voice activity detection terminal is also provided, including: a storage medium 20 and a processor 21; The storage medium 20 stores computer-executable instructions; The processor 21 executes the computer-executable instructions stored in the storage medium 20 to implement the voice activity detection method as described above.

[0072] Among them, the processor 21 may include one or more processing cores. The processor 21 executes various functions and processes data of the present application by running or executing instructions, programs, code sets or instruction sets stored in the storage medium 20, calling data stored in the storage medium 20. The processor 21 can be at least one of a special application integrated circuit, a digital signal processor, a digital signal processing device, a programmable logic device, a field programmable gate array, a central processing unit, a controller, a microcontroller and a microprocessor. It can be understood that for different devices, the electronic device used to implement the functions of the above-mentioned processor 21 can also be other.

[0073] The storage medium 20 may be used to store instructions, programs, codes, code sets or instruction sets. The storage medium 20 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, and instructions for implementing the above-mentioned voice activity detection method, etc.; the data storage area may store data involved in the above-mentioned voice activity detection method, etc.

[0074] In another embodiment of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the voice activity detection method as described above.

[0075] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, or a CD, etc., which may store program codes.

[0076] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A voice activity detection method, characterized in that: include: Acquire a speech training sample carrying a speech label, wherein the speech label is used to indicate whether a speech signal exists in each frame of the speech training sample; Perform feature extraction based on the speech training sample to obtain a square root amplitude spectrum feature sample and an Fbank feature sample; Constructing a voice activity detection model, wherein the voice activity detection model is used to perform feature fusion on the root amplitude spectrum features and Fbank features obtained by performing feature extraction on the voice data to obtain fused features, and output a probability value of the presence of a voice signal in each frame of the voice data based on the fused features; Using the square root amplitude spectrum feature samples and the Fbank feature samples, the voice activity detection model is trained to obtain a trained voice activity detection model; The trained voice activity detection model is used to perform voice activity detection.

2. The method according to claim 1, characterized in that The voice activity detection model comprises a feature fusion module, N feature processing modules and an output module connected in sequence, where N is a positive integer, wherein: The feature fusion module is used to perform feature fusion on the root magnitude spectrum feature and Fbank feature obtained by feature extraction of the speech data to obtain a fusion feature; N feature processing modules are used to process the fused features to obtain output features; The output module is used to obtain the probability value of the existence of a speech signal in each frame of the speech data according to the output feature.

3. The method as claimed in claim 2, characterized in that The feature fusion module includes a first linear layer, a gated convolution unit, a batch normalization layer and a first parameterized ReLU layer, and the feature fusion module is specifically used for: Using the first linear layer, filtering the square root amplitude spectrum features to obtain high-dimensional features; Performing feature splicing on the high-dimensional features and the Fbank features to obtain spliced ​​features; Using the gated convolution unit, processing the splicing feature to obtain a first feature; The first feature is processed by using the batch normalization layer and the first parameterized ReLU layer to obtain the fused feature.

4. The method as claimed in claim 2, characterized in that Each of the feature processing modules includes two first feature processing submodules, one second feature processing submodule and a first normalization layer connected in sequence, wherein: Each of the first feature processing submodules includes a first data shaping layer, a grouped linear layer, a second parameterized ReLU layer and a second normalization layer. The first feature processing submodule is specifically used to: Using the first data shaping layer, the input feature is reshaped to obtain a second feature; Using the grouped linear layer, processing the second feature to obtain a third feature; Using the second parameterized ReLU layer and the second normalization layer, the third feature is processed to obtain a first processed feature; The second feature processing submodule includes a second data shaping layer, a group gated recurrent unit and a third normalization layer, and the second feature processing submodule is specifically used to: Using the second data shaping layer, data shaping is performed on the first processing feature output by the second first feature processing submodule connected in sequence, so as to obtain a fourth feature; Using the group gated recurrent unit, processing the fourth feature to obtain a fifth feature; Using the third normalization layer, processing the fifth feature to obtain a second processed feature; The first normalization layer is specifically used to process a third processing feature obtained by summing the first processing feature output by the second first feature processing submodule connected in sequence and the second processing feature to obtain the output feature of the feature processing module.

5. The method as claimed in claim 2, characterized in that The output module includes a second linear layer, and the output module is specifically used for: The output feature is processed using the second linear layer and the Sigmoid activation function to obtain a probability value of the presence of a speech signal in each frame of the speech data.

6. The method according to claim 1, characterized in that The step of obtaining a speech training sample having a speech tag includes: Obtain a clean voice sample, and use the VAD algorithm in WebRTC to detect whether there is a voice signal in the clean voice sample to obtain a detection result; According to the detection result, adding the voice tag to the clean voice sample; The clean speech samples are subjected to data enhancement processing to obtain the speech training samples.

7. The method according to claim 1, characterized in that The method of extracting features according to the speech training samples to obtain root magnitude spectrum feature samples and Fbank feature samples includes: The speech training samples are framed, windowed and fast Fourier transformed, and modulo-sampled to obtain amplitude spectrum samples; The amplitude spectrum samples are squared, passed through a Mel filter bank and logarithm is taken to obtain Fbank feature samples; The square root of the amplitude spectrum sample is taken to obtain a square root amplitude spectrum feature sample.

8. A voice activity detection system, characterized in that: include: An acquisition unit, configured to acquire a speech training sample carrying a speech label, wherein the speech label is used to indicate whether a speech signal exists in each frame of the speech training sample; An extraction unit, used for performing feature extraction based on the speech training sample to obtain a root magnitude spectrum feature sample and an Fbank feature sample; A construction unit is used to construct a voice activity detection model, wherein the voice activity detection model is used to perform feature fusion on the root amplitude spectrum features and Fbank features obtained by performing feature extraction on the voice data to obtain fused features, and output a probability value of the presence of a voice signal in each frame of the voice data based on the fused features; A training unit, configured to train the voice activity detection model using the square root magnitude spectrum feature samples and the Fbank feature samples to obtain a trained voice activity detection model; The detection unit is used to perform voice activity detection using the trained voice activity detection model.

9. A voice activity detection terminal, characterized in that: include: Storage media and processors; The storage medium stores computer-executable instructions; The processor executes the computer-executable instructions stored in the storage medium to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images