Voice activity detection model training method, voice activity detection method and device

Abstract

The present disclosure provides a training method for a voice activity detection model, a voice activity detection method, and a device. The training method includes obtaining a training set, wherein the training set includes multiple voice training samples; performing conversion processing on the voice training samples to obtain target logarithmic Mel-spectrogram features; processing the target logarithmic Mel-spectrogram features using a gated convolution layer and a maximum pooling layer to obtain an encoding result, wherein the convolutional encoding module includes a gated convolution layer and a maximum pooling layer; processing the encoding result using a first fully connected layer to obtain a predicted label, wherein the predicted label represents whether a voice signal exists in the voice training sample; processing the encoding result using a residual decoding module to obtain a predicted result; inputting the predicted label and the predicted result into a loss function to output a loss result; iteratively adjusting the network parameters of an initial voice detection model according to the loss result to obtain a trained voice activity detection model, wherein the initial voice detection model includes a convolutional encoding module, a first fully connected layer, and a residual decoding module.

Description

Technical Field

[0001] The present disclosure relates to the technical field of speech data processing, and more specifically, to a method for training a voice activity detection model, a voice activity detection method, a device for training a voice activity detection model, a voice activity detection device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Voice activity detection (VAD) aims to identify the presence of speech signals in audio signals contaminated by various background noises. It is often used as a front-end preprocessor and significantly impacts the performance of back-end tasks. For example, in automatic speech recognition (ASR), studies have shown that even with minimal background noise, half of the word error rate is associated with front-end VAD mismatch. In speech coding tasks, VAD can be effectively utilized to reduce average bitrate and multi-channel interference. VAD applications also include speech separation, enhancement, and speaker identification.

[0003] In implementing the concepts of this disclosure, the inventors discovered at least the following issues with the related art: When considering integrating a front-end voice activity detection module with more complex subsequent speech tasks, several key factors need to be considered. First, the desired VAD model needs to be lightweight to operate efficiently in resource-constrained environments. Second, it also needs to have low latency to ensure real-time performance and immediate response. Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a method for training a voice activity detection model, a voice activity detection method, a device for training a voice activity detection model, a voice activity detection device, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] One aspect of an embodiment of the present disclosure provides a method for training a voice activity detection model, including:

[0006] Obtaining a training set, wherein the training set includes a plurality of speech training samples;

[0007] For each of the above speech training samples, convert the above speech training samples to obtain target logarithmic Mel spectrum features;

[0008] The target logarithmic mel spectrum features are processed using a convolutional coding module to obtain a coding result, wherein the convolutional coding module includes a gated convolution layer and a maximum pooling layer;

[0009] Processing the encoding result using a first fully connected layer to obtain a predicted label, wherein the predicted label indicates whether a speech signal exists in the speech training sample;

[0010] The residual decoding module is used to process the above encoding results to obtain the prediction results;

[0011] Input the above predicted labels and the above predicted results into the loss function and output the loss result;

[0012] The network parameters of the initial speech detection model are iteratively adjusted according to the above loss results to obtain a trained voice activity detection model, wherein the initial speech detection model includes the above convolutional encoding module, the first fully connected layer and the above residual decoding module.

[0013] According to an embodiment of the present disclosure, the above-mentioned speech training sample is converted to obtain target logarithmic Mel spectrum features, including:

[0014] Performing frame processing on the speech training sample to obtain multiple speech sub-signals;

[0015] For each of the above-mentioned speech sub-signals, performing short-time Fourier transform processing on the above-mentioned speech sub-signal to obtain frequency domain information;

[0016] Perform log-Mel filtering on the above frequency domain information to obtain the initial log-Mel spectrum features;

[0017] The target logarithmic Mel-spectrogram feature is generated based on the multiple initial logarithmic Mel-spectrogram features.

[0018] According to an embodiment of the present disclosure, the above-mentioned speech training sample is subjected to frame processing to obtain multiple speech sub-signals, including:

[0019] Based on a preset step size, the speech training sample is segmented and processed using a preset Hanning window to obtain the plurality of speech sub-signals.

[0020] According to an embodiment of the present disclosure, the gated convolution layer includes a plurality of convolution layers;

[0021] The convolutional coding module is used to process the target logarithmic Mel spectrum features to obtain the encoding results, including:

[0022] The convolutional layer in the first part is used to process the target logarithmic Mel spectrum features to obtain speech features;

[0023] The convolutional layer in the second part is used to process the target log-mel spectrum features to obtain the mask weights.

[0024] Generate initial convolution features based on the above speech features and the above mask weights;

[0025] The above-mentioned maximum pooling layer is used to process the above-mentioned initial convolution features to generate the above-mentioned encoding result.

[0026] According to an embodiment of the present disclosure, the residual decoding module includes i residual blocks connected in sequence and a second fully connected layer connected to the last residual block;

[0027] The residual decoding module is used to process the above encoding results to obtain the prediction results, including:

[0028] When i is equal to 1, the encoding result is processed using the i-th residual block to obtain an initial output result;

[0029] When i is not equal to 1, the initial output result of the i-1th residual block is processed by the i-th residual block to obtain the target output result;

[0030] The target output result output by the last residual block is processed by the second fully connected layer to obtain the above prediction result.

[0031] According to an embodiment of the present disclosure, for the i-th residual block:

[0032] Processing input information using a convolution block to obtain initial output features, wherein the convolution block includes multiple convolution layers, and the input information represents an encoding result of the i-th residual block or the initial output result;

[0033] A target output feature is generated based on the input information and the initial output feature, wherein the target output feature represents the initial output result or the target output result output by the i-th residual block.

[0034] According to an embodiment of the present disclosure, the above-mentioned prediction label and the above-mentioned prediction result are input into the loss function, and the loss result is output, including:

[0035] Based on a preset activation function, the predicted label and the predicted result are input into a binary cross entropy function, and the loss result is output, wherein the binary cross entropy function is generated according to hyperparameters determined by the importance between the convolutional encoding module and the residual decoding module.

[0036] Another aspect of the present disclosure provides a voice activity detection method, including:

[0037] Get detected voice information;

[0038] Inputting the detected voice information into a voice activity detection model and outputting a recognition result, wherein the recognition result indicates whether a voice signal exists in the detected voice information;

[0039] The above-mentioned voice activity detection model is trained based on the above-mentioned method.

[0040] Another aspect of the embodiments of the present disclosure provides a training device for a voice activity detection model, comprising:

[0041] A first acquisition module is used to acquire a training set, wherein the training set includes a plurality of speech training samples;

[0042] A conversion module is used to convert each of the above-mentioned speech training samples to obtain target logarithmic Mel spectrum features;

[0043] A first processing module is configured to process the target logarithmic Mel spectrum feature using a convolutional coding module to obtain an encoding result, wherein the convolutional coding module includes the gated convolution layer and the maximum pooling layer;

[0044] a second processing module, configured to process the encoding result using the first fully connected layer to obtain a predicted label, wherein the predicted label indicates whether a speech signal exists in the speech training sample;

[0045] A third processing module is used to process the encoding result using the residual decoding module to obtain a prediction result;

[0046] A loss calculation module is used to input the above-mentioned prediction label and the above-mentioned prediction result into the loss function and output the loss result;

[0047] An iterative adjustment module is used to iteratively adjust the network parameters of the above-mentioned initial speech detection model according to the above-mentioned loss result to obtain a trained voice activity detection model, wherein the initial speech detection model includes the above-mentioned convolutional encoding module, the first fully connected layer and the above-mentioned residual decoding module.

[0048] Another aspect of the present disclosure provides a voice activity detection device, including:

[0049] The second acquisition module is used to acquire the detected voice information;

[0050] a detection module, configured to input the detected voice information into a voice activity detection model and output a recognition result, wherein the recognition result indicates whether a voice signal exists in the detected voice information;

[0051] The above-mentioned voice activity detection model is trained based on the above-mentioned method.

[0052] Another aspect of an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

[0053] Another aspect of an embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0054] Another aspect of an embodiment of the present disclosure provides a computer program product, which includes computer-executable instructions, and the instructions are used to implement the method described above when executed.

[0055] According to an embodiment of the present disclosure, a speech activity detection model is constructed by converting speech training samples into target logarithmic Mel-spectrogram features, then using a convolutional encoding module containing only gated convolutional layers and maximum pooling layers to generate an encoding result based on the target logarithmic Mel-spectrogram features. Based on the encoding result, a prediction label and a prediction result are generated using a first fully connected layer and a residual decoding module, respectively. The network parameters of the initial speech detection model are adjusted based on the loss results calculated from the prediction label and the prediction result. Since the convolutional encoding module only contains gated convolutional layers and maximum pooling layers and deletes the batch normalization layer, the number of model parameters is reduced, thereby achieving model lightweighting without significantly reducing the model's prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0057] Figure 1 Schematically illustrates an exemplary system architecture to which a training method for a voice activity detection model or a voice activity detection method according to an embodiment of the present disclosure can be applied;

[0058] Figure 2 The following schematically shows a flow chart of a method for training a voice activity detection model according to an embodiment of the present disclosure;

[0059] Figure 3 The following schematically shows a structural diagram of a voice activity detection model according to an embodiment of the present disclosure;

[0060] Figure 4 Schematically shows a structural diagram of a convolutional coding module according to an embodiment of the present disclosure;

[0061] Figure 5 Schematically shows a structural diagram of a residual decoding module according to an embodiment of the present disclosure;

[0062] Figure 6 The following schematically shows a flow chart of a voice activity detection method according to an embodiment of the present disclosure;

[0063] Figure 7 A block diagram schematically illustrates a training apparatus for a voice activity detection model according to an embodiment of the present disclosure;

[0064] Figure 8 The following schematically shows a block diagram of a voice activity detection apparatus according to an embodiment of the present disclosure;

[0065] Figure 9 A block diagram of an electronic device suitable for implementing the above-described method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0066] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0067] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0068] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0069] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0070] Traditional VAD methods primarily rely on energy-based features, such as time-domain power, spectral characteristics, short-term energy, and spectral entropy. However, accurately distinguishing between human voices and noise using traditional methods is challenging in environments with low signal-to-noise ratios (SNRs). In recent years, with the development of neural networks and their application in speech processing, deep learning models such as deep fully connected networks and convolutional neural networks have also garnered research attention in this field. While these methods have shown improved performance compared to traditional methods, they are still significantly affected by noise conditions.

[0071] To further overcome these limitations, related technologies have proposed some more complex models. For example, by using Multi-Resolution CochleaGram (MRCG) to train the bottleneck deep neural network (bDNN), the network can achieve better performance, but at the same time requires a higher computational burden. Some related technologies have proposed an Adaptive Context Attention Model (ACAM) based on the attention mechanism, which utilizes contextual information during training and performs better than bDNN. However, the training process of ACAM is not stable. Some related technologies have proposed a VAD model (Spectro-Temporal Attention-based Model, STAM) based on time-frequency attention. This model is built on the basis of ACAM with the same feature input and can improve training stability.

[0072] Because front-end voice activity detection modules are often combined with more complex subsequent speech tasks, in addition to noise robustness, the desired VAD model must meet two important requirements: lightweight model size and low latency. Model size is not only related to spatial complexity but also affects decoding time, which directly determines latency performance. For example, in the ACAM and STAM methods, the use of contextual information makes VAD non-causal, meaning that input features must be constructed after waiting for several time frames. However, this high-latency, non-causal design is clearly incompatible with online speech tasks such as streaming automatic speech recognition.

[0073] In view of this, the embodiments of the present disclosure provide a training method for a voice activity detection model, a voice activity detection method, and a device. The training method includes obtaining a training set, wherein the training set includes multiple voice training samples; for each voice training sample, performing conversion processing on the voice training sample to obtain a target logarithmic Mel-spectrogram feature; using a gated convolution layer and a maximum pooling layer to process the target logarithmic Mel-spectrogram feature to obtain an encoding result, wherein the convolutional encoding module includes a gated convolution layer and a maximum pooling layer; using a first fully connected layer to process the encoding result to obtain a predicted label, wherein the predicted label represents whether there is a voice signal in the voice training sample; using a residual decoding module to process the encoding result to obtain a predicted result; inputting the predicted label and the predicted result into a loss function and outputting a loss result; iteratively adjusting the network parameters of the initial voice detection model according to the loss result to obtain a trained voice activity detection model, wherein the initial voice detection model includes a convolutional encoding module, a first fully connected layer, and a residual decoding module.

[0074] Figure 1 The following schematically illustrates an exemplary system architecture 100 to which a training method for a voice activity detection model or a voice activity detection method according to an embodiment of the present disclosure can be applied. Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0075] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links.

[0076] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0077] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0078] Server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using terminal devices 101, 102, and 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal device.

[0079] It should be noted that the voice activity detection model training method or voice activity detection method provided in the embodiments of the present disclosure can generally be performed by the server 105. Accordingly, the voice activity detection model training device or voice activity detection device provided in the embodiments of the present disclosure can generally be set in the server 105. The voice activity detection model training method or voice activity detection method provided in the embodiments of the present disclosure can also be performed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the voice activity detection model training device or voice activity detection device provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Alternatively, the voice activity detection model training method or voice activity detection method provided in the embodiments of the present disclosure can also be performed by the terminal devices 101, 102, or 103, or can also be performed by other terminal devices different from the terminal devices 101, 102, or 103. Accordingly, the training device of the voice activity detection model or the voice activity detection device provided in the embodiments of the present disclosure may also be provided in the terminal device 101, 102, or 103, or in other terminal devices different from the terminal device 101, 102, or 103.

[0080] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0081] Figure 2 The flowchart of the method for training a voice activity detection model according to an embodiment of the present disclosure is schematically shown. Figure 3 The structural diagram of the voice activity detection model according to an embodiment of the present disclosure is schematically shown.

[0082] like Figure 2 As shown, the training method of the voice activity detection model includes operations S201 to S207.

[0083] In operation S201, a training set is obtained, wherein the training set includes a plurality of speech training samples;

[0084] In operation S202, for each speech training sample, the speech training sample is converted to obtain a target logarithmic Mel spectrum feature;

[0085] In operation S203, a convolutional coding module is used to process the target logarithmic mel spectrum feature to obtain an encoding result, wherein the convolutional coding module includes a gated convolution layer and a maximum pooling layer;

[0086] In operation S204, the encoding result is processed using the first fully connected layer to obtain a predicted label, wherein the predicted label represents whether a speech signal exists in the speech training sample;

[0087] In operation S205, the encoding result is processed using a residual decoding module to obtain a prediction result;

[0088] In operation S206, the predicted label and the predicted result are input into the loss function, and the loss result is output;

[0089] In operation S207, network parameters of the initial speech detection model are iteratively adjusted according to the loss result to obtain a trained voice activity detection model, wherein the initial speech detection model includes a convolutional encoding module, a first fully connected layer, and a residual decoding module.

[0090] According to the embodiments of the present disclosure, Figure 3 As shown, for each speech training sample in the training set, it is converted into a target log-Mel spectrum feature X. The target log-Mel spectrum feature is then input into the convolutional coding module, so that the gated convolution layer and the maximum pooling layer in the convolutional coding module generate a coding result based on the target log-Mel spectrum feature. Since the convolutional coding module of the present disclosure only includes the gated convolution layer and the maximum pooling layer in the spectrum attention module of the gated convolutional neural network (CNN), and deletes the batch normalization (BN) layer of the spectrum attention module of the STAM model, it effectively reduces the number of parameters without seriously affecting the performance. At the same time, the coding result is also input into an additional first fully connected layer to output a binary result of whether it contains speech, that is, a predicted label. If the binary result is 1, it means that there is a speech signal in the speech training sample, and if it is 0, it means that there is no speech signal in the speech training sample.

[0091] According to an embodiment of the present disclosure, the encoding result is processed by a residual decoding module to obtain a prediction result, which is also a binary result of whether it contains speech. The prediction label and the prediction result are input into the loss function, and the loss result is output. The network parameters of the initial speech detection model are iteratively adjusted according to the loss result to obtain a trained voice activity detection model.

[0092] According to an embodiment of the present disclosure, a speech activity detection model is constructed by converting speech training samples into target logarithmic Mel-spectrogram features, then using a convolutional encoding module containing only gated convolutional layers and maximum pooling layers to generate an encoding result based on the target logarithmic Mel-spectrogram features. Based on the encoding result, a prediction label and a prediction result are generated using a first fully connected layer and a residual decoding module, respectively. The network parameters of the initial speech detection model are adjusted based on the loss results calculated from the prediction label and the prediction result. Since the convolutional encoding module only contains gated convolutional layers and maximum pooling layers and deletes the batch normalization layer, the number of model parameters is reduced, thereby achieving model lightweighting without significantly reducing the model's prediction accuracy.

[0093] According to an embodiment of the present disclosure, a speech training sample is converted to obtain a target logarithmic Mel-spectrogram feature, including: framing the speech training sample to obtain multiple speech sub-signals; performing short-time Fourier transform processing on each speech sub-signal to obtain frequency domain information; performing logarithmic Mel-spectrogram filtering on the frequency domain information to obtain an initial logarithmic Mel-spectrogram feature; and generating a target logarithmic Mel-spectrogram feature based on the multiple initial logarithmic Mel-spectrogram features.

[0094] According to an embodiment of the present disclosure, in order to obtain a frame-level feature signal, the input speech training sample is divided into frames to obtain multiple speech sub-signals. Secondly, the speech sub-signals of each frame are transformed using a 1024-point short-time Fourier transform (STFT) to obtain frequency domain information, and then the frequency domain information is passed through a log-Mel filter with D=80 to obtain the initial log-Mel spectrum feature. Finally, the initial log-Mel spectrum features of multiple frames are spliced ​​to obtain the target log-Mel spectrum feature with contextual information, which includes information of the current frame and past frames.

[0095] According to an embodiment of the present disclosure, since the target logarithmic mel-spectrogram features and predicted labels of the input voice activity detection model are constructed using a series of frames, that is, the feature vector at time index T contains information of the current and past frames, it is given by formula (1):

[0096]

[0097] in, is the target logarithmic Mel spectrum feature, represents the set of relative time indices of the frames under consideration, Represents the frame-level logarithmic Mel spectrum acoustic features. Similarly, the vector of the predicted label at the current time step is shown in formula (2):

[0098]

[0099] Where T represents the transpose of the vector / matrix. It is obvious from the considered feature formation that only the current and past information is used when detecting the state of the current frame, without the need for future frames, which leads to the causal nature of the proposed voice activity detection model. In particular, when When it is 1, it represents the current frame speech signal; when When it is 0, it means there is no speech signal in the current frame.

[0100] According to an embodiment of the present disclosure, a speech training sample is framed to obtain a plurality of speech sub-signals, including: segmenting the speech training sample using a preset Hanning window based on a preset step size to obtain a plurality of speech sub-signals.

[0101] According to an embodiment of the present disclosure, the preset step size and the preset Hanning window can be adjusted according to actual conditions, for example, the preset step size is 10 milliseconds and the Hanning window is 25 milliseconds.

[0102] Figure 4 The structural diagram of the convolutional coding module according to an embodiment of the present disclosure is schematically shown.

[0103] According to an embodiment of the present disclosure, the gated convolution layer includes multiple convolution layers.

[0104] The convolutional coding module is used to process the target logarithmic Mel-spectrogram features to obtain the coding result, including: using the convolution layer of the first part to process the target logarithmic Mel-spectrogram features to obtain speech features; using the convolution layer of the second part to process the target logarithmic Mel-spectrogram features to obtain mask weights; generating initial convolution features based on the speech features and the mask weights; and using the maximum pooling layer to process the initial convolution features to generate the coding result.

[0105] According to an embodiment of the present disclosure, the convolution kernel size in the convolution layer is 3×3. The input and output channels increase from {1, 2} to {8, 16} according to the number of convolution layers. With each increase in the number of convolution layers, the input and output channels of the corresponding layer become twice that of the previous layer. The window size of the pooling layer is 2×2. The number of hidden units and output units in the first fully connected layer is 256 and 1, respectively. In this embodiment, the number of convolution layers is empirically set to 4.

[0106] It should be noted that the above parameters can be set according to actual needs, and the setting of the above parameters does not limit the scope of protection of the present disclosure. For example, the number of convolutional layers or the size of the convolution kernel can be set according to actual needs.

[0107] According to an embodiment of the present disclosure, the convolutional coding module is composed of a gated convolution layer consisting of multiple convolutional layers and a maximum pooling layer, such as Figure 4As shown in the dotted box in . Given the target input logarithmic Mel spectrum feature , the convolutional layer at the top is responsible for extracting speech features , the convolutional layer below generates the corresponding mask weights Voice Features and mask weights Multiply to generate the initial convolution feature, which is then input into the maximum pooling layer. The maximum pooling layer outputs the encoding result E, and the encoding result E is fed into an additional first fully connected layer to obtain the predicted label for loss calculation during training. Among them, the predicted label can be considered as a predicted output of the voice activity detection model.

[0108] Figure 5 The schematic diagram shows the structure of the residual decoding module according to the embodiment of the present disclosure.

[0109] According to the embodiments of the present disclosure, Figure 5 As shown, the residual decoding module includes i residual blocks connected in sequence and a second fully connected layer connected to the last residual block.

[0110] The residual decoding module is used to process the encoding result to obtain the prediction result, including:

[0111] When i is equal to 1, the i-th residual block is used to process the encoding result to obtain the initial output result; when i is not equal to 1, the i-th residual block is used to process the initial output result output by the i-1-th residual block to obtain the target output result; the second fully connected layer is used to process the target output result output by the last residual block to obtain the prediction result.

[0112] According to an embodiment of the present disclosure, the predicted label It is the output obtained by a first fully connected layer only on the basis of the encoding result E. This means that the encoding result E already contains some semantic information, and it is hoped to retain this information as much as possible in the residual decoding module. To achieve this, residual connections are used in the residual decoding module, which helps to retain information from the previous stage and only train the error module of the feature information. By stacking multiple residual blocks, the residual block can achieve a large receptive field with a small number of parameters, which benefits from the properties of the convolution kernel in the residual block. Four residual convolution blocks (i.e., the residual blocks disclosed in the present invention) can be configured in the residual decoding module, each residual convolution block includes at least two convolution layers, and multiple residual convolution blocks form a pipeline structure.

[0113] According to the embodiment of the present disclosure, referring to Figure 5After the encoding result E is input into the residual decoding module, the first residual block generates an initial output result according to the encoding result E. The initial output result is input into the second residual block, which outputs the target output result. The target output result of the second residual block is input into the third residual block, which outputs the target output result. And so on. The target output result output by the last residual block is processed by the second fully connected layer to obtain the prediction result. .

[0114] According to an embodiment of the present disclosure, for the i-th residual block: the input information is processed using a convolution block to obtain an initial output feature, wherein the convolution block includes multiple convolution layers, and the input information represents the encoding result or the initial output result of the input i-th residual block; based on the input information and the initial output feature, a target output feature is generated, wherein the target output feature represents the initial output result or the target output result output by the i-th residual block.

[0115] According to an embodiment of the present disclosure, the convolution kernel size of the convolution layer in the convolution block is 3×3, the number of channels in each convolution layer is {1, 4, 1}, and the number of output units in the second fully connected layer is 1. The number of residual blocks can be specifically set according to actual needs, for example, 4.

[0116] According to the embodiment of the present disclosure, for each residual block, assuming it is the first residual block, the convolution layer in the first residual block extracts features from the encoding result E to obtain an initial output feature. The sum of the initial output feature and the encoding result E is used as the output of the first residual block, that is, the target output feature. Then, the target output feature output by the first residual block is used as the input information of the second residual block, and the same processing method as the first residual block is performed. Similarly, the target output feature output by the last residual block is input to the second fully connected layer to obtain the prediction result. .

[0117] According to an embodiment of the present disclosure, the predicted label and the predicted result are input into the loss function, and the loss result is output, including: based on a preset activation function, the predicted label and the predicted result are input into a binary cross entropy function, and the loss result is output, wherein the binary cross entropy function is generated according to the hyperparameters determined by the importance between the convolutional encoding module and the residual decoding module.

[0118] According to an embodiment of the present disclosure, the Adam optimizer can be used during training, and the learning rate is becomes The learning rate of each round is attenuated by 0.8 relative to the previous step. The predicted label output of the convolutional coding module is used And the prediction result of the output of the residual decoding module To calculate the binary cross entropy (CE) loss relative to the true value, the loss result is shown in formula (3):

[0119]

[0120] in, represents the binary cross entropy loss function, Represents the sigmoid activation function, hyperparameter Used to specify the importance of the convolutional encoding module and the residual decoding module. It can be set according to actual needs, for example, 0.7. It should be noted that all frames are considered in the loss calculation.

[0121] In one embodiment, the present disclosure performs model training and experimental verification on an Intel Xeon E5-2680 CPU and an NVIDIA GeForce GTX3090 GPU. The model is built using Python and the PyTorch framework. In terms of hyperparameter configuration, all experiments are conducted using the Adam optimizer, with a learning rate of becomes , the learning rate of each round decays by 0.8 relative to the previous step.

[0122] Two datasets, QUT-NOISE-TIMIT and LibriSpeech, were used to validate the VAD performance described in this paper. The QUT-NOISE-TIMIT dataset was created by mixing the TIMIT dataset with the QUT-NOISE background noise dataset. This mixing operation yielded 600 hours of noisy speech data across ten scenarios at six different signal-to-noise ratio (SNR) levels (-10, -5, 0, 5, 10, and 15 dB). To maintain the independence of the training and test sets, 100 hours of data were randomly selected as the training set, and another 100 hours as the test set. This ensured that the training and test sets contained no overlapping segments of speech or background noise.

[0123] Because LibriSpeech was primarily collected for English ASR tasks, this paper mixed the LibriSpeech-dev-clean dataset with NoiseX-92 to create another noisy test dataset. This dataset contains 15 hours of recordings from 15 environments and 6 different SNR levels (-10, -5, 0, 5, 10, 15) dB. Since this paper focuses on VAD under low SNR conditions, only the results with SNR∈{-10, -5, 0, 5} dB are tested.

[0124] According to an embodiment of the present disclosure, the QUT-NOISE-TIMIT dataset is used for both training and testing, while the LibriSpeech dataset is used only for testing. ACAM, STAM, and the proposed model were tested on these two datasets. STAM's input signal is non-causal, while the proposed model's input signal is causal.

[0125] According to the embodiments of the present disclosure, the Area Under the Curve (AUC) is used to measure the accuracy of VAD. This metric represents the area under the Receiver Operating Characteristic (ROC) curve. Furthermore, the THOP package in Python is used to calculate the number of parameters and floating-point operations (FLOPs), which are used to measure the spatial complexity and computational complexity of the model, respectively.

[0126] Tables 1 and 2 compare the prediction results for two datasets, respectively, according to the embodiments of the present disclosure. It can be clearly seen on both datasets that the proposed method achieves the best performance, outperforming the existing best non-causal STAM model, regardless of the noise level. This is primarily due to the fact that the proposed convolutional coding module is already capable of performing raw VAD prediction, and residual decoding provides further label refinement.

[0127] Table 1. AUC scores on the QUT-NOISE-TIMIT dataset.

[0128]

[0129] Table 2. AUC scores on the noisy LibriSpeech dataset.

[0130]

[0131] It can be observed from Table 3 that, compared with the existing best non-causal STAM model, in addition to the superiority in performance, the method proposed in this disclosure introduces fewer FLOPs and parameters, showing wider applicability.

[0132] Table 3. Comparison of model complexity.

[0133]

[0134] Table 4 shows a more complete comparison with the STAM model on the QUT-NOISE-TIMIT dataset, which takes into account several types of additive noise and different signal-to-noise ratio (SNR) noise levels. It can be seen that in the CAFE-FOODCOURTB, REVERB-CARPARK, and REVERB-POOL environments, the method proposed in the present disclosure achieves better results than the non-causal STAM, where REVERB in the latter two represents the reverberation environment. Under the remaining conditions, the performance of the two methods is almost equivalent. In the two reverberation environments, the AUC of the voice activity detection model proposed in the present disclosure is 2.8% higher than that of STAM on average, showing stronger anti-reverberation robustness.

[0135] Table 4. AUC scores for different environments on the QUT-NOISE-TIMIT dataset.

[0136]

[0137] Moreover, the real-time factor of the voice activity detection model proposed in the present disclosure on the QUT-NOISE-TIMIT dataset is about 0.03, which is acceptable for real-time applications.

[0138] In general, compared with STAM, the voice activity detection model proposed in the present disclosure can not only solve the problem of non-causality, but also improve the accuracy of VAD by an average of 0.74% while reducing the number of parameters and calculations.

[0139] Figure 6 The flowchart of the voice activity detection method according to an embodiment of the present disclosure is schematically shown.

[0140] like Figure 6 As shown, the voice activity detection method includes operations S601 to S602.

[0141] In operation S601, detecting voice information is acquired;

[0142] In operation S602 , the detection voice information is input into a voice activity detection model, and a recognition result is output, wherein the recognition result indicates whether a voice signal exists in the detection voice information.

[0143] According to an embodiment of the present disclosure, the detection voice information can be collected by a voice collection device such as a microphone, and the collected detection voice information is input into the voice activity detection model of the present disclosure to identify whether there is a voice signal in the detection voice information.

[0144] According to an embodiment of the present disclosure, during the training process of a voice activity detection model, speech training samples are converted into target logarithmic Mel-spectrogram features. A convolutional encoding module comprising only gated convolutional layers and maximum pooling layers is then used to generate an encoding result based on the target logarithmic Mel-spectrogram features. Based on the encoding result, a first fully connected layer and a residual decoding module are used to generate a predicted label and a predicted result, respectively. The network parameters of the initial voice detection model are adjusted based on the loss calculated from the predicted label and the predicted result, thereby achieving a voice activity detection model. Because the convolutional encoding module only comprises gated convolutional layers and maximum pooling layers and eliminates the batch normalization layer, the model's parameter count is reduced, thereby achieving a lightweight model without significantly reducing the model's prediction accuracy, thereby facilitating efficient operation in resource-constrained environments.

[0145] Figure 7 A block diagram of a training apparatus for a voice activity detection model according to an embodiment of the present disclosure is schematically shown.

[0146] like Figure 7 As shown, the training device 700 for the voice activity detection model includes a first acquisition module 710 , a conversion module 720 , a first processing module 730 , a second processing module 740 , a third processing module 750 , a loss calculation module 760 and an iterative adjustment module 770 .

[0147] A first acquisition module 710 is configured to acquire a training set, wherein the training set includes a plurality of speech training samples;

[0148] The conversion module 720 is used to convert each speech training sample to obtain a target logarithmic Mel spectrum feature;

[0149] A first processing module 730 is configured to process the target logarithmic Mel spectrum feature using a convolutional coding module to obtain a coding result, wherein the convolutional coding module includes a gated convolution layer and a maximum pooling layer;

[0150] A second processing module 740 is configured to process the encoding result using the first fully connected layer to obtain a predicted label, wherein the predicted label indicates whether a speech signal exists in the speech training sample;

[0151] The third processing module 750 is used to process the encoding result using the residual decoding module to obtain a prediction result;

[0152] The loss calculation module 760 is used to input the predicted label and the predicted result into the loss function and output the loss result;

[0153] The iterative adjustment module 770 is used to iteratively adjust the network parameters of the initial speech detection model according to the loss result to obtain a trained voice activity detection model, wherein the initial speech detection model includes a convolutional encoding module, a first fully connected layer and a residual decoding module.

[0154] According to an embodiment of the present disclosure, a speech activity detection model is constructed by converting speech training samples into target logarithmic Mel-spectrogram features, then using a convolutional encoding module containing only gated convolutional layers and maximum pooling layers to generate an encoding result based on the target logarithmic Mel-spectrogram features. Based on the encoding result, a prediction label and a prediction result are generated using a first fully connected layer and a residual decoding module, respectively. The network parameters of the initial speech detection model are adjusted based on the loss results calculated from the prediction label and the prediction result. Since the convolutional encoding module only contains gated convolutional layers and maximum pooling layers and deletes the batch normalization layer, the number of model parameters is reduced, thereby achieving model lightweighting without significantly reducing the model's prediction accuracy.

[0155] According to an embodiment of the present disclosure, the conversion module 720 includes a framing unit, a conversion unit, a filtering unit, and a first generating unit.

[0156] A framing unit, configured to perform framing processing on the speech training sample to obtain multiple speech sub-signals;

[0157] a conversion unit, configured to perform short-time Fourier transform processing on each speech sub-signal to obtain frequency domain information;

[0158] A filtering unit is used to perform log-Mel filtering on the frequency domain information to obtain initial log-Mel spectrum features;

[0159] The first generating unit is configured to generate a target logarithmic Mel spectrum feature according to a plurality of initial logarithmic Mel spectrum features.

[0160] According to an embodiment of the present disclosure, the framing unit includes a framing sub-unit.

[0161] The framing subunit is used to perform segmentation processing on the speech training sample based on a preset step size and using a preset Hanning window to obtain multiple speech sub-signals.

[0162] According to an embodiment of the present disclosure, the gated convolution layer includes a plurality of convolutional layers;

[0163] According to an embodiment of the present disclosure, the first processing module 730 includes a first convolution unit, a second convolution unit, a second generation unit, and a third generation unit.

[0164] A first convolution unit is used to process the target logarithmic Mel spectrum feature using the convolution layer of the first part to obtain the speech feature;

[0165] The second convolution unit is used to process the target logarithmic Mel spectrum features using the convolution layer of the second part to obtain the mask weight;

[0166] A second generation unit is used to generate initial convolution features according to the speech features and the mask weights;

[0167] The third generation unit is used to process the initial convolution features using the maximum pooling layer to generate the encoding result.

[0168] According to an embodiment of the present disclosure, the residual decoding module includes i residual blocks connected in sequence and a second fully connected layer connected to the last residual block;

[0169] According to an embodiment of the present disclosure, the third processing module 740 includes a first processing unit, a second processing unit, and an obtaining unit.

[0170] A first processing unit is configured to, when i is equal to 1, process the encoding result using the i-th residual block to obtain an initial output result;

[0171] A second processing unit is configured to, when i is not equal to 1, process an initial output result output by the (i-1)th residual block using the (i)th residual block to obtain a target output result;

[0172] A unit is obtained, which is used to use the second fully connected layer to process the target output result output by the last residual block to obtain a prediction result.

[0173] According to an embodiment of the present disclosure, any one of the first processing unit and the second processing unit includes a convolution subunit and a generation subunit.

[0174] A convolution subunit is configured to process input information using a convolution block to obtain initial output features, wherein the convolution block includes multiple convolution layers, and the input information represents the encoding result or initial output result of the input i-th residual block;

[0175] The generating subunit is used to generate a target output feature according to the input information and the initial output feature, wherein the target output feature represents the initial output result or the target output result output by the i-th residual block.

[0176] According to an embodiment of the present disclosure, the loss calculation module 760 includes a loss calculation unit.

[0177] A loss calculation unit is configured to input the predicted label and the predicted result into a binary cross entropy function based on a preset activation function, and output a loss result, wherein the binary cross entropy function is generated according to hyperparameters determined by the importance between the convolutional encoding module and the residual decoding module.

[0178] Figure 8 The block diagram of a voice activity detection apparatus according to an embodiment of the present disclosure is schematically shown.

[0179] like Figure 8 As shown, the voice activity detection apparatus 800 includes a second acquisition module 810 and a detection module 820 .

[0180] The second acquisition module 810 is used to acquire the detected voice information;

[0181] The detection module 820 is configured to input the detected voice information into a voice activity detection model and output a recognition result, wherein the recognition result indicates whether a voice signal exists in the detected voice information.

[0182] According to an embodiment of the present disclosure, during the training process of a voice activity detection model, speech training samples are converted into target logarithmic Mel-spectrogram features. A convolutional encoding module comprising only gated convolutional layers and maximum pooling layers is then used to generate an encoding result based on the target logarithmic Mel-spectrogram features. Based on the encoding result, a first fully connected layer and a residual decoding module are used to generate a predicted label and a predicted result, respectively. The network parameters of the initial voice detection model are adjusted based on the loss calculated from the predicted label and the predicted result, thereby achieving a voice activity detection model. Because the convolutional encoding module only comprises gated convolutional layers and maximum pooling layers and eliminates the batch normalization layer, the model's parameter count is reduced, thereby achieving a lightweight model without significantly reducing the model's prediction accuracy, thereby facilitating efficient operation in resource-constrained environments.

[0183] According to the embodiments of the present invention, any number of modules, units, and sub-units, or at least part of the functions of any number of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, units, and sub-units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, units, and sub-units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, units, and sub-units can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is executed.

[0184] For example, any of the first acquisition module 710, the conversion module 720, the first processing module 730, the second processing module 740, the third processing module 750, the loss calculation module 760, and the iterative adjustment module 770, or the second acquisition module 810 and the detection module 820 can be combined into one module / unit / sub-unit for implementation, or any of these modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functionality of one or more of these modules / units / sub-units can be combined with at least part of the functionality of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first acquisition module 710, the conversion module 720, the first processing module 730, the second processing module 740, the third processing module 750, the loss calculation module 760, and the iterative adjustment module 770, or the second acquisition module 810 and the detection module 820 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware by any other reasonable means of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the first acquisition module 710, the conversion module 720, the first processing module 730, the second processing module 740, the third processing module 750, the loss calculation module 760, and the iterative adjustment module 770, or the second acquisition module 810 and the detection module 820 can be at least partially implemented as a computer program module, which can perform the corresponding function when executed.

[0185] It should be noted that the training device and voice activity detection device of the voice activity detection model in the embodiments of the present disclosure correspond to the training method and voice activity detection method of the voice activity detection model in the embodiments of the present disclosure. The description of the training device and voice activity detection device of the voice activity detection model specifically refers to the training method and voice activity detection method of the voice activity detection model, and will not be repeated here.

[0186] Figure 9 A block diagram of an electronic device suitable for implementing the above-described method according to an embodiment of the present disclosure is schematically shown. Figure 9 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0187] like Figure 9As shown, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 902 or programs loaded from a storage unit 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or related chipsets and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0188] Various programs and data required for the operation of the electronic device 900 are stored in the RAM 903. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. The processor 901 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 902 and / or the RAM 903. It should be noted that the programs may also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0189] According to an embodiment of the present disclosure, electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to bus 904. System 900 may also include one or more of the following components connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 908 including a hard disk; and a communication section 909 including a network interface card such as a LAN card or modem. Communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. Removable media 911, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 910 as needed, so that computer programs read from the removable media can be installed into storage section 908 as needed.

[0190] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0191] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0192] According to embodiments of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0193] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 902 and / or the RAM 903 described above and / or one or more memories other than the ROM 902 and the RAM 903 .

[0194] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the training method of the voice activity detection model or the voice activity detection method provided by the embodiment of the present disclosure.

[0195] When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0196] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0197] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0198] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or coupled in various ways, and all such combinations and / or couplings fall within the scope of the present disclosure.

[0199] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for training a voice activity detection model, comprising: Acquire a training set, wherein the training set includes a plurality of speech training samples; For each of the speech training samples, convert the speech training sample to obtain a target logarithmic Mel spectrum feature; Processing the target logarithmic Mel spectrum feature using a convolutional coding module to obtain an encoding result, wherein the convolutional coding module includes a gated convolution layer and a maximum pooling layer; Processing the encoding result using a first fully connected layer to obtain a predicted label, wherein the predicted label indicates whether a speech signal exists in the speech training sample; Processing the encoding result using a residual decoding module to obtain a prediction result; Input the predicted label and the predicted result into a loss function, and output a loss result; Iteratively adjust network parameters of an initial speech detection model according to the loss result to obtain a trained voice activity detection model, wherein the initial speech detection model includes the convolutional coding module, the first fully connected layer and the residual decoding module.

2. The method according to claim 1, wherein The speech training sample is converted to obtain target logarithmic Mel spectrum features, including: Performing frame processing on the speech training sample to obtain multiple speech sub-signals; For each of the speech sub-signals, performing short-time Fourier transform processing on the speech sub-signal to obtain frequency domain information; Performing log-mel filtering on the frequency domain information to obtain initial log-mel spectrum features; The target logarithmic Mel-spectrogram feature is generated according to the multiple initial logarithmic Mel-spectrogram features.

3. The method according to claim 2, wherein: The speech training sample is framed to obtain a plurality of speech sub-signals, including: Based on a preset step size, the speech training sample is segmented and processed using a preset Hanning window to obtain a plurality of speech sub-signals.

4. The method according to claim 1, wherein the gated convolutional layer comprises a plurality of convolutional layers; in, The target logarithmic Mel spectrum feature is processed using a convolutional coding module to obtain a coding result, including: Processing the target logarithmic Mel-spectrogram features using the convolutional layer of the first part to obtain speech features; Using the convolutional layer of the second part to process the target logarithmic Mel spectrum features to obtain mask weights; Generating an initial convolution feature according to the speech feature and the mask weight; The initial convolutional features are processed using the maximum pooling layer to generate the encoding result.

5. The method according to claim 1, wherein The residual decoding module includes i residual blocks connected in sequence and a second fully connected layer connected to the last residual block; The coding result is processed by a residual decoding module to obtain a prediction result, including: When i is equal to 1, the encoding result is processed using the i-th residual block to obtain an initial output result; When i is not equal to 1, the initial output result of the i-1th residual block is processed by the i-th residual block to obtain the target output result; The target output result output by the last residual block is processed by the second fully connected layer to obtain the prediction result.

6. The method according to claim 5, wherein: For the i-th residual block: Processing input information using a convolution block to obtain initial output features, wherein the convolution block includes multiple convolution layers, and the input information represents an encoding result of the i-th residual block or the initial output result; A target output feature is generated according to the input information and the initial output feature, wherein the target output feature represents an initial output result or a target output result output by the i-th residual block.

7. The method according to claim 1, wherein Inputting the predicted label and the predicted result into a loss function and outputting a loss result includes: Based on a preset activation function, the predicted label and the predicted result are input into a binary cross entropy function, and the loss result is output, wherein the binary cross entropy function is generated according to a hyperparameter determined by the importance between the convolutional encoding module and the residual decoding module.

8. A method for detecting voice activity, comprising: Get detected voice information; Inputting the detected voice information into a voice activity detection model and outputting a recognition result, wherein the recognition result indicates whether a voice signal exists in the detected voice information; The voice activity detection model is trained based on the method according to any one of claims 1 to 7.

9. A training device for a voice activity detection model, comprising: A first acquisition module is configured to acquire a training set, wherein the training set includes a plurality of speech training samples; A conversion module, configured to convert each of the speech training samples to obtain a target logarithmic Mel spectrum feature; A first processing module is configured to process the target logarithmic Mel spectrum feature using a convolutional coding module to obtain an encoding result, wherein the convolutional coding module includes a gated convolution layer and a maximum pooling layer; a second processing module, configured to process the encoding result using the first fully connected layer to obtain a predicted label, wherein the predicted label indicates whether a speech signal exists in the speech training sample; A third processing module, configured to process the encoding result using a residual decoding module to obtain a prediction result; A loss calculation module, configured to input the predicted label and the predicted result into a loss function and output a loss result; An iterative adjustment module is used to iteratively adjust network parameters of an initial speech detection model according to the loss result to obtain a trained voice activity detection model, wherein the initial speech detection model includes the convolutional coding module, the first fully connected layer and the residual decoding module.

10. A voice activity detection device, comprising: The second acquisition module is used to acquire the detected voice information; a detection module, configured to input the detected voice information into a voice activity detection model and output a recognition result, wherein the recognition result indicates whether a voice signal exists in the detected voice information; The voice activity detection model is trained based on the method according to any one of claims 1 to 7.

Images