A method, device, equipment and storage medium for speech enhancement

Abstract

The embodiment of the present invention discloses a speech enhancement method, device, equipment and storage medium. The method comprises: inputting a noisy speech signal to be processed into a target speech enhancement model; wherein the target speech enhancement model comprises a coding network, a noise reduction network and a decoding network; performing a coding operation on the noisy speech signal through the coding network to obtain an output coded speech feature; inputting the coded speech feature into the noise reduction network to obtain an output noise reduction speech feature; and determining an enhanced speech signal corresponding to the noisy speech signal based on the coded speech feature, the noise reduction speech feature and the decoding network. The embodiment of the present invention solves the problem that the frequency domain speech enhancement method requires a time domain transformation by directly inputting the noisy speech signal into the target speech enhancement model, which not only shortens the calculation time of speech enhancement, but also improves the effect of speech enhancement.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of speech processing technology, and in particular to a speech enhancement method, apparatus, device and storage medium. Background Art

[0002] Speech enhancement is essentially speech denoising, that is, extracting the purest original speech from the noisy speech signal. Traditional speech enhancement methods make certain assumptions about the characteristics of pure speech and noisy speech and the relationship between them. This assumption limits the denoising effect of speech enhancement and has poor applicability. Speech enhancement methods based on deep learning do not require any assumptions. With the help of a neural network structure, they directly learn the nonlinear mapping relationship between pure speech and noisy speech, and have a significant noise suppression effect.

[0003] Most deep learning-based speech enhancement methods use short-time Fourier transform to transform noisy speech into the frequency domain for speech enhancement, and then use inverse short-time Fourier transform to transform into the time domain for waveform synthesis to enhance speech. Existing speech enhancement models are more suitable for speech enhancement of noisy speech in the frequency domain. On the one hand, speech enhancement in the frequency domain ignores the phase information of the speech signal, which will reduce the speech enhancement effect. On the other hand, existing methods involve the time-frequency transformation process, which increases the calculation time of the speech enhancement process. Operations such as framing and windowing in the time-frequency transformation process will also affect the final speech enhancement effect. Summary of the invention

[0004] The embodiments of the present invention provide a speech enhancement method, apparatus, device and storage medium to shorten the calculation time of the speech enhancement process and improve the speech enhancement effect.

[0005] In a first aspect, an embodiment of the present invention provides a method for speech enhancement, the method comprising:

[0006] Inputting the noisy speech signal to be processed into a target speech enhancement model; wherein the target speech enhancement model comprises an encoding network, a noise reduction network and a decoding network;

[0007] Performing encoding operation on the noisy speech signal through the encoding network to obtain output encoded speech features;

[0008] Inputting the encoded speech features into the denoising network to obtain output denoised speech features;

[0009] Based on the encoded speech features, the noise reduction speech features and the decoding network, an enhanced speech signal corresponding to the noisy speech signal is determined.

[0010] In a second aspect, an embodiment of the present invention further provides a speech enhancement device, the device comprising:

[0011] A noisy speech signal input module, used to input the noisy speech signal to be processed into a target speech enhancement model; wherein the target speech enhancement model includes an encoding network, a noise reduction network and a decoding network;

[0012] A speech coding module, used for performing a coding operation on the noisy speech signal through the coding network to obtain output coded speech features;

[0013] A speech denoising module, used for inputting the encoded speech features into the denoising network to obtain output denoised speech features;

[0014] A speech enhancement module is used to determine an enhanced speech signal corresponding to the noisy speech signal based on the encoded speech features, the noise reduction speech features and the decoding network.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] A memory for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned speech enhancement methods.

[0019] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions are used to execute any of the above-mentioned speech enhancement methods when executed by a computer processor.

[0020] The embodiment of the present invention directly inputs the noisy speech signal into the target speech enhancement model, and the target speech enhancement model enhances the noisy speech signal in the time domain through the encoding network, the noise reduction network and the decoding network, thereby solving the problem that the frequency domain speech enhancement method needs to perform time-frequency conversion. It not only shortens the calculation time of the speech enhancement process, but also retains the phase information of the speech signal as much as possible, thereby improving the effect of speech enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a speech enhancement method provided by Embodiment 1 of the present invention;

[0022] Figure 2A is a schematic diagram of a network structure of a coding network provided in Embodiment 1 of the present invention;

[0023] Figure 2B is a schematic diagram of a network structure of a local enhanced voltage transformation module provided in Embodiment 1 of the present invention;

[0024] Figure 2C is a schematic diagram of a network structure of a dense block module provided in Embodiment 1 of the present invention;

[0025] Figure 2D is a schematic diagram of a network structure of a noise reduction network provided in Embodiment 1 of the present invention;

[0026] Figure 2E is a schematic diagram of a network structure of a decoding network provided in Embodiment 1 of the present invention;

[0027] Figure 3 is a flow chart of a speech enhancement method provided by Embodiment 2 of the present invention;

[0028] Figure 4 is a schematic diagram of a speech enhancement device provided in Embodiment 3 of the present invention;

[0029] Figure 5 It is a structural schematic diagram of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0031] Embodiment 1

[0032] Figure 1 This is a flow chart of a speech enhancement method provided in the first embodiment of the present invention. This embodiment is applicable to the case of enhancing speech signals. The method can be executed by a speech enhancement device. The device can be implemented in software and / or hardware. The device can be configured in a terminal device. For example, the terminal device can be a mobile terminal, a laptop computer, a desktop computer, a server, a tablet computer, and other intelligent terminals. Specifically, the following steps are included:

[0033] S110: Input the noisy speech signal to be processed into a target speech enhancement model.

[0034] Among them, the speech signal is a mechanical wave signal, which is an information carrier of the wavelength and intensity changes of the mechanical wave. In this embodiment, the noisy speech signal contains a pure speech signal and a noise signal, wherein the noise signal is a random signal that interferes with the pure speech signal. Exemplarily, the types of noise signals include but are not limited to white noise, additive noise, Gaussian noise, and sharp noise. In this embodiment, the noisy speech signal may contain one type of noise signal or multiple types of noise signals. The type of noise signal in the noisy speech signal is not limited here.

[0035] In this embodiment, the target speech enhancement model includes an encoding network, a noise reduction network and a decoding network.

[0036] S120. Perform encoding operation on the noisy speech signal through the encoding network to obtain output encoded speech features.

[0037] In one embodiment, optionally, the encoding network includes a dimensional expansion module, a one-dimensional convolution module and an activation function; the dimensional expansion module is used to perform dimensional expansion processing on the noisy speech signal and output an expanded speech signal; the one-dimensional convolution module is used to perform a convolution operation on the expanded speech signal output by the dimensional expansion module and output a coded convolution speech feature.

[0038] Figure 2A Schematic diagram of a network structure of a coding network provided by the first embodiment of the present invention. B×L Indicated by, where B represents the number of samples input into the target speech enhancement model at one time, L represents the sample signal length, and the signal length of the noisy speech signal satisfies B×L. Specifically, the tensor of the noisy speech signal is represented by [B,L]. Figure 2A The unsqueeze in the dimensionality expansion module represents a dimensionality expansion module. Specifically, the dimensionality expansion module expands the dimension of the noisy speech signal, and the output expanded speech signal X' has a tensor size of [B, 1, L]. The formula of the dimensionality expansion module can be expressed as:

[0039] X'=unsqueeze(X)

[0040] The one-dimensional convolution module performs a one-dimensional convolution operation on the expanded speech signal X' output by the dimension expansion module, and outputs the coded convolution speech signal feature X", exemplarily, the number of input channels of the one-dimensional convolution module is 1, the number of output channels is 256, the convolution kernel size is 2, and the moving step of the convolution kernel is 1. Accordingly, the tensor size of the coded convolution speech signal feature X" is [B, 256, L-1]. The formula of the one-dimensional convolution module can be expressed as:

[0041] X' = ​​Conv1d(X')

[0042] The encoded convolution speech feature X" is passed through an activation function. For example, the activation function may be a relu activation function to obtain the encoded speech feature E, and the tensor size of E is [B, 256, L-1]. The formula is expressed as:

[0043] E = relu(X')

[0044] S130: Input the encoded speech features into the denoising network to obtain output denoised speech features.

[0045] In one embodiment, optionally, the noise reduction network includes a convolution module, a two-order transformation module, a dense block module and a deconvolution module; the convolution module is used to output convolution speech features based on the encoded speech features output by the encoding network; the two-order transformation module is used to extract features from the convolution speech features output by the convolution module based on the attention mechanism, and output transformed speech features; the dense block module is used to output dense speech features based on the transformed speech features output by the two-order transformation module; the deconvolution module is used to output denoised speech features based on the convolution speech features output by the convolution module and the dense speech features output by the dense block.

[0046] In one embodiment, optionally, the convolution module includes a preprocessing module, a first convolution module and a second convolution module; the preprocessing module is used to output preprocessed speech features based on the encoded speech features output by the encoding network; the first convolution module is used to perform convolution processing on the preprocessed speech features output by the preprocessing module, and output first convolution speech features; the second convolution module is used to perform convolution processing on the first convolution speech features output by the first convolution module, and output convolution speech features.

[0047] In one embodiment, optionally, the preprocessing module includes a group normalization module, a one-dimensional convolution module and a blocking module; the group normalization module is used to output group normalized speech features based on the encoded speech features output by the encoding network; the one-dimensional convolution module is used to output preprocessed convolution features based on the group normalized speech features output by the group normalization module; the blocking module is used to output preprocessed speech features based on the preprocessed convolution features output by the one-dimensional convolution module.

[0048] The encoded speech feature E passes through the group normalization module to obtain the group normalized speech feature S_GN, whose tensor size is [B, 256, L-1]. The formula of the group normalization module can be expressed as:

[0049] S_GN=GroupNorm(E)

[0050] The group normalized speech feature S_GN is passed through a one-dimensional convolution module to obtain a preprocessed convolution feature S_GN'. For example, the number of input channels of the one-dimensional convolution module is 256, the number of output channels is 64, the convolution kernel size is 2, and the moving step of the convolution kernel is 1. Correspondingly, the tensor size of the preprocessed convolution feature S_GN' is [B, 64, L-1]. The formula of the one-dimensional convolution module can be expressed as:

[0051] S_GN′=Conv1d(S_GN)

[0052] Among them, the preprocessed convolution feature S_GN' passes through the block module to obtain the preprocessed speech feature S_Seg. Exemplarily, the block module divides the preprocessed convolution feature S_GN' of length L-1 into S block sequence features of size K. If it cannot be divided into integer blocks, a zero padding operation is performed at the beginning and end of the preprocessed convolution feature S_GN'. Correspondingly, the tensor size of the preprocessed speech feature S_Seg is [B, 64, K, S].

[0053] Among them, the preprocessed speech feature S_Seg passes through the first convolution module to obtain the first convolution speech feature S_Conv2d_1. Specifically, the first convolution module performs two-dimensional convolution processing, batch normalization processing and activation function processing on the preprocessed speech feature S_Seg in sequence, and outputs the first convolution speech feature S_Conv2d_1. Exemplarily, the number of input channels of the first convolution module is 64, the number of output channels is 96, the convolution kernel size is 2, the moving step of the convolution kernel is 1, and padding=1. Padding means adding a unit-sized element with a value of 0 around the tensor of the preprocessed speech feature S_Seg. The purpose is to prevent the output first convolution speech feature S_Conv2d_1 from being too small. Correspondingly, the tensor size of the first convolution speech feature S_Conv2d_1 is [B, 96, K+1, S+1]. Among them, the formula of the first convolution module can be expressed as:

[0054] S_Conv2d_1=relu(BatchNorm2d(Conv2d(S_Seg)))

[0055] Among them, the first convolution speech feature S_Conv2d_1 passes through the second convolution module to obtain the convolution speech feature S_Conv2d_2. Specifically, the second convolution module performs two-dimensional convolution processing, batch normalization processing and activation function processing on the first convolution speech feature S_Conv2d_1 in sequence, and outputs the convolution speech feature S_Conv2d_2. Exemplarily, the number of input channels of the second convolution module is 96, the number of output channels is 128, the convolution kernel size is 2, the moving step of the convolution kernel is 1, and padding = 1. Correspondingly, the tensor size of the convolution speech feature S_Conv2d_2 is [B, 128, K+2, S+2]. The formula is expressed as:

[0056] S_Conv2d_2=relu(BatchNorm2d(Conv2d(S_Conv2d_1)))

[0057] In one embodiment, optionally, the two-stage transformation module includes a local transformation module and a global transformation module; the local transformation module is used to output local speech features based on the convolution speech features output by the convolution module; the global transformation module is used to output transformed speech features based on the local speech features output by the local transformation module. In one embodiment, optionally, the local transformation module includes a first local dimension transposition module, a local enhanced transformation module, a second local dimension transposition module and a local group normalization module; the global transformation module includes a first global dimension transposition module, a global enhanced transformation module, a second global dimension transposition module and a global group normalization module.

[0058] Take the local transformation module as an example. The convolution speech feature S_Conv2d_2 passes through the first local dimension transposition module in the local transformation module to obtain the first local transposed speech feature Local_Input, whose tensor size is [B, K+2, S+2, 128]. The formula of the first local dimension transposition module can be expressed as:

[0059] Local_Input=permute(S_Conv2d_2)

[0060] Among them, the first local transposed speech feature Local_Input is subjected to the local enhanced transformation module to obtain the local enhanced speech feature Local_LN2, whose tensor size is [B, K+2, S+2, 128].

[0061] Figure 2BIt is a schematic diagram of the network structure of a local enhancement transformation module provided in the first embodiment of the present invention. Specifically, the local enhancement transformation module includes a multi-head attention module (Multi-Head Attention), a first layer normalization module (Layer Norm), a feed-forward network module (Feed-forward Network) and a second layer normalization module. Among them, the first local transposed speech feature Local_Input is input into the multi-head attention module in the local enhancement transformation module to obtain the multi-head attention speech feature Mul_Head, whose tensor size is [B, K+2, S+2, 128]. Among them, the formula of the multi-head attention module can be expressed as:

[0062] Mul_Head=MultiheadAttention(Local_Input)

[0063] The advantage of this setting is that multi-head attention can be calculated from different dimensions, thus preventing the target speech enhancement model from overfitting.

[0064] Among them, the multi-head attention speech feature Mul_Head is added to the first local transposed speech feature Local_Input through a jump connection and input into the first layer normalization module to obtain the first layer normalized speech feature Local_LN, whose tensor size is [B, K+2, S+2, 128]. Among them, the formula of the first layer normalization module can be expressed as:

[0065] Local_LN=LayerNorm(Local_Input+Mul_Head)

[0066] The advantage of this setting is that the first-layer normalization module helps to ensure the stability of data feature distribution and accelerate the convergence speed of the model.

[0067] Among them, the first layer of normalized speech features Local_LN passes through the feedforward network module to obtain the feedforward speech features Local_FFN, whose tensor size is [B, K+2, S+2, 128]. Specifically, the feedforward network module includes a gated recurrent unit (GRU), an activation function (ReLU) and a linear layer (Linear). Among them, the gated recurrent unit can solve the problem of short-term memory and regulate the information flow. The activation function can enhance the expressiveness of the model. The linear layer plays a role in mapping the learned "distributed feature representation" to the sample label space.

[0068] Among them, the formula of the feedforward network module can be expressed as:

[0069] Local_FFN=Linear(Relu(GRU(Local_LN)))

[0070] Among them, the feedforward speech feature Local_FFN is added to the first-layer normalized speech feature Local_LN through a jump connection and input into the second-layer normalization module to obtain the local enhanced variable-pressure speech feature Local_LN2, whose tensor size is [B, K+2, S+2, 128]. Among them, the formula of the second-layer normalization module can be expressed as:

[0071] Local_LN2=LayerNorm(Local_FFN+Local_LN)

[0072] The advantage of this setting is that the second-layer normalization module helps to ensure the stability of data feature distribution and accelerate the convergence of the model.

[0073] Among them, the local enhanced transformed speech feature Local_LN2 is subjected to the second local dimension transposition module to obtain the second local transposed speech feature Local_LN2', whose tensor size is [B, 128, K+2, S+2]. The formula is expressed as:

[0074] Local_LN2'=permute(Local_LN2)

[0075] The second local transposed speech feature Local_LN2' is passed through the local group normalization module to obtain the local group speech feature, and the local group speech feature and the convolution speech feature S_Conv2d_2 are added through a jump connection to obtain the local speech feature Local_Out output by the local transformation module, whose tensor size is [B, 128, K+2, S+2]. The formula of the above steps can be expressed as:

[0076] Local_Out=GroupNorm(Local_LN2)+S_Conv2d_2

[0077] Among them, illustratively, the tensor size of the input data of the local group normalization module is [B, C, H, W]. The local group normalization module groups the data based on the number of channels C, normalizes the data in each group, and calculates the mean of (C / group)*H*W.

[0078] The advantage of this setting is that the local group normalization module helps to ensure the stability of data feature distribution and accelerate the convergence of the model.

[0079] Specifically, the implementation of the global transformation module is similar to that of the local transformation module, which will not be described in detail here. Finally, the double-stage transformation module outputs the transformed speech feature Global_Out, whose tensor size is [B, 128, K+2, S+2].

[0080] In one embodiment, optionally, the dense block module includes at least two two-dimensional convolution modules, the first two-dimensional convolution module outputs a first sub-dense speech feature based on the transformed speech feature output by the two-order transformation module; the Nth two-dimensional convolution module outputs a dense speech feature based on the transformed speech feature output by the two-order transformation module and the sub-dense speech features output by the first N-1 two-dimensional convolution modules; wherein N represents the number of two-dimensional convolution modules contained in the dense block module.

[0081] The number of two-dimensional convolution modules in the dense block module is not limited here. In one embodiment, optionally, the number of two-dimensional convolution modules in the dense block module is 5. Each two-dimensional convolution module performs two-dimensional convolution processing, group normalization processing and activation function processing on the input data in sequence.

[0082] Figure 2C Schematic diagram of a network structure of a dense block module provided in the first embodiment of the present invention. Specifically, the transformed speech feature Global_Out passes through the first two-dimensional convolution module in the dense block module to obtain the first sub-dense speech feature Dense_Out1. The formula of the first two-dimensional convolution module can be expressed as:

[0083] Dense_Out1=PReLU(GroupNorm(Conv2d(Global_Out)))

[0084] Among them, the i-th two-dimensional convolution module outputs the i-th sub-dense speech feature Dense_Out based on the variable pressure speech feature Local_Out and the sub-dense speech features output by the first i-1 two-dimensional convolution modules respectively. i Among them, the formula of the i-th two-dimensional convolution module can be expressed as:

[0085] Dense_Out i =PReLU(GroupNorm(Conv2d(Dense_Out i-1 +...+Global_Out)))

[0086] Among them, the Nth two-dimensional convolution module outputs the dense speech feature Dense_Out based on the variable pressure speech feature Local_Out and the sub-dense speech features output by the first N-1 two-dimensional convolution modules, and its tensor size is [B, 128, K+2, S+2]. The formula is expressed as:

[0087] Dense_Out=PReLU(GroupNorm(Conv2d(Dense_Out N-1 +...+Global_Out)))

[0088] Specifically, the number of input channels of the i-th two-dimensional convolution module is in_channels=128*(i), the number of output channels is 128, the convolution kernel size is (2,3), and the number of spaces between the convolution kernels is (2 i-1 ,1).

[0089] The advantage of this setting is that, on the one hand, the dense block module can avoid the problem of gradient disappearance, and on the other hand, the use of the dense block module can improve the parameter utilization of the target speech enhancement model.

[0090] In one embodiment, optionally, the deconvolution module includes a first merging module, a first deconvolution module, a second merging module and a second deconvolution module; the first merging module is used to perform a merging operation on the convolution speech features output by the second convolution module and the dense speech features output by the dense block module and output them to the first deconvolution module; the second merging module is used to perform a merging operation on the first convolution speech features output by the first convolution module and the first deconvolution speech features output by the first deconvolution module and output them to the second deconvolution module.

[0091] Specifically, the first merging module merges the convolution speech feature S_Conv2d_2 and the second dimension of the dense speech feature Dense_Out to obtain the first merged speech feature Cat1, whose tensor size is [B, 256, K+2, S+2]. The formula of the first merging module can be expressed as:

[0092] Cat1=cat((Dense_Out,S_Conv2d_2),1)

[0093] Among them, the first merged speech feature Cat1 passes through the first deconvolution module to obtain the first deconvolution speech feature S_ConvT_1. Specifically, the first deconvolution module performs two-dimensional deconvolution processing, batch normalization processing and activation function processing on the first merged speech feature Cat1 in sequence. Exemplarily, the number of input channels of the first deconvolution module is 256, the number of output channels is 96, the convolution kernel size is 2, and padding = 1. Correspondingly, the tensor size of the first deconvolution speech feature S_ConvT_1 is [B, 96, K+1, S+1]. Among them, the formula of the first deconvolution module can be expressed as:

[0094] S_ConvT_1=relu(BatchNorm2d(ConvTranspose2d(Cat1)))

[0095] The second merging module merges the second dimension of the first convolution speech feature S_Conv2d_1 and the first deconvolution speech feature S_ConvT_1 to obtain the second merged speech feature Cat2, whose tensor size is [B, 192, K+1, S+1]. The formula of the second merging module can be expressed as:

[0096] Cat2=cat((S_Conv2d_1,S_ConvT_1),1)

[0097] Among them, the second merged speech feature Cat2 passes through the second deconvolution module to obtain the second deconvolution speech feature S_ConvT_2. Specifically, the second deconvolution module performs two-dimensional deconvolution processing, batch normalization processing and activation function processing on the second merged speech feature Cat2 in sequence. Exemplarily, the number of input channels of the second deconvolution module is 192, the number of output channels is 34, the convolution kernel size is 2, and padding = 1. Correspondingly, the tensor size of the second deconvolution speech feature S_ConvT_2 is [B, 64, K, S]. Among them, the formula of the second deconvolution module can be expressed as:

[0098] S_ConvT_2=relu(BatchNorm2d(ConvTranspose2d(Cat2)))

[0099] In one embodiment, optionally, the deconvolution module also includes a superposition module and a one-dimensional convolution module. Specifically, the second deconvolution speech feature S_ConvT_2 passes through the superposition module to obtain the superposition speech feature Overlap_Out, whose tensor size is [B, 64, L-1]. Among them, the processing process of the superposition module is completely opposite to the processing process of the block module. The superposition speech feature Overlap_Out passes through the one-dimensional convolution module to obtain the noise reduction speech feature Denoise_Out. Exemplarily, the number of input channels of the one-dimensional convolution module is 64, the number of output channels is 256, and the convolution kernel size is 1. Correspondingly, the tensor size of the noise reduction speech feature Denoise_Out is [B, 256, L-1]. Among them, the formula of the one-dimensional convolution module can be expressed as:

[0100] Denoise_Out=Conv1d(Overlap_Out)

[0101] Figure 2D1 is a schematic diagram of a network structure of a noise reduction network provided in Embodiment 1 of the present invention. Specifically, the noise reduction network includes a convolution module, a two-order transformer module, a dense block module and a deconvolution module. Among them, the convolution module includes a group normalization module, a one-dimensional convolution module and a block module, and the deconvolution module includes a first merging module, a first deconvolution module, a second merging module, a second deconvolution module, a superposition module and a one-dimensional convolution module.

[0102] Specifically, the encoded speech feature E is input into the group normalization module in the convolution module to obtain the group normalized speech feature S_GN, the group normalized speech feature S_GN is input into the one-dimensional convolution module in the convolution module to obtain the preprocessed convolution feature S_GN', the preprocessed convolution feature S_GN' is input into the blocking module in the convolution module to obtain the preprocessed speech feature S_Seg, the preprocessed speech feature S_Seg is input into the first convolution module to obtain the first convolution speech feature S_Conv2d_1, the first convolution speech feature S_Conv2d_1 is input into the second convolution module to obtain the convolution speech feature S_Conv2d_2.

[0103] Specifically, the convolution speech feature S_Conv2d_2 is input into the two-stage transformer module to obtain the transformer speech feature Global_Out, and the transformer speech feature Global_Out is input into the dense block module to obtain the dense speech feature Dense_Out.

[0104] Specifically, the convolution speech feature S_Conv2d_2 and the dense speech feature Dense_Out are input into the first merging module in the deconvolution module to obtain the first merged speech feature Cat1, the first merged speech feature Cat1 is input into the first deconvolution module in the deconvolution module to obtain the first deconvolution speech feature S_ConvT_1, the first convolution speech feature S_Conv2d_1 and the first deconvolution speech feature S_ConvT1 are input into the second merging module in the deconvolution module to obtain the second merged speech feature Cat2, the second merged speech feature Cat2 is input into the second deconvolution module in the deconvolution module to obtain the second deconvolution speech feature S_ConvT_2, the second deconvolution speech feature S_ConvT_2 is input into the superposition module in the deconvolution module to obtain the superposition speech feature Overlap_Out, the superposition speech feature Overlap_Out is input into the one-dimensional convolution module in the deconvolution module to obtain the noise reduction speech feature Denoise_Out.

[0105] S140: Determine an enhanced speech signal corresponding to the noisy speech signal based on the encoded speech features, the noise reduction speech features and the decoding network.

[0106] Specifically, the encoded speech feature E and the noise reduction speech feature Denoise_Out are multiplied and input into the decoding network to obtain an enhanced speech signal corresponding to the noisy speech signal output by the decoding network. In one embodiment, optionally, the decoding network includes a one-dimensional deconvolution module, an activation function and a dimensional compression module.

[0107] Figure 2E Schematic diagram of a network structure of a decoding network provided in Embodiment 1 of the present invention. Specifically, the multiplication result of the encoded speech feature E and the noise reduction speech feature Denoise_Out passes through a one-dimensional deconvolution module to obtain a decoded deconvolution speech feature D_ConvT. Exemplarily, the number of input channels of the one-dimensional deconvolution module is 256, the number of output channels is 1, the convolution kernel size is 2, and the convolution kernel step size is 1. Correspondingly, the tensor size of the decoded deconvolution speech feature D_ConvT is [B, 1, L]. The formula is expressed as:

[0108] D_ConvT=ConvTranspose1d(E*Denoise_Out)

[0109] Among them, the decoded deconvolution speech feature D_ConvT is activated through the activation function to obtain the activated speech feature D_Relu, whose tensor size is [B, 1, L]. The formula is expressed as:

[0110] D_Relu=relu(D_ConvT)

[0111] Among them, squeeze represents the dimension compression module, and the activated speech feature D_Relu passes through the dimension compression module to obtain the enhanced speech signal Estimate_Out, whose tensor size is [B,L]. The formula is expressed as:

[0112] Estimate_Out=squeeze(D_Relu,dim=1)

[0113] The technical solution of this embodiment directly inputs the noisy speech signal into the target speech enhancement model. The target speech enhancement model enhances the noisy speech signal in the time domain through the encoding network, the noise reduction network and the decoding network, thereby solving the problem that the frequency domain speech enhancement method needs to perform time-frequency conversion. It not only shortens the calculation time of the speech enhancement process, but also retains the phase information of the speech signal as much as possible, thereby improving the effect of speech enhancement.

[0114] Embodiment 2

[0115] Figure 3It is a flow chart of a speech enhancement method provided in the second embodiment of the present invention. The technical solution of this embodiment is a further refinement based on the above embodiment. Optionally, the training method of the target speech enhancement model includes: inputting the acquired training speech signal into the initial speech enhancement model to be trained to obtain an output predicted speech signal; determining a loss function based on the predicted speech signal and a standard speech signal corresponding to the training speech signal; based on the loss function, updating the model parameters of the initial speech enhancement module until the loss function converges to obtain a trained target speech enhancement model.

[0116] The specific implementation steps of this embodiment include:

[0117] S210: Input the acquired training speech signal into the initial speech enhancement model to be trained to obtain an output predicted speech signal.

[0118] Specifically, the training speech signal may be obtained from a speech data set. Exemplarily, the speech data set includes but is not limited to a VoiceBank data set and a CommonVoice data set.

[0119] Taking the VoiceBank dataset as an example, 30 speakers are selected from the VoiceBank dataset, each of which contains 400 sentences and lasts 4.3 hours. The speech signals of 28 speakers are selected as clean speech signals for training, and the speech signals of 2 speakers are selected as clean speech signals for testing.

[0120] Randomly select 10 types of noise from the DEMAND noise database, synthesize the noise with the above-mentioned pure speech signal for training, and obtain the training speech signal. The signal-to-noise ratio of the training speech signal is randomly selected from 0dB, 5dB, 10dB and 15dB to fit the noise in the real scene. Using the same method, synthesize the noise with the above-mentioned pure speech signal for testing to obtain the test speech signal. The signal-to-noise ratio of the test speech signal is randomly selected from 2.5dB, 7.5dB, 12.5dB and 17.5dB.

[0121] It should be noted that the above parameter value settings are exemplary descriptions and are not intended to be limiting. Users can set parameter values ​​according to actual needs.

[0122] S220: Determine a loss function based on the predicted speech signal and a standard speech signal corresponding to the training speech signal.

[0123] Specifically, the standard speech signal is a pure speech signal used to synthesize the training speech signal, that is, a speech signal selected from a speech data set.

[0124] Exemplarily, the loss function may be a scale invariant signal-to-noise ratio (SI-SNR). The loss function SI-SNR satisfies the formula:

[0125]

[0126]

[0127] in, represents the predicted speech signal, S represents the standard speech signal, ||S|| 2 Indicates the power of a standard speech signal.

[0128] S230. Based on the loss function, the model parameters of the initial speech enhancement module are updated until the loss function converges to obtain a trained target speech enhancement model.

[0129] Based on the above embodiment, optionally, before training the initial speech enhancement model, hyperparameter configuration data is obtained, and the initial speech enhancement model is trained based on the hyperparameter configuration data.

[0130] Specifically, the hyperparameter configuration data includes but is not limited to at least one of the optimizer (Optimizer), model iteration number (epoch), early termination parameter (early_stop), learning rate (learning_rate), minimum learning rate (min_learning_rate), tolerance (patience), factor (factor), weight decay parameter (weight_decay), clipping ratio (clip_norm), discard parameter (dropout), normalization parameter (norm), hidden channel number (hidden_channels), input channel number (in_channels) and output channel number (out_channels).

[0131] Among them, Adam can be selected as the optimizer to speed up the training of the initial speech enhancement model; the number of model iterations can be set to 30 times; the early termination parameter can be set to 10, which means that the training will be stopped if the error of the initial speech enhancement model has not decreased within 10 epochs to solve the problem of model overfitting; the learning rate represents the update amplitude of each model parameter update. If the learning rate is too large, the model parameters to be learned may fluctuate around the minimum value. If the learning rate is too small, the convergence speed of the model parameters will be too slow. For example, the learning rate can be set to 5e -4 ; The minimum learning rate represents the minimum update amplitude each time the model is updated. For example, the minimum learning rate can be set to 1e-8 ; Tolerance means that when the early termination parameter is activated, if the model error has not decreased compared with the previous epoch training, the training will be stopped after patience epochs. For example, the tolerance can be set to 2; Factor means the coefficient multiplied when the learning rate changes. For example, the factor can be set to 0.5; The weight decay parameter can be used to prevent overfitting of model training. The default value is 0; The purpose of the interception ratio is to limit the parameter weights updated by back propagation to a suitable range. For example, the interception ratio can be set to 5; Dropout means that during the training process of the deep learning network, for the neural network unit, it is temporarily discarded from the network according to a certain probability, in order to prevent the model from overfitting. The default value 0 is used here, which means no discarding. The normalization parameter is to speed up the convergence of the model. Here, the value LayerNorm represents layer normalization; The number of hidden channels represents the number of channels used in the intermediate processing of the initial speech enhancement model; The number of input channels represents the number of channels of the input tensor of the initial speech enhancement model; The number of output channels represents the number of channels output by the initial speech enhancement model.

[0132] Based on the above embodiments, the method may optionally further include: obtaining a test speech signal, inputting the test speech signal into a trained target speech enhancement model, and determining a test index parameter based on an output result and a pure speech signal corresponding to the test speech signal; if the test index parameter does not meet a preset index range, correcting the hyperparameter configuration data, and continuing to train the initial speech enhancement model based on the corrected hyperparameter configuration data until a trained target speech enhancement model is obtained.

[0133] Wherein, illustratively, the test indicator parameter may be a scale-invariant signal-to-noise ratio.

[0134] S240: Input the noisy speech signal to be processed into a target speech enhancement model.

[0135] S250. Perform encoding operation on the noisy speech signal through the encoding network to obtain output encoded speech features.

[0136] S260: Input the encoded speech features into the noise reduction network to obtain output noise reduction speech features.

[0137] S270: Determine an enhanced speech signal corresponding to the noisy speech signal based on the encoded speech features, the noise reduction speech features and the decoding network.

[0138] The technical solution of this embodiment, by setting a double-order transformer module in the target speech enhancement model, extracts local and global features in the training speech signal, so that the target speech enhancement model learns the nonlinear relationship between the noise signal and the pure speech signal, solves the problem of the loss of signal features in the existing speech enhancement method, and thus improves the speech enhancement effect. Furthermore, this embodiment solves the problem of poor training effect of the initial speech enhancement model by training the initial speech enhancement model based on the training speech signal and the test speech signal, improves the generalization ability of the trained target speech enhancement model, and further improves the speech enhancement effect.

[0139] Embodiment 3

[0140] Figure 4 Schematic diagram of a speech enhancement device provided in Embodiment 3 of the present invention. This embodiment is applicable to the case of enhancing speech signals. The device can be implemented in software and / or hardware, and can be configured in a terminal device. The speech enhancement device includes: a noisy speech signal input module 310, a speech encoding module 320, a speech noise reduction module 330, and a speech enhancement module 340.

[0141] The noisy speech signal input module 310 is used to input the noisy speech signal to be processed into the target speech enhancement model; wherein the target speech enhancement model includes an encoding network, a noise reduction network and a decoding network;

[0142] The speech coding module 320 is used to perform a coding operation on the noisy speech signal through a coding network to obtain output coded speech features;

[0143] A speech denoising module 330, for inputting the encoded speech features into a denoising network to obtain output denoised speech features;

[0144] The speech enhancement module 340 is used to determine an enhanced speech signal corresponding to the noisy speech signal based on the encoded speech features, the noise reduction speech features and the decoding network.

[0145] The technical solution of this embodiment directly inputs the noisy speech signal into the target speech enhancement model. The target speech enhancement model enhances the noisy speech signal in the time domain through the encoding network, the noise reduction network and the decoding network, thereby solving the problem that the frequency domain speech enhancement method needs to perform time-frequency conversion. It not only shortens the calculation time of the speech enhancement process, but also retains the phase information of the speech signal as much as possible, thereby improving the effect of speech enhancement.

[0146] On the basis of the above technical solution, optionally, the noise reduction network includes a convolution module, a two-order transformation module, a dense block module and a deconvolution module; the convolution module is used to output convolution speech features based on the encoded speech features output by the encoding network; the two-order transformation module is used to extract features of the convolution speech features output by the convolution module based on the attention mechanism, and output the transformed speech features; the dense block module is used to output dense speech features based on the transformed speech features output by the two-order transformation module; the deconvolution module is used to output denoised speech features based on the convolution speech features output by the convolution module and the dense speech features output by the dense block.

[0147] Based on the above technical solution, optionally, the two-order transformation module includes a local transformation module and a global transformation module; the local transformation module is used to output local speech features based on the convolution speech features output by the convolution module; the global transformation module is used to output transformed speech features based on the local speech features output by the local transformation module.

[0148] On the basis of the above technical solution, optionally, the dense block module includes at least two two-dimensional convolution modules, the first two-dimensional convolution module outputs a first sub-dense speech feature based on the transformed speech feature output by the two-order transformation module; the Nth two-dimensional convolution module outputs a dense speech feature based on the transformed speech feature output by the two-order transformation module and the sub-dense speech features output by the first N-1 two-dimensional convolution modules; wherein N represents the number of two-dimensional convolution modules contained in the dense block module.

[0149] On the basis of the above technical solution, optionally, the convolution module includes a preprocessing module, a first convolution module and a second convolution module; the preprocessing module is used to output preprocessed speech features based on the encoded speech features output by the encoding network; the first convolution module is used to perform convolution processing on the preprocessed speech features output by the preprocessing module, and output first convolution speech features; the second convolution module is used to perform convolution processing on the first convolution speech features output by the first convolution module, and output convolution speech features.

[0150] On the basis of the above technical solution, optionally, the deconvolution module includes a first merging module, a first deconvolution module, a second merging module and a second deconvolution module; the first merging module is used to perform a merging operation on the convolution speech features output by the second convolution module and the dense speech features output by the dense block module and output them to the first deconvolution module; the second merging module is used to perform a merging operation on the first convolution speech features output by the first convolution module and the first deconvolution speech features output by the first deconvolution module and output them to the second deconvolution module.

[0151] On the basis of the above technical solution, optionally, the device further includes a training module of a target speech enhancement model, which is used to:

[0152] Inputting the acquired training speech signal into the initial speech enhancement model to be trained to obtain an output predicted speech signal;

[0153] Determining a loss function based on the predicted speech signal and a standard speech signal corresponding to the training speech signal;

[0154] Based on the loss function, the model parameters of the initial speech enhancement module are updated until the loss function converges to obtain the trained target speech enhancement model.

[0155] The speech enhancement device provided in the embodiment of the present invention can be used to execute the speech enhancement method provided in the embodiment of the present invention, and has the corresponding functions and beneficial effects of the execution method.

[0156] It is worth noting that in the embodiment of the above-mentioned speech enhancement device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0157] Embodiment 4

[0158] Figure 5 It is a structural diagram of an electronic device provided by Embodiment 4 of the present invention. The embodiment of the present invention provides services for the implementation of the speech enhancement method of the above embodiment of the present invention, and can configure the speech enhancement device in the above embodiment. Figure 5 A block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 5 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0159] like Figure 5 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0160] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0161] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0162] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0163] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0164] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and / or any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 20. Figure 5As shown, the network adapter 20 communicates with other modules of the electronic device 12 via the bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0165] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the speech enhancement method provided in the embodiment of the present invention.

[0166] The above electronic device solves the problem that the frequency domain speech enhancement method needs to perform time-frequency conversion, which not only shortens the calculation time of the speech enhancement process, but also retains the phase information of the speech signal as much as possible, thereby improving the effect of speech enhancement.

[0167] Embodiment 5

[0168] Embodiment 5 of the present invention further provides a storage medium including computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to execute a speech enhancement method, the method comprising:

[0169] Inputting the noisy speech signal to be processed into a target speech enhancement model; wherein the target speech enhancement model includes an encoding network, a noise reduction network and a decoding network;

[0170] Through the encoding network, the noisy speech signal is encoded to obtain the output encoded speech features;

[0171] Input the encoded speech features into the denoising network to obtain output denoised speech features;

[0172] Based on the encoded speech features, the denoised speech features and the decoding network, an enhanced speech signal corresponding to the noisy speech signal is determined.

[0173] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0174] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0175] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0176] Computer program code for performing operations of the invention may be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and also conventional procedural programming languages such as "C" language or similar programming languages. The program code may be executed entirely on the user computer, partly on the user computer, as a standalone software package, partly on the user computer, partly on the remote computer, or entirely on the remote computer or server. In cases involving remote computers, the remote computer may be connected to a user computer through any kind of network including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider to connect over the Internet).

[0177] Of course, the computer executable instructions of a storage medium including computer executable instructions provided in an embodiment of the present invention are not limited to the above method operations, and can also execute related operations in the speech enhancement method provided in any embodiment of the present invention.

[0178] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A speech enhancement method, characterized in that: include: Inputting the noisy speech signal to be processed into a target speech enhancement model; wherein the target speech enhancement model comprises an encoding network, a noise reduction network and a decoding network; Performing encoding operation on the noisy speech signal through the encoding network to obtain output encoded speech features; Inputting the encoded speech features into the denoising network to obtain output denoised speech features; Determining an enhanced speech signal corresponding to the noisy speech signal based on the encoded speech features, the denoised speech features and the decoding network; Among them, the noise reduction network includes a convolution module, a two-order transformation module, a dense block module and a deconvolution module; the convolution module is used to output convolution speech features based on the encoded speech features output by the encoding network; the two-order transformation module is used to extract features of the convolution speech features output by the convolution module based on the attention mechanism, and output the transformed speech features; the dense block module is used to output dense speech features based on the transformed speech features output by the two-order transformation module; the deconvolution module is used to output denoised speech features based on the convolution speech features output by the convolution module and the dense speech features output by the dense block.

2. The method according to claim 1, characterized in that The two-stage transformation module includes a local transformation module and a global transformation module; the local transformation module is used to output local speech features based on the convolution speech features output by the convolution module; the global transformation module is used to output transformed speech features based on the local speech features output by the local transformation module.

3. The method according to claim 1, characterized in that The dense block module includes at least two two-dimensional convolution modules, the first two-dimensional convolution module outputs a first sub-dense speech feature based on the transformed speech feature output by the two-order transformation module; the Nth two-dimensional convolution module outputs a dense speech feature based on the transformed speech feature output by the two-order transformation module and the sub-dense speech features output by the first N-1 two-dimensional convolution modules; wherein N represents the number of two-dimensional convolution modules contained in the dense block module.

4. The method according to claim 1, characterized in that: The convolution module includes a preprocessing module, a first convolution module and a second convolution module; the preprocessing module is used to output preprocessed speech features based on the encoded speech features output by the encoding network; the first convolution module is used to perform convolution processing on the preprocessed speech features output by the preprocessing module and output first convolution speech features; the second convolution module is used to perform convolution processing on the first convolution speech features output by the first convolution module and output convolution speech features.

5. The method according to claim 4, characterized in that The deconvolution module includes a first merging module, a first deconvolution module, a second merging module and a second deconvolution module; the first merging module is used to perform a merging operation on the convolution speech features output by the second convolution module and the dense speech features output by the dense block module and output them to the first deconvolution module; the second merging module is used to perform a merging operation on the first convolution speech features output by the first convolution module and the first deconvolution speech features output by the first deconvolution module and output them to the second deconvolution module.

6. The method according to any one of claims 1 to 5, characterized in that: The training method of the target speech enhancement model includes: Inputting the acquired training speech signal into the initial speech enhancement model to be trained to obtain an output predicted speech signal; Determining a loss function based on the predicted speech signal and a standard speech signal corresponding to the training speech signal; Based on the loss function, the model parameters of the initial speech enhancement module are updated until the loss function converges, thereby obtaining a trained target speech enhancement model.

7. A speech enhancement device, characterized in that: include: A noisy speech signal input module, used to input the noisy speech signal to be processed into a target speech enhancement model; wherein the target speech enhancement model includes an encoding network, a noise reduction network and a decoding network; A speech coding module, used for performing a coding operation on the noisy speech signal through the coding network to obtain output coded speech features; A speech denoising module, used for inputting the encoded speech features into the denoising network to obtain output denoised speech features; A speech enhancement module, configured to determine an enhanced speech signal corresponding to the noisy speech signal based on the encoded speech features, the noise reduction speech features and the decoding network; Among them, the noise reduction network includes a convolution module, a two-order transformation module, a dense block module and a deconvolution module; the convolution module is used to output convolution speech features based on the encoded speech features output by the encoding network; the two-order transformation module is used to extract features of the convolution speech features output by the convolution module based on the attention mechanism, and output the transformed speech features; the dense block module is used to output dense speech features based on the transformed speech features output by the two-order transformation module; the deconvolution module is used to output denoised speech features based on the convolution speech features output by the convolution module and the dense speech features output by the dense block.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the speech enhancement method as described in any one of claims 1-6.

9. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to perform the speech enhancement method as described in any one of claims 1 to 6 when executed by a computer processor.

Images