Ultra-lightweight speech enhancement neural network system and method based on adaptive convolution

Abstract

The invention discloses an ultra-lightweight speech enhancement neural network system and method based on adaptive convolution. The system comprises an encoder; a frequency spectrum compression module, which is used for carrying out dynamic range and frequency dimension compression on a noisy speech frequency spectrum; the self-adaptive convolution attention module is used for carrying out frequency dimension downsampling and feature extraction analysis on the compressed feature map through a self-adaptive convolution layer and time channel attention, and then sending the feature map into an intensifier; the intensifier is a dual-path packet recurrent neural network and is used for modeling sub-band time features and intra-frame frequency features of the voice; the decoder comprises a self-adaptive transpose convolution attention module which is used for performing frequency dimension upsampling and feature reconstruction on the feature map output by the enhancer; and the frequency spectrum expansion module is used for expanding the frequency dimension of the feature map output by the self-adaptive transpose convolution attention module. According to the invention, high-performance speech enhancement can be realized with extremely low calculation complexity.

Description

Technical Field

[0001] The present invention belongs to the field of speech enhancement, and in particular relates to an ultra-lightweight speech enhancement neural network system based on adaptive convolution and a method thereof. Background Art

[0002] Speech enhancement is one of the core issues in audio signal processing. Its main purpose is to recover speech from signals polluted by environmental noise, interfering human voices, reverberation, and echo, and to improve speech quality and intelligibility. It has been widely used in mobile communications, teleconferencing, Bluetooth headsets, hearing aids, and speech recognition front-ends. In recent years, deep learning models have been widely used in speech enhancement. Through a data-driven approach, deep neural networks can directly learn the nonlinear mapping relationship from mixed speech to clean speech, and have achieved performance far exceeding that of traditional methods in many scenarios. However, the most advanced speech enhancement neural networks currently have problems with excessive memory and computing resource consumption, which makes them difficult to deploy on widely used end-side devices. In scenarios where computing resources are limited and real-time requirements are high, ultra-lightweight neural networks are required.

[0003] At present, ultra-lightweight speech enhancement neural networks have been proposed, such as the Grouped Temporal Convolutional Recurrent Network (GTCRN), which uses a grouping strategy, a sub-band feature extraction module, and a temporal recurrent attention mechanism to reduce model complexity and improve performance (Rong X, Sun T, Zhang X, et al. GTCRN: A Speech Enhancement Model Requiring Ultralow Computational Resources [C] / / ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024: 971-975.). However, although these models have extremely low computational complexity, there is still room for further optimization in terms of performance.

[0004] Typical designs that improve performance while maintaining low computational overhead include modern convolutional neural network designs (Liu Z, Mao H, Wu CY, et al. A convnet for the 2020s [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2022: 11976-11986.), dynamic convolution (Chen Y, Dai X, Liu M, et al. Dynamic convolution: Attention over convolution kernels [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2020: 11030-11039.), etc. However, these strategies have not been fully applied in ultra-lightweight speech enhancement models. Among them, dynamic convolution is an advanced convolutional neural network design in the field of computer vision. It uses the attention mechanism to aggregate image information and obtain a set of attention scores. The scores are used to weight several candidate convolution kernels to obtain a convolution kernel independent of each image, and the convolution kernel is used to convolve the image. However, in the field of speech enhancement, performing dynamic convolution on the entire spectrogram does not satisfy causality and cannot adapt to rapidly changing speech spectral features. Summary of the invention

[0005] In view of the above problems existing in the prior art, the present invention proposes an ultra-lightweight speech enhancement neural network system and method based on adaptive convolution, which can achieve high-performance speech enhancement with extremely low computational complexity.

[0006] The technical solution adopted by the system of the present invention is:

[0007] An ultra-lightweight speech enhancement neural network system based on adaptive convolution, the system includes an encoder, an enhancer and a decoder, wherein the encoder includes: a spectrum compression module for compressing the dynamic range and frequency dimension of the noisy speech spectrum; an adaptive convolution attention module for downsampling and feature extraction analysis of the compressed feature map in the frequency dimension through an adaptive convolution layer and time channel attention, and then sending the feature map to the enhancer;

[0008] The enhancer includes a dual-path grouped recurrent neural network to model the sub-band temporal features and intra-frame frequency features of speech;

[0009] The decoder includes: an adaptive transposed convolutional attention module, which is used to upsample and reconstruct the feature map output by the enhancer in the frequency dimension through an adaptive transposed convolutional layer and time channel attention; a spectrum expansion module, which is used to expand the frequency dimension of the feature map output by the adaptive transposed convolutional attention module.

[0010] The present invention also provides a method for utilizing the ultra-lightweight speech enhancement neural network system based on adaptive convolution, the specific steps of the method are:

[0011] In the encoding stage, short-time Fourier transform is first used to transform the input noisy signal into a spectrum, and the spectrum compression module compresses the input spectrum in terms of dynamic range and frequency dimensions; then a 2-layer adaptive convolutional attention module is used to downsample and extract features from the feature map in the frequency dimension, and the feature channels are mapped to a high-dimensional space; finally, a 3-layer adaptive convolutional attention module is used to further extract and analyze the feature map;

[0012] In the enhancement stage, the enhancer is used to analyze and model the sub-band temporal features and intra-frame frequency features of the feature map output by the encoder;

[0013] In the decoding stage, a 3-layer adaptive transposed convolutional attention module is first used to analyze and reconstruct the feature map; then a 2-layer adaptive transposed convolutional attention module is used to upsample the feature map in the frequency dimension and further reconstruct the features; finally, a spectrum expansion module is used to expand the feature map in the frequency dimension to obtain a spectrum mask with the same dimension as the input spectrum, and the spectrum mask is multiplied by the input noisy spectrum, and an inverse short-time Fourier transform is performed to obtain an enhanced speech signal.

[0014] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0015] (1) The present invention uses adaptive convolution to enable convolution to obtain stronger adaptive representation capabilities for frame-level local features;

[0016] (2) The present invention uses the joint attention of channels and convolution kernels to enhance the network's attention to key channels and key candidate convolution kernels;

[0017] (3) The present invention uses an organic combination of deep convolution and point-by-point convolution to improve the network's feature extraction and reconstruction capabilities while ensuring low computational complexity;

[0018] (4) The system of the present invention can achieve advanced speech enhancement performance with extremely low parameter and computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a structural diagram of a network system of the present invention;

[0020] Figure 2 This is a structural diagram of an adaptive convolutional attention module in an embodiment of the present invention;

[0021] Figure 3 This is a structural diagram of the adaptive transposed convolution attention module in an embodiment of the present invention;

[0022] Figure 4 This is a structural diagram of the temporal recurrent channel convolution kernel joint attention generation module in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The technical solution of the present invention is fully and clearly described below in conjunction with the accompanying drawings and embodiments.

[0024] The network system structure diagram of the present invention is as follows Figure 1 As shown, the system includes an encoder, an enhancer and a decoder, the encoder includes: a spectrum compression module, which is used to compress the dynamic range and frequency dimension of the noisy speech spectrum; an adaptive convolution attention module, which is used to downsample and extract features in the frequency dimension of the compressed feature map through an adaptive convolution layer and time channel attention, and then send the feature map to the enhancer; the enhancer includes a dual-path grouped recurrent neural network, which is used to model the sub-band time features and intra-frame frequency features of the speech; the decoder includes: an adaptive transposed convolution attention module, which is used to upsample and reconstruct the features in the frequency dimension of the feature map output by the enhancer through an adaptive transposed convolution layer and time channel attention; a spectrum expansion module, which is used to expand the frequency dimension of the feature map output by the adaptive transposed convolution attention module.

[0025] The main implementation method of the system of the present invention is:

[0026] In the encoding stage, short-time Fourier transform is first used to transform the input noisy signal into a spectrum, and the spectrum compression module compresses the input spectrum in terms of dynamic range and frequency dimensions. Then, a 2-layer adaptive convolutional attention module is used to downsample and extract features from the feature map in the frequency dimension, and the feature channels are mapped to a high-dimensional space at the same time. Finally, a 3-layer adaptive convolutional attention module is used to further extract and analyze the feature map.

[0027] In the enhancement stage, the enhancer is used to analyze and model the sub-band temporal features and intra-frame frequency features of the feature map output by the encoder;

[0028] In the decoding stage, a 3-layer adaptive transposed convolutional attention module is first used to analyze and reconstruct the feature map; then a 2-layer adaptive transposed convolutional attention module is used to upsample the feature map in the frequency dimension and further reconstruct the features; finally, a spectrum expansion module is used to expand the feature map in the frequency dimension to obtain a spectrum mask with the same dimension as the input spectrum, and the spectrum mask is multiplied by the input noisy spectrum, and an inverse short-time Fourier transform is performed to obtain an enhanced speech signal.

[0029] The spectrum compression module described in this embodiment includes dynamic range compression and frequency dimension compression. The dynamic range compression uses logarithmic and power operations to compress the spectrum values. The frequency dimension compression keeps the frequency points in the 0-2kHz band unchanged. A triangular filter bank based on an equivalent rectangular bandwidth frequency scale is used to compress the frequency points in the 2-8kHz band. Specifically, assuming that the single-frame input spectrum is X=X R +jX I , where X R is the real part of the spectrum, X I The spectrum compression module maps the input complex spectrum into three channels, Indicates that

[0030]

[0031] Among them, log 10 (·) represents the logarithmic operation, and ERB[·] represents the filtering operation based on the triangular filter bank of the equivalent rectangular bandwidth frequency scale. Finally, In each frequency dimension, one frequency point is moved toward the low frequency and high frequency directions respectively to form the feature maps of the other 6 channels. They are concatenated together in the channel dimension as the input of subsequent modules, which is used to expand the receptive field of the frequency dimension.

[0032] The spectrum expansion module described in this embodiment is implemented by using a transposed matrix of a triangular filter bank equivalent matrix based on an equivalent rectangular bandwidth frequency scale to interpolate the feature map in the frequency dimension.

[0033] The structure diagram of the adaptive convolutional attention module described in this embodiment is as follows: Figure 2As shown, it includes layer normalization with affine transformation, time cycle channel convolution kernel joint attention generation module, and sequentially connected input time channel attention, adaptive depth convolution layer, first batch normalization, first parameter rectified linear unit activation function, first adaptive point-by-point convolution layer, Gaussian error linear unit activation function, second adaptive point-by-point convolution layer, second batch normalization, second parameter rectified linear unit activation function and output time channel attention. Among them, the first two layers of adaptive convolution attention modules of the encoder have the function of frequency dimension downsampling, and the stride of the convolution kernel frequency dimension of its adaptive depth convolution layer is set to 2, while the stride of the following three layers of adaptive convolution attention modules is set to 1. The temporal recurrent channel convolution kernel joint attention generation module analyzes the input features to obtain the input temporal channel attention map, the output temporal channel attention map, the frame-level convolution kernel attention map of the adaptive deep convolution layer, the frame-level convolution kernel attention map of the first adaptive point-by-point convolution layer, and the frame-level convolution kernel attention map of the second adaptive point-by-point convolution layer; the input and output temporal channel attentions multiply the feature maps with the input and output temporal channel attention maps respectively; each frame-level convolution kernel attention map is input into the corresponding adaptive deep convolution layer and adaptive point-by-point convolution layer. The adaptive deep convolution layer and the adaptive point-by-point convolution layer are deep convolution layers and point-by-point convolution layers implemented in an adaptive manner, wherein the deep convolution layer refers to a convolution layer whose grouping number is equal to the number of input and output channels, and the point-by-point convolution layer refers to a convolution layer whose convolution kernel size is 1×1. The adaptive deep convolution layer does not change the number of channels of the feature map, the first adaptive point-by-point convolution layer maps the number of channels of the feature map from the number of input channels to the number of hidden channels, and the second adaptive point-by-point convolution layer maps the number of channels of the feature map from the number of hidden channels to the number of output channels.

[0034] The structure diagram of the adaptive transposed convolution attention module described in this embodiment is as follows Figure 3 As shown in the figure, the difference between its structure and the adaptive convolution attention module lies only in the adaptive deep transposed convolution layer and the output activation function. The adaptive deep transposed convolution layer is a deep transposed convolution layer implemented in an adaptive manner, wherein the deep transposed convolution layer refers to a transposed convolution layer in which the number of groups is equal to the number of input and output channels. Among them, the adaptive transposed convolution attention modules after the decoder have the function of frequency dimension upsampling, and the stride of the convolution kernel frequency dimension of its adaptive deep transposed convolution layer is set to 2; the output activation function is expressed as a learnable sigmoid activation function in the last layer of the adaptive transposed convolution attention module, and as a parametric rectified linear unit activation function in other adaptive transposed convolution attention modules. Among them, the calculation formula of the learnable sigmoid activation function is

[0035] F out =2×sigmoid(W sigmoid *F in )

[0036] Among them, F in and F out They represent the input feature map and the output feature map, respectively, and their shapes are B×C×T×F, where B, C, T, and F represent the batch size, number of channels, time domain dimension, and frequency domain dimension, respectively; W sigmoid represents a learnable affine transformation weight with a shape of 1×1×1×F; * represents element-wise multiplication; sigmoid(·) represents a conventional sigmoid activation function.

[0037] The temporal cyclic channel convolution kernel joint attention generation module described in this embodiment is as follows Figure 4 As shown in the figure, it is composed of an energy average pooling layer, a recurrent neural network, a fully connected layer and an attention generation layer connected in sequence along the frequency dimension. Among them, the recurrent neural network performs time series analysis and modeling on the feature map of energy average pooling; the attention generation layer first divides the output features of the fully connected layer into 5 parts along the channel dimension. The first 2 parts use the sigmoid activation function to obtain the input and output time channel attention maps, and the last 3 parts use the softmax activation function with a temperature coefficient to obtain the frame-level convolution kernel attention map of the adaptive deep convolution layer, the first adaptive point-by-point convolution layer and the second adaptive point-by-point convolution layer. Among them, the softmax activation function with a temperature coefficient first divides the input features by a temperature coefficient during the training process of the neural network. The temperature coefficient changes with the training annealing to ensure the training optimization efficiency of all candidate convolution kernels; and during the test process, there is no need to divide by the temperature coefficient.

[0038] The adaptive depth convolution layer, adaptive depth transposed convolution layer, first adaptive point-by-point convolution layer, and second adaptive point-by-point convolution layer described in this embodiment are all adaptive convolution, which is a causal frame-by-frame dynamic convolution, and has the ability to adapt to the spectral features of each frame. The specific implementation method is that the adaptive depth convolution layer, adaptive depth transposed convolution layer, first adaptive point-by-point convolution layer, and second adaptive point-by-point convolution layer in different modules respectively receive the corresponding frame-level convolution kernel attention map input, and use it to weighted sum the candidate convolution kernels of the corresponding layer to obtain the frame-level adaptive convolution kernel W L (t), that is

[0039]

[0040] Among them, L represents the layer to which the adaptive convolution kernel belongs; W i L ,i=0,1,…K-1 represents the i-th candidate convolution kernel of this layer. The candidate convolution kernel is a set of fixed learnable parameters of this layer. There are K candidates in each layer. The shape of each candidate convolution kernel is the same as the shape of the convolution kernel of the corresponding conventional convolution layer. Represents the frame-level convolution kernel attention map of the i-th candidate convolution kernel input of this layer; then use the adaptive convolution kernel W i L (t) Perform convolution or transposed convolution operation on the feature map input to this layer.

[0041] The enhancer described in this embodiment includes two dual-path grouped recurrent neural networks connected in series, each of which is composed of an intra-frame grouped recurrent neural network for modeling intra-frame frequency characteristics of speech and an inter-frame grouped recurrent neural network for modeling sub-band time characteristics of speech, which are connected in sequence.

[0042] The specific settings of this embodiment are: the speech enhancement object is audio data with a sampling rate of 16kHz, the frame length and frame shift of the short-time Fourier transform are set to 512 (32ms) and 256 (16ms) respectively, the number of Fourier transform points is 512, and the window function is a square root Hanning window. The spectrum compression module keeps the 65 frequency bands within 0-2kHz unchanged and maps the 192 frequency bands within 2-8kHz into 64 frequency bands. The number of input channels, hidden channels, output channels, kernel size and stride of the first two layers of the encoder’s adaptive convolutional attention module are [9, 16, 16, (1, 5), (1, 2)] and [16, 16, 16, (1, 5), (1, 2)], respectively. The kernel size and stride correspond to the time dimension and frequency dimension, respectively. The corresponding hyperparameters of the adaptive convolutional attention module of the last three layers of the encoder and the adaptive transposed convolutional attention module of the first three layers of the decoder are [16, 16, 16, (3, 3), (1, 1)]; the corresponding hyperparameters of the adaptive transposed convolutional attention module of the last two layers of the decoder are [16, 16, 16, (1, 5), (1, 2)] and [16, 1, 4, (1, 5), (1, 2)]. The number of candidate convolutional kernels for all adaptive convolutional layers is set to 8. The number of hidden units of the recurrent neural network of all time-cycle channel convolution kernel joint attention generation modules is 32. The number of hidden units of the inter-frame grouped recurrent neural network of the two dual-path grouped recurrent neural network modules is 16, the number of hidden units of the intra-frame grouped recurrent neural network is 8, and the number of groups is 2. According to the above parameter settings, the number of parameters of the model is 135K, and the number of calculations per second is 38MMACs. During the training process, the temperature coefficient of the softmax activation function with a temperature coefficient gradually decays exponentially from 30 to 1, and the decay exponent is 0.9. The calculation formula is

[0043] τ(e)=30×0.9 e

[0044] Here, τ(e) is the temperature coefficient of the e-th training epoch.

[0045] In order to illustrate the effect of the present invention, the speech and noise data of the 5th Deep Noise Suppression Challenge (DNS5) dataset were used to synthesize a dataset of about 200 hours for experiments. The signal-to-noise ratio was set to -5 to 15 dB, and all noisy data and clean speech data were downsampled to 16 kHz. The GTCRN model (Rong X, Sun T, Zhang X, et al. GTCRN: ASpeech Enhancement Model Requiring Ultralow Computational Resources [C] / / ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024: 971-975.) was selected as the baseline model for comparison. The evaluation indicators of speech enhancement performance include scale-invariant signal-to-noise ratio (SISNR), perceptual evaluation of speech quality (PESQ), extended short-time objective intelligibility (ESTOI) and deep noise suppression mean opinion score (DNSMOS), including signal preservation quality SIG, noise suppression quality BAK and overall quality OVRL. The higher the better. The comparison results on the test set are shown in Table 1, where the computational complexity is measured by the number of multiply-accumulate operations per second (MACs).

[0046] Table 1 Comparison results

[0047]

[0048] The results in Table 1 prove that the computational complexity of the system of the present invention is comparable to that of GTCRN, while being able to achieve significantly better enhancement performance.

Claims

1. An ultra-lightweight speech enhancement neural network system based on adaptive convolution, the system comprising an encoder, an enhancer and a decoder, characterized in that: The encoder includes: a spectrum compression module, which is used to compress the dynamic range and frequency dimension of the noisy speech spectrum; an adaptive convolution attention module, which is used to perform frequency dimension downsampling and feature extraction analysis on the compressed feature map through an adaptive convolution layer and time channel attention, and then send the feature map to the enhancer; The enhancer includes a dual-path grouped recurrent neural network to model the sub-band temporal features and intra-frame frequency features of speech; The decoder includes: an adaptive transposed convolutional attention module, which is used to upsample and reconstruct the feature map output by the enhancer in the frequency dimension through an adaptive transposed convolutional layer and time channel attention; a spectrum expansion module, which is used to expand the frequency dimension of the feature map output by the adaptive transposed convolutional attention module.

2. The ultra-lightweight speech enhancement neural network system based on adaptive convolution according to claim 1, characterized in that: The adaptive convolutional attention module includes layer normalization with affine transformation, a time-recurring channel convolution kernel joint attention generation module, and sequentially connected input time channel attention, an adaptive depth convolution layer, a first batch normalization, a first parameter rectified linear unit activation function, a first adaptive point-by-point convolution layer, a Gaussian error linear unit activation function, a second adaptive point-by-point convolution layer, a second batch normalization, a second parameter rectified linear unit activation function and an output time channel attention.

3. The ultra-lightweight speech enhancement neural network system based on adaptive convolution according to claim 2, characterized in that: The temporal recurrent channel convolution kernel joint attention generation module is composed of an energy average pooling layer, a recurrent neural network, a fully connected layer and an attention generation layer connected in sequence along the frequency dimension; wherein the recurrent neural network performs time series analysis and modeling on the feature map of energy average pooling; the attention generation layer first divides the output features of the fully connected layer into 5 parts along the channel dimension, the first 2 parts use the sigmoid activation function to obtain the input and output time channel attention maps, and the last 3 parts use the softmax activation function with a temperature coefficient to obtain the frame-level convolution kernel attention maps of the adaptive deep convolution layer, the first adaptive point-by-point convolution layer and the second adaptive point-by-point convolution layer.

4. The ultra-lightweight speech enhancement neural network system based on adaptive convolution according to claim 3, characterized in that: During the training process of the neural network, the softmax activation function with a temperature coefficient first divides the input feature by a temperature coefficient, and the temperature coefficient changes with training annealing to ensure the training optimization efficiency of all candidate convolution kernels.

5. The ultra-lightweight speech enhancement neural network system based on adaptive convolution according to claim 1, characterized in that: The adaptive convolution layer and the adaptive transposed convolution layer respectively accept the corresponding frame-level convolution kernel attention map input, use the frame-level convolution kernel attention map to weighted sum the candidate convolution kernels, and obtain the frame-level adaptive convolution kernel, so that the convolution kernel obtains a stronger adaptive representation capability for the frame-level local features; and then use the adaptive convolution kernel to perform convolution operation or transposed convolution operation on the feature map.

6. The ultra-lightweight speech enhancement neural network system based on adaptive convolution according to claim 1, characterized in that: The adaptive transposed convolution attention module includes layer normalization with affine transformation, a time-recurring channel convolution kernel joint attention generation module, and sequentially connected input time channel attention, an adaptive depth transposed convolution layer, a first batch normalization, a parameter rectified linear unit activation function, a first adaptive point-by-point convolution layer, a Gaussian error linear unit activation function, a second adaptive point-by-point convolution layer, a second batch normalization, an output activation function and an output time channel attention.

7. The ultra-lightweight speech enhancement neural network system based on adaptive convolution according to claim 6, characterized in that: The output activation function is expressed as a learnable sigmoid activation function in the last layer and as a parameterized rectified linear unit activation function in other layers.

8. The ultra-lightweight speech enhancement neural network system based on adaptive convolution according to claim 1, characterized in that: The spectrum compression module includes dynamic range compression and frequency dimension compression, wherein the dynamic range compression uses logarithmic and power operations to compress the spectrum values; the frequency dimension compression keeps each frequency point in the 0-2kHz band unchanged, and uses a triangular filter group based on an equivalent rectangular bandwidth frequency scale to achieve compression for each frequency point in the 2-8kHz band.

9. The ultra-lightweight speech enhancement neural network system based on adaptive convolution according to claim 1, characterized in that: The enhancer includes two series-connected dual-path grouped recurrent neural networks, each of which is composed of an intra-frame grouped recurrent neural network for modeling intra-frame frequency characteristics of speech and an inter-frame grouped recurrent neural network for modeling sub-band time characteristics of speech, which are connected in sequence.

10. A method for utilizing the ultra-lightweight speech enhancement neural network system based on adaptive convolution as claimed in claim 1, characterized in that: The specific steps of this method are: In the encoding stage, short-time Fourier transform is first used to transform the input noisy signal into a spectrum, and the spectrum compression module compresses the input spectrum in terms of dynamic range and frequency dimensions; then a 2-layer adaptive convolutional attention module is used to downsample and extract features from the feature map in the frequency dimension, and the feature channels are mapped to a high-dimensional space; finally, a 3-layer adaptive convolutional attention module is used to further extract and analyze the feature map; In the enhancement stage, the enhancer is used to analyze and model the sub-band temporal features and intra-frame frequency features of the feature map output by the encoder; In the decoding stage, a 3-layer adaptive transposed convolutional attention module is first used to analyze and reconstruct the feature map; then a 2-layer adaptive transposed convolutional attention module is used to upsample the feature map in the frequency dimension and further reconstruct the features; finally, a spectrum expansion module is used to expand the feature map in the frequency dimension to obtain a spectrum mask with the same dimension as the input spectrum, and the spectrum mask is multiplied by the input noisy spectrum, and an inverse short-time Fourier transform is performed to obtain an enhanced speech signal.

Images