Multi-stage full-band speech enhancement method based on spectrum compression and neural network

Abstract

The present invention discloses a multi-stage full-band speech enhancement method based on spectrum compression and neural network. The steps are as follows: designing a full-band speech spectrum compression curve, initializing the spectrum compression module of MHA-DPCRN using the designed parameters; synthesizing simulated noisy reverberation speech; performing short-time Fourier transform on the simulated noisy reverberation speech and the corresponding pure speech with reverberation to obtain the short-time Fourier spectra of both; using the short-time Fourier spectrum to train the MHA-DPCRN model weights; (5) performing short-time Fourier transform on the noisy reverberation speech to be enhanced to obtain the short-time Fourier spectrum; inputting the short-time Fourier spectrum of the noisy reverberation speech into the trained model to obtain the short-time Fourier spectrum of the enhanced speech, and performing inverse short-time Fourier transform to obtain the time domain signal of the enhanced speech. The method of the present invention can perform relatively thorough noise suppression on full-band speech in complex noise scenarios while retaining good speech quality.

Description

Technical Field

[0001] The present invention belongs to the field of speech enhancement, and is mainly aimed at the noise suppression problem of full-band speech. Background Art

[0002] Based on the sampling rate, speech is generally categorized as narrowband (8kHz), wideband (16kHz), ultra-wideband (32kHz), and full-band (48kHz). In everyday speech collection, background noise often interferes, coming from sources such as washing machines, air conditioners, car horns, and construction sites. This interference can cause speech distortion, affecting both speech clarity and intelligibility, and negatively impacting subsequent speech processing tasks such as automatic speech recognition. In recent years, noise suppression models based on deep neural networks have been continuously proposed, achieving increasingly superior performance. Currently, most neural network models are primarily designed for wideband speech processing. This is because wideband speech already contains the majority of the energy and important harmonic information in human speech, effectively preserving the speaker's timbre and complete speech information. As people's requirements for voice quality continue to increase, full-band voice processing tasks are also receiving more attention. Compared with broadband voice, full-band voice retains more high-frequency voice information. Although these frequency bands do not contain key voice content components, they enhance the metallic feel of the voice, making the voice sound clearer and more transparent. This is also very important for audio-visual and communication scenarios with high-fidelity requirements. However, neural network models for processing wideband speech are often not directly applicable to processing full-band speech. This is due to several reasons. First, while maintaining the same spectral resolution, the dimensionality of the full-band speech spectrum is three times that of the wideband speech spectrum. Directly using the spectrum as model input would triple the computational complexity, which is clearly unacceptable for models with real-time processing requirements. Second, the energy, structural characteristics, and signal-to-noise ratio (SNR) of the high- and low-frequency bands of the full-band speech spectrum differ significantly. The low-frequency band (below 5kHz) contains most of the speech energy and harmonic structure, resulting in a relatively high SNR. The high-frequency band (5kHz-24kHz) is primarily composed of breathy sounds and voiceless consonants, which contain less speech energy and therefore generally have a lower SNR. Furthermore, the dimensionality of the high-frequency band is approximately four times that of the low-frequency band. When processing both bands in parallel, the model tends to focus on the high-frequency band, while the processing performance of the low-frequency band, which contains the main speech components, is reduced. To address these issues, researchers have recently proposed different solutions, driving the development of full-band speech enhancement models.

[0003] RNNoise (Valin J MA Hybrid DSP / Deep Learning Approach to Real-TimeFull-Band Speech Enhancement[C] / / 2018IEEE 20th International Workshop onMultimedia Signal Processing(MMSP).IEEE,2018) and PercepNet (Valin JM,Isik U,Phansalkar N,et al.A Perceptually-Motivated Approach for Low-Complexity,Real-Time Enhancement of Fullband Speech[J].Interspeech,pp.2482–2486,2020) both proposed using spectral envelope filtering for full-band speech enhancement. They used an equivalent rectangular bandwidth filter bank to compress the full-band speech amplitude spectrum dimension to 22 dimensions and 34 dimensions, used it as the neural network input and output the corresponding spectral envelope gain coefficient. This processing method greatly reduces the computational complexity of the model and is very beneficial to the real-time operation of the model. However, the loss of spectral resolution caused by compression will inevitably cause residual noise and even produce obvious artificial noise.

[0004] A sub-band processing method is proposed for the ultra-wideband speech enhancement model S-DCCRN (Lv S, Y Fu, Xing M, et al. S-DCCRN: SuperWide Band DCCRN with learnable complex feature for speech enhancement [J]. 2021, arXiv: 2111.0838). Since the low-frequency and high-frequency components of speech have obvious differences in characteristics, the speech spectrum can be split first, and two sub-band modules are used to process the low-frequency band spectrum and the high-frequency band spectrum respectively. Finally, a global module is used to integrate the two processed frequency bands to avoid artificial noise caused by the discontinuity of the spectrum connection. This model takes into account the differences between high and low frequency bands and performs block processing. The processing results are also significantly better than RNNoise and PercepNet. However, the model still needs to introduce a module at the end to smoothly integrate the results of the block processing, which additionally increases the computational complexity of the model.

[0005] In addition, a multi-stage progressive model processing framework has recently achieved outstanding results in the DeepNoiseSuppressionChallenge competition (Li, A., Liu, W., Luo, X., Yu, G., Zheng, C., Li, XA SimultaneousDenoising and Dereverberation Framework with Target Decoupling[J]. Interspeech, pp. 2801-2805, 2021). This method decomposes noise suppression into two stages: amplitude spectrum estimation and complex spectrum recovery. It guides the network to first estimate the short-time Fourier amplitude spectrum of the pure speech, improve the overall signal-to-noise ratio of the signal, and then the complex spectrum recovery network suppresses the residual noise components more thoroughly. Summary of the Invention

[0006] Taking into account the advantages and existing shortcomings of the above-mentioned models, the present invention combines the processing concepts of spectrum compression, block processing, and sub-band connection, specifically designs a spectrum compression curve, and integrates it into the deep neural network as a trainable module. This not only guides the direction of network learning, but also allows the network to adaptively adjust the compression coefficient during parameter training to achieve the optimal spectrum recovery effect. Inspired by the multi-stage progressive enhancement structure, the present invention also divides the method into two parts: amplitude spectrum enhancement and complex spectrum recovery. The amplitude spectrum enhancement process improves the overall signal-to-noise ratio of the speech, and the subsequent complex spectrum recovery process can more carefully restore the pure speech.

[0007] The technical solution adopted in the present invention is:

[0008] The multi-stage full-band speech enhancement method based on spectrum compression and neural network includes the following steps:

[0009] Step 1: designing a full-band speech spectrum compression curve and calculating compression module parameters according to the compression curve;

[0010] Step 2: Use the clear speech dataset, the noise dataset, and the room impulse response dataset to synthesize simulated noisy reverberant speech;

[0011] Step 3, performing short-time Fourier transform on the simulated noisy reverberation speech data and the corresponding pure reverberation speech data to obtain short-time Fourier spectra of the two;

[0012] Step 4: construct an MHA-DPCRN model, which is composed of a cascade of an amplitude spectrum enhancement network (MHAN) and a complex spectrum enhancement network (DPCRN); use the compression module parameters obtained in step 1 to initialize the spectrum compression module of the MHA-DPCRN; use the short-time Fourier spectrum obtained in step 3 to train the MHA-DPCRN model weights;

[0013] Step 5: Perform short-time Fourier transform on the noisy reverberation speech signal to be enhanced to obtain a short-time Fourier spectrum;

[0014] Step 6: Input the short-time Fourier spectrum obtained in step 5 into the MHA-DPCRN model trained in step 4, and output the short-time Fourier spectrum of the enhanced speech;

[0015] Step 7: Perform inverse short-time Fourier transform on the short-time Fourier spectrum of the enhanced speech to obtain a time domain signal of the enhanced speech.

[0016] Furthermore, in step 1, the spectrum compression curve keeps the frequency points in the 0kHz-5kHz band unchanged, and performs logarithmic smooth compression on the frequency points in the 5kHz-24kHz band. Let the original frequency band and the compressed frequency band be f and f respectively. cmp , the specific conversion formula is as follows:

[0017]

[0018] Furthermore, in step 4, the amplitude spectrum enhancement network MHAN is used to estimate the short-time Fourier amplitude spectrum of the noisy reverberant speech acted by the real mask to improve the speech signal-to-noise ratio, and the complex spectrum enhancement network DPCRN cascaded with it directly estimates the short-time Fourier complex spectrum of the reverberant clean speech to achieve a more thorough noise suppression effect; during the training process, the amplitude spectrum enhancement network MHAN is first trained separately, and then the converged amplitude spectrum enhancement network MHAN is cascaded with the complex spectrum enhancement network DPCRN and trained jointly.

[0019] Furthermore, in step 4, a linear layer for implementing spectrum compression is added before the encoder of the complex spectrum enhancement network DPCRN. Both this linear layer and the first linear layer of the amplitude spectrum enhancement network MHAN are initialized using the designed compression module parameters; the parameters of the linear layer for mapping high-frequency bands, i.e., above 5 kHz, are set to trainable, and the parameters for mapping low-frequency bands, i.e., below 5 kHz, are set to non-trainable.

[0020] Compared with the equivalent rectangular bandwidth filter bank compression method, the present invention improves the resolution of the compressed spectrum, especially in the low-frequency band, ensuring that the spectrum resolution remains unchanged. The purpose is to accurately restore the harmonic components of the low-frequency band and ensure that the main information of the speech is not lost. Compared with the molecular band processing method, the present invention ensures that the connection between the low-frequency band and the high-frequency band is smooth through the continuity of the compression curve, does not generate artificial noise, and does not need to add an additional integration module. Finally, the compression mapping matrix obtained from the compression curve is part of the model's trainable parameters, and it will converge to the optimal compression parameters as the training progresses. The spectrum compression method proposed by the present invention greatly reduces the computational complexity of the neural network when processing the full-band speech spectrum. At the same time, the mid- and low-frequency bands of the speech achieve better noise suppression and sound quality improvement effects. The two-stage progressive enhancement network structure also has stronger noise suppression capabilities under low signal-to-noise ratio conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of the method processing in the training stage and the enhancement stage of the present invention.

[0022] Figure 2 It is the function image of the full-band compression function.

[0023] Figure 3 This is a schematic diagram of spectrum compression achieved by compression transformation matrix, where the upper right is the compression matrix, the lower right is the short-time Fourier amplitude spectrum of the speech before compression, and the upper left is the short-time Fourier amplitude spectrum after compression.

[0024] Figure 4(a) shows the overall framework of the MHA-DPCRN model. MHAN refers to the multi-head attention mechanism network, DPCRN refers to the dual-channel convolutional recurrent neural network, and X refers to the short-time Fourier spectrum of the input noisy reverberant speech. refers to the estimated pure speech amplitude spectrum mask, Refers to the short-time Fourier spectrum after MHAN enhancement, Refers to the short-time Fourier spectrum after DPCRN enhancement.

[0025] Figure 4(b) shows the specific structure of the MHAN network. SCB refers to the spectrum compression module, ISCB refers to the inverse spectrum compression module, Masked multi-head self-attention refers to the causal multi-head self-attention mechanism module, Add&norm refers to the addition and normalization operations, Feed-forward refers to the linear layer, and B refers to the number of module repetitions within the dotted line.

[0026] Figure 4(c) shows the specific structure of the DPCRN network. Encoder refers to the encoder, DPRNN refers to the dual-channel recurrent neural network, Real part Decoder refers to the complex spectrum real part decoder, Imaginary part Decoder refers to the complex spectrum imaginary part decoder, Skip connection refers to the skip transfer operation, and couple refers to the integration of the real and imaginary parts into the complex spectrum.

[0027] Figure 5 Schematic diagram of the structure of the DPCRN encoder in an embodiment of the present invention.

[0028] Figure 6 Schematic diagram of the structure of the DPCRN dual-channel recurrent neural network in an embodiment of the present invention.

[0029] Figure 7 Schematic diagram of the structure of the DPCRN real (imaginary) decoder in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] This embodiment provides a multi-stage full-band speech enhancement method based on spectrum compression and neural network. Figure 1 As shown, the following steps are included:

[0031] Step 1: Design a full-band speech spectrum compression curve and calculate the compression module parameters based on the curve;

[0032] Step 2: Use the clear speech dataset, the noise dataset, and the room impulse response dataset to synthesize simulated noisy reverberant speech;

[0033] Step 3: Perform short-time Fourier transform on the simulated noisy reverberation speech data and the corresponding clean reverberation speech data to obtain their short-time Fourier spectra;

[0034] Step 4: construct an MHA-DPCRN model, which is composed of a cascade of an amplitude spectrum enhancement network (MHAN) and a complex spectrum enhancement network (DPCRN); use the compression module parameters obtained in step 1 to initialize the SCB modules of MHAN and DPCRN; use the short-time Fourier spectrum obtained in step 3 to train the weights of the MHA-DPCRN model;

[0035] Step 5: Perform short-time Fourier transform on the noisy reverberation speech signal to be enhanced to obtain a short-time Fourier spectrum;

[0036] Step 6: Input the short-time Fourier spectrum obtained in step 5 into the MHA-DPCRN model trained in step 4, and output the short-time Fourier spectrum of the enhanced speech;

[0037] Step 7: Perform inverse short-time Fourier transform on the short-time Fourier spectrum of the enhanced speech to obtain a time domain signal of the enhanced speech.

[0038] 1. Design of spectrum compression curve and corresponding compression transformation matrix

[0039] Different from the full-band logarithmic compression of Mel spectrum, considering that the short-time Fourier spectrum of speech needs to be restored in the speech enhancement task, in order to avoid the compression loss of the low-frequency band (below 5kHz) containing the main speech information, the present invention does not perform compression transformation on the low-frequency band, that is, the compressed frequency band f cmp The relationship with the original frequency band f is:

[0040] f cmp =f,0≤f≤5kHz (1)

[0041] Since the high frequency band (above 5kHz) contains less speech energy components but occupies more frequency band dimensions, the present invention uses logarithmic compression to reduce the frequency band dimensions corresponding to the high frequency band to the level of the low frequency band, so that the model will not affect the processing performance of the low frequency band due to excessive focus on the high frequency band characteristics. At the same time, the present invention takes into account the need for smooth connection between the high frequency band and the low frequency band to avoid the generation of artificial noise. When designing the logarithmic compression function, it ensures that the connection frequency points are continuous and differentiable. The relationship is as follows:

[0042]

[0043] In summary, the full-band compression function is as follows:

[0044]

[0045] The corresponding function graph is Figure 2 .

[0046] Therefore, based on the transformation function, a triangular filter bank is designed using a method similar to the Mel transform matrix (HMFayek, Speech Processing for Machine Learning: Filter banks, Mel-Frequency Cepstral Coefficients (MFCCs) and What's In-Between, 2016, doi: https: / / haythamfayek.com / 2016 / 04 / 21 / speech-processing-for-machine-learning.html), and the spectrum compression transformation matrix of the present invention can be calculated. Figure 3 It shows how the compression transformation matrix can achieve spectrum compression, where the upper right is the compression matrix, the lower right is the short-time Fourier amplitude spectrum of the speech before compression, and the upper left is the short-time Fourier amplitude spectrum after compression.

[0047] 2. MHA-DPCRN model and its training strategy

[0048] The MHA-DPCRN model is mainly composed of two parts: a multi-head self-attention mechanism network MHAN (Nicolson A, Paliwal KK. Masked multi-head self-attention for causal speech enhancement [J]. Speech Communication, 2020, 125 (3): 80-96.) and a dual-channel convolutional recurrent network DPCRN (Le X, Chen H, Chen K, et al. DPCRN: Dual-Path Convolution Recurrent Network for Single Channel Speech Enhancement [J]. Proc. Interspeech, 2021, 2811-2815.). As shown in Figure 4 (a), the present invention uses it to form a two-stage progressive model processing framework. First, the amplitude spectrum mask of the clean speech is estimated by MHAN, and the amplitude spectrum of the noisy signal is used to generate a speech short-time Fourier spectrum with a higher signal-to-noise ratio. The speech short-time Fourier spectrum is then used as the input of DPCRN to restore clearer and higher-quality speech. Another innovation of the present invention is that the compression transformation matrix is ​​integrated into the network structure to form an end-to-end system, namely the SCB (Spectrum Compression Block) linear layer in the figure. The parameters of these linear layers are initialized by the compression transformation matrix parameters designed by the present invention. The initialized parameters are not completely fixed, but are continuously updated as the network training progresses until the network converges to the ideal result. At this time, the network can learn the optimal compression parameters according to the desired compression mode. In addition, since the present invention performs a compression transformation on the short-time Fourier spectrum of speech, there is a loss of spectral information in the high-frequency band. Therefore, the present invention also adds an inverse compression transformation matrix ISCB (Inverse Spectrum Compression Block) to recover the lost information. Here, the parameters of ISCB do not require specific initialization and are learned by the network itself.

[0049] The input of the MHA-DPCRN model is the short-time Fourier spectrum of the noisy reverberation signal, denoted as X, Where N is the total number of time domain frames, K is the total number of frequency points, After the model is input, MHAN (denoted as ) Estimate the amplitude spectrum mask The short-time Fourier spectrum of the first-stage noise suppression signal is obtained for:

[0050]

[0051]

[0052] |X| means taking the magnitude spectrum, and ⊙ means element-wise multiplication. After the second stage DPCRN (denoted as ) is processed, and the obtained speech enhancement signal is:

[0053]

[0054] Assume that the short-time Fourier spectrum of the real pure speech signal is S, According to the loss function, the error between the enhanced signal and the pure speech signal is calculated, and then the error gradient is transferred back to update the model parameters. It is worth noting that the present invention first trains MHAN separately, and sets the loss function

[0055]

[0056] Subsequently, the pre-trained MHAN model and DPCRN model are based on the loss function Conduct joint training

[0057]

[0058] 1) Model parameter setting

[0059] During the model training process, the sampling rate of all audio data is 48kHz, the window length of the short-time Fourier transform is 25ms, and the coverage rate between frames is 50%. Therefore, the dimension of the full-band spectrum obtained by the Fourier transform is 601. The present invention sets the dimension after compression transformation to 256, of which the first 125 dimensions correspond to the uncompressed low-frequency band information, and the last 131 dimensions correspond to the compressed high-frequency band information. The parameters mapping the low-frequency band in the compression transformation matrix are fixed, while the parameters mapping the high-frequency band are set to be trainable. In order to achieve batch operations to improve computing efficiency, this embodiment fixes the duration of each input speech segment to 8s. In actual processing, the real part information and the imaginary part information of the short-time Fourier spectrum X are often divided into two different channels, respectively denoted as X r and X i , so the actual input short-time Fourier spectrum is a real tensor of size (N, K, 2), and n, k, and c are also used to represent the time domain frame, frequency point, and feature channel dimensions.

[0060] In the MHAN part, the present invention sets the number of MHAN blocks B to 5 and the number of terminals to 8, as shown in FIG4(b).

[0061] In the DPCRN encoder part, this embodiment uses five two-dimensional convolutional layers (Conv2D), whose feature channel number (filters), convolution kernel size (Kernel-size), and stride (Stride) are {16, 32, 48, 64, 80}, {(2, 5), (2, 3), (2, 3), (2, 3), (2, 1)}, {(1, 2), (1, 1), (1, 1), (1, 1), (1, 1)}, and each Conv2D is followed by a normalization layer (Batch Normalization) and a PReLU activation function. The specific framework of the encoder is as follows Figure 5 .

[0062] The encoder extracts features from the input short-time Fourier spectrum across more channels while reducing the feature resolution. The encoder output then serves as the input to the DPRNN, assuming a tensor of size (N′, K′, C′). In the intra-chunk portion, the RNN uses a bidirectional long short-term memory (BLSTM) network with 80 hidden units to establish connections between frequency points within each frame. This is followed by a linear layer with 128 units and an instant normalization layer. The output is then added to the input to produce the final intra-chunk output. To establish connections between time-domain frames within the inter-chunk frame, a transpose layer is used to transform the dimensions of the intra-chunk output tensor from (n, k, c) to (k, n, c). In the inter-chunk, RNN uses a long short-term memory network (LSTM) to establish the connection between the time domain frame information. The units are 80, followed by a fully connected layer with 128 units and an instant normalization layer. The output result is then added to the input to obtain the final output of the inter-chunk. The specific framework is as follows Figure 6 .

[0063] The decoder uses five deconvolution layers (ConvTrans2D), with feature channel numbers, kernel sizes, and strides of {64, 48, 32, 16, 1}, {(2, 1), (2, 3), (2, 3), (2, 3), (2, 5)}, and {(1, 1), (1, 1), (1, 1), (1, 1), (1, 2)}. Each ConvTrans2D is followed by a batch normalization layer and a PReLU activation function. The output of the DPRNN or deconvolution layer is connected to the output of the corresponding convolution layer (ConvOutput) through a channel-wise concatenation structure before being fed into the next deconvolution layer. The specific framework is as follows: Figure 7 .

[0064] 2) Loss function and training strategy

[0065] The loss function used in this invention is the compressed minimum mean square error (MSE) function. Assuming that a short-time Fourier transform time-frequency point is The compressed time-frequency point is defined as where θ Y represents the phase angle of the complex number, β represents the compression coefficient, is a compression symbol. Then its actual part and the imaginary part It can be further written as

[0066]

[0067]

[0068]

[0069] So the loss function is defined according to the real part, imaginary part and amplitude respectively.

[0070]

[0071]

[0072] The loss function of the above training process can be expressed as

[0073]

[0074]

[0075] The second-order norm and infinity norm here are essentially calculating the MSE, which takes advantage of its convex function characteristics, making it more conducive to the derivative operation in back propagation to converge to the minimum point of the loss function. During the training process, the Adam optimizer is used, and the learning rate is updated using the warmup strategy.

[0076]

[0077] Where α is the learning rate, Ψ represents the number of training steps, while Ψ represents the number of warm-up steps. This learning rate update method causes the learning rate to slowly increase from zero at the beginning of training to avoid parameter divergence caused by an excessively large initial learning rate. As the error decreases, the learning rate is gradually reduced to achieve model convergence. The batch size for training is set to 12, and the maximum training cycle is 150 epochs.

[0078] 3. Dataset and evaluation metrics

[0079] This example uses the DNS4 (Deep Noise Suppression 4) speech and noise dataset for model training. It contains approximately 70,000 audio files (total duration approximately 600 hours). This example uses 90% of the dataset as the training set and 10% as the validation set. The room impulse response (RIR) required to generate reverberation comes from openSLR26 and openSLR28. The clear speech is randomly convolved with the RIR and then added to the noise to generate a noisy reverberation signal. The signal-to-noise ratio is randomly selected in the range of [-5dB, 15dB].

[0080] To evaluate the model's generalization ability, the enhancement effect was tested using both simulated and real-world datasets. The simulated dataset used 824 test data points from VCTK / DEMAND, while the real-world dataset used 735 data points from the blind set of DNS4. The speech acquisition environment in this dataset is more complex, making speech enhancement more challenging. For the simulated dataset, objective evaluation metrics used were PESQ, STOI, CSIG (speech quality level), CBAK (noise suppression level), and COVL (overall quality level). Higher values ​​indicate better enhancement. To further compare the results, the results were compared with those of previous full-band state-of-the-art models on the dataset, including RNNoise, Percep Net, DCCRN, DCCRN+, Deep Filter Net, and S-DCCRN. For the real-world dataset, SIG (speech quality level), BAK (noise suppression level), and OVRL (overall quality level) scores based on the P.835 subjective evaluation criteria were presented. The model used for comparison was NSNet2.

[0081] 4. Experimental results

[0082] The test results of the simulated dataset and the real dataset are shown in Tables 1 and 2. It can be seen that the present invention achieves the most superior processing results on both the simulated dataset and the real dataset, which not only better preserves the speech information but also significantly suppresses the environmental noise.

[0083] Table 1 Simulation data test results

[0084]

[0085] Table 2 Real data test results

[0086]

Claims

1. A multi-stage full-band speech enhancement method based on spectrum compression and neural network, characterized in that: The method comprises the following steps: Step 1: designing a full-band speech spectrum compression curve and calculating compression module parameters according to the compression curve; Step 2: Use the clear speech dataset, the noise dataset, and the room impulse response dataset to synthesize simulated noisy reverberant speech; Step 3, performing short-time Fourier transform on the simulated noisy reverberation speech data and the corresponding pure reverberation speech data to obtain short-time Fourier spectra of the two; Step 4: construct an MHA-DPCRN model, which is composed of a cascade of an amplitude spectrum enhancement network (MHAN) and a complex spectrum enhancement network (DPCRN); use the compression module parameters obtained in step 1 to initialize the spectrum compression module of the MHA-DPCRN; use the short-time Fourier spectrum obtained in step 3 to train the MHA-DPCRN model weights; Step 5: Perform short-time Fourier transform on the noisy reverberation speech signal to be enhanced to obtain a short-time Fourier spectrum; Step 6: Input the short-time Fourier spectrum obtained in step 5 into the MHA-DPCRN model trained in step 4, and output the short-time Fourier spectrum of the enhanced speech; Step 7: Perform inverse short-time Fourier transform on the short-time Fourier spectrum of the enhanced speech to obtain a time domain signal of the enhanced speech.

2. The multi-stage full-band speech enhancement method based on spectrum compression and neural network according to claim 1, characterized in that: In step 1, the spectrum compression curve keeps the frequency points in the 0kHz-5kHz band unchanged, and performs logarithmic smooth compression on the frequency points in the 5kHz-24kHz band. Let the original frequency band and the compressed frequency band be f and f respectively. cmp , the specific conversion formula is as follows:

3. The multi-stage full-band speech enhancement method based on spectrum compression and neural network according to claim 1, characterized in that: In step 4, the magnitude spectrum enhancement network MHAN is used to estimate the short-time Fourier magnitude spectrum of the real mask acting on the noisy reverberant speech to improve the speech signal-to-noise ratio, and the complex spectrum enhancement network DPCRN cascaded with it directly estimates the short-time Fourier complex spectrum of the reverberant clean speech to achieve a more thorough noise suppression effect; During the training process, the magnitude spectrum enhancement network MHAN is first trained separately, and then the converged magnitude spectrum enhancement network MHAN is cascaded with the complex spectrum enhancement network DPCRN for joint training.

4. The multi-stage full-band speech enhancement method based on spectrum compression and neural network according to claim 1, characterized in that: In step 4, a linear layer for implementing spectrum compression is added before the encoder of the complex spectrum enhancement network DPCRN. Both this linear layer and the first linear layer of the magnitude spectrum enhancement network MHAN are initialized using the designed compression module parameters. The parameters of the linear layer for mapping high-frequency bands, i.e., above 5 kHz, are set to be trainable, and the parameters for mapping low-frequency bands, i.e., below 5 kHz, are set to be non-trainable.

Images