A Speech Enhancement Method Based on Score Domain Information Interaction in Dual-Path UNET

Abstract

The present invention relates to a speech enhancement method based on a two-way UNET with fractional domain information interaction. The method comprises the following steps: collecting pure speech and noise; mixing the collected data according to a target signal-to-noise ratio, extracting corresponding amplitude spectrum features, phase spectrum features, and fractional domain features of noisy speech as network input; calculating the network training targets, including speech mask and noise mask, from the amplitude spectrum; constructing a network model based on speech and noise information interaction and a loss function based on the time domain spectrum, frequency domain spectrum, and speech noise mask; inputting the processed fractional domain features into the network, minimizing the back propagation of the loss function, and optimizing the parameters in the network model; extracting fractional domain features of the speech signal to be enhanced, inputting them into the trained network, and outputting the target, reconstructing the enhanced signal from the amplitude spectrum and phase spectrum. The present invention characterizes more speech features through fractional domains of different orders, and establishes information interaction and collaborative work between speech and noise between the two-way branches to achieve better speech enhancement effects.

Description

Technical Field

[0001] The present invention belongs to the field of speech enhancement, and in particular relates to a speech enhancement method based on fractional domain information interaction of a dual-path UNET. Background Art

[0002] Deep learning-based speech enhancement algorithms outperform traditional methods when dealing with non-stationary noise while avoiding the generation of musical noise, making them a key research area in speech enhancement in recent years. Current deep learning-based speech enhancement algorithms often use training features from the time domain, frequency domain, and the human ear's perceptual domain. Depending on the training objective, these algorithms can be categorized into frequency domain amplitude estimation, time domain waveform estimation, and contrast ratio mask estimation. These include directly estimating the frequency domain amplitude spectrum of noisy speech; simultaneously estimating the frequency domain amplitude spectrum and phase spectrum of noisy speech; estimating the frequency domain amplitude using Mel-domain features; estimating the frequency domain amplitude using both the time domain waveform and the frequency domain log-power spectrum as training features; and estimating the frequency domain amplitude using modules such as self-attention mechanisms or multi-scale filters to capture more effective information between frequency and time domain features and improve feature utilization.

[0003] The deep neural network-based speech enhancement methods discussed above still have certain limitations when describing speech features. This is reflected in the fact that commonly used feature representations focus on the time-frequency domain and the perceptual domain. However, there are actually many transform-domain methods for representing signals in the time-frequency domain, such as the wavelet transform. The fractional Fourier transform is also a commonly used transform-domain tool. The fractional Fourier transform offers a new perspective on signal transformation based on the Fourier transform, mapping the signal into a new transform domain located between the time and frequency domains—the fractional domain. It is applicable to many pure and noisy signals that would normally be coupled in the time-frequency domain. However, in the fractional Fourier plane with an appropriately rotated angle, this coupling is eliminated, allowing for better separation of the original signal from the noise signal. Therefore, it is widely used in signal analysis and time-frequency analysis. The fractional Fourier transform is applicable to speech because different orders of Fourier transform have significant differences in their energy concentration for speech and noise.

[0004] In order to address the limitations of feature expression, the present invention provides a score domain information interaction network model based on a dual-path UNET and applies it to a deep speech enhancement method. Summary of the Invention

[0005] The purpose of the present invention is to solve the limitation problem of feature expression in existing speech enhancement technology and provide a multi-order feature expression method based on fractional domain.

[0006] To solve the above technical problems, the technical solution adopted by the present invention is: a speech enhancement method based on score domain information interaction of two-way UNET, comprising the following steps:

[0007] Step 1: Collect training set speech, including clean speaker speech s and noise data n;

[0008] Step 2: Extract training data and training features. The collected speech and noise are mixed according to the target signal-to-noise ratio, and the corresponding amplitude spectrum features, phase spectrum features, and fractional domain features of the noisy speech are extracted as input data for the network model.

[0009] Step 3: Extract training targets. Calculate the network training targets based on the Fourier amplitude spectrum obtained in step 2: speech mask and noise mask.

[0010] Step 4: Construct a network model for the interaction between speech and noise information based on a dual-path UNET and a loss function based on time domain spectrum, frequency domain spectrum, and speech noise mask;

[0011] Step 5: Train the model. Input the score domain features processed in step 2 into the network model for training. The obtained network is solidified for subsequent prediction results.

[0012] Step 6: Test the results. Extract the score domain features of the speech signal to be enhanced according to step 2, input the features into the network model trained in step 5 for prediction, output the estimated speech mask and noise mask, and combine them with the amplitude spectrum and phase spectrum obtained in step 2 to reconstruct the predicted enhanced speech signal.

[0013] Furthermore, the specific steps of mixing according to different signal-to-noise ratios in step 2 are:

[0014] Step 2.1 Mix speech s and noise n according to the signal-to-noise ratio formula and set the weight factor according to the signal-to-noise ratio snr Update the noise n = n * θ, and the noisy speech is represented by y = s + n;

[0015] Step 2.2 Calculate the amplitude spectrum and phase spectrum of speech, noise, and noisy speech by using short-time Fourier transform, where the number of frames obtained after the audio s, n, y is windowed and framed is T, and the amplitude spectrum and phase spectrum S of the audio are obtained by short-time Fourier transform STFT. mag / phase (Voice), N mag / phase (noise), Y mag / phase (noisy speech);

[0016] Step 2.3 extracts R-order fractional complex spectra of the noisy speech. The dimension of each order fractional spectrum is T×W, where T represents the time frame and W represents the frame length. Combine these R-order fractional spectra into a one-dimensional vector with the dimension of R*T*W.

[0017] Furthermore, the step of extracting the fractional spectrum of the noisy speech in step 2.3 is as follows:

[0018] Noisy speech signal y t The fractional spectrum of order p is calculated as follows:

[0019] F p {y t (w)}(u)=∫Kp(u,w)y t (w)dw (1)

[0020] Let α=pπ / 2, when α≠Zπ (Z is an integer), the kernel function K p (u, w) is:

[0021]

[0022] When α=Zπ, the kernel function is δ(wu), and when α=(2Z+1)π, the kernel function is δ(w+u); where t represents the number of frames, and the value range is t∈[1, T]; w represents the frequency, and the value range is w∈[1, W].

[0023] Furthermore, the specific steps of calculating the training target speech mask and noise mask in step 3 are:

[0024] Calculate the speech mask IRM s The formula is:

[0025]

[0026] Calculate the noise mask IRM n The formula is:

[0027]

[0028] Among them S mag and N mag are the amplitude spectra of speech and noise calculated in step 2 respectively.

[0029] Furthermore, the network model based on speech and noise information interaction constructed in step 4 includes a speech channel and a noise channel, wherein the speech channel and the noise channel have the same network structure, including an encoding layer, a residual multi-head self-attention module and a decoding layer, and an interaction module is interspersed in the middle to communicate information between speech and noise. Finally, the estimation results of the two paths are fused, and the weights and biases of the neural network are adjusted according to the back propagation of the loss function. Finally, the neural network reaches the global optimal solution, the neural network model training is completed, and the trained weights are solidified for each algorithm call.

[0030] Furthermore, the structure of the speech channel model constructed in step 4 is:

[0031] The encoder structure of the speech channel is: Assume that the output of the i-layer encoder is

[0032]

[0033] Where Fusion represents the information fusion module of speech and noise, PReLU represents the activation function, BN represents batch normalization, and Conv represents the complex convolution layer. The noise channel has the same structure as the speech channel, and the output of the noise channel i-layer encoder is recorded as

[0034] The dilated residual self-attention module (DRA) is set after the encoding layer. The DRA module structure is: forward propagation layer FF, residual self-attention module RSA, and then through the dilated convolution layer DC to output the features of this module and The specific steps are as follows: the forward propagation layer (FF) contains two linear layers, each followed by an activation layer PReLU and dropout. The Fusion module fuses the resulting features of the speech and noise channels and inputs them as the output of the FF to the next module RSA. The residual self-attention module (RSA) contains two residual modules (Res) and a self-attention module (SA). Each Res includes a complex convolution layer with the same parameter settings as the convolution layer in the encoding layer. The number of input channels is the same as the input feature dimension. The input of Res and the output of the complex convolution layer are added as the output of Res. The Fusion module fuses the outputs of the Res modules of speech and noise as the input of the next module SA. SA combines the outputs of temporal attention (TA) and feature attention (FA), specifically expressed as:

[0035] k=Reshape(Conv((·) Res )), k∈{K,Q,V} (6)

[0036]

[0037] (·) TA / FA =(·) Res +Conv(Reshape * (TA / FA)) (8)

[0038] Among them (·) represents or / means or T represents the time dimension, F represents the feature dimension, and C represents the number of channels. When calculating the TA output feature, Reshape represents When calculating FA, Reshape means Reshape *Represents the reverse operation of Reshape; the attention at the time level and the feature level are combined as the output of SA. The specific operation is:

[0039] (·) SA =PReLU(BN(Conv(Concat{(·) TA , (·) FA , (·) Res}))) (9)

[0040] Concat represents a concatenation operation. The number of input and output channels is (3C, C) respectively. The output of module RSA is:

[0041]

[0042] Where FF represents the first forward propagation layer, Indicates element-by-element addition; the dilated convolution layer DC further expands the feature reading range and contains N dilated convolution blocks DB. Each DB contains two convolution layers, and the dilation parameter of the first layer is set to 2 i -1 , the second layer expansion parameter is set to 2 N-i , where i represents the i-th DB, and there are N DBs in total. With the help of the residual structure, the input of the DB and the output of the second convolutional layer are added together to obtain the result which is then fused as the output of the DB. The output of the last block is recorded as the output of the DRA module as (.) DRA , input to the decoder;

[0043] After the DRA module, a decoding layer (Decoder) is set, which includes a deconvolution layer, a gating module (Gated), and a fusion module. The parameter settings of the deconvolution layer are opposite to those of the convolution layer in the encoder. The specific structure is as follows: the deconvolution layer has the opposite structure to the corresponding convolution layer; each convolution layer is connected to the Gated layer, which includes a convolution layer BN and PReLU. Specifically, the output of the deconvolution layer is connected to the features of the corresponding encoding layer. After the convolution layer, BN, and PReLU, the output of the gate module is multiplied element-by-element with the features of the corresponding encoding layer to obtain the output of the gate module. The features of the speech and noise channels are then fused through the information fusion module to obtain the output of the convolution layer (·). dec .

[0044] After the decoding layer, the output results of the speech channel and the noise channel are merged and the predicted value is output.

[0045] Furthermore, the structure of the speech and noise information fusion module Fusion is:

[0046]

[0047] Signal1 can represent the voice channel or noise channel The output of the previous layer, Simnal2 is the opposite, i represents the number of layers, fusion represents the output of the fusion module, represents the real part, represents the imaginary part, j represents the sign of the imaginary part;

[0048] Furthermore, the decoder is connected to the joint module Merge to combine the features of the speech channel and the noise channel. Specifically, the noise channel spectrum estimation value is recorded as The waveform estimation is recorded as The speech channel spectrum estimation is denoted as The time domain waveform is recorded as The network's speech estimate is set to and The estimated values ​​of the noise channel and speech channel are output to the network for the loss function calculation in the next part.

[0049] Furthermore, the step 4 establishes the loss function of the time domain spectrum, frequency domain spectrum, and speech noise mask, and the specific steps are:

[0050]

[0051] The specific definition is:

[0052]

[0053] in ||·|| represents the L2 norm, the same below;

[0054]

[0055]

[0056] Furthermore, the step of reconstructing the speech signal in step 6 is specifically as follows: the reconstruction method is the same as the time domain estimation value of the speech channel in step 4 The calculation steps are the same.

[0057] The method of the present invention maps the speech signal to the fractional domain, characterizes more speech features through fractional domains of different orders, enriches the extracted speech features, and utilizes the differences in speech and noise distribution in the fractional domain to propose a speech enhancement method based on the fractional domain information interaction of a two-way UNET. In the two-way branch system, one path is used to estimate the speech mask and the other path is used to estimate the noise mask. An information interaction mechanism between speech and noise is established between the two-way branches. The two-way branches work together to promote speech recovery and achieve better speech enhancement effects. The present invention utilizes multi-order fractional domains to characterize and enrich speech signal features, and uses a two-way branch system to establish an information interaction mechanism, which can significantly improve the enhancement effect of speech signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of a speech enhancement method based on the interaction of score domain information in a dual-path UNET;

[0059] Figure 2 This is a schematic diagram of the score domain information interaction network based on the dual-path UNET;

[0060] Figure 3 This is a schematic diagram of the framework of a speech enhancement method based on score domain information interaction of a dual-path UNET. DETAILED DESCRIPTION

[0061] like Figures 1 to 3 As shown in FIG, a speech enhancement method based on the score domain information interaction of a dual-path UNET is proposed. The network training process includes the following steps:

[0062] Step 1: Collect training speech, including clean speaker speech s and noise data n, with a sampling rate of 16kHz. The data comes from a public dataset.

[0063] Step 2: Extract training data and training features. The collected speech and noise are mixed according to the target signal-to-noise ratio, and the corresponding amplitude spectrum features, phase spectrum features, and fractional domain features of the noisy speech are extracted as input data for the network model.

[0064] Step 2.1: Mix speech s and noise n using the signal-to-noise ratio formula and set the weight factor according to the experimental signal-to-noise ratio snr Update the noise n = n * θ, the noisy speech is represented by y = s + n, and snr is set to three values: -5dB, 0dB, and 5dB;

[0065] Step 2.2: Calculate the amplitude and phase spectra of speech, noise, and noisy speech using short-time Fourier transform. The number of frames obtained by windowing audio s, n, and y is T. The window length is a Hamming window with a window length of W = 256 and a frame shift inc = 128.

[0066] The amplitude spectrum and phase spectrum S of the audio are obtained by short-time Fourier transform STFT mag / phase (Voice), N mag / phase (noise), Y mag / phase (noisy speech);

[0067] Step 2.3: Extract R-order fractional complex spectra of the noisy speech. The dimension of each order fractional spectrum is T×W, where T represents the time frame and W represents the frame length. Combine the R-order fractional spectra into a one-dimensional vector with the dimension of R*T*W.

[0068] Among them, the noisy speech signal frame y t The fractional spectrum of order p is calculated as follows:

[0069] F p {y t (w)}(u)=∫K p (u,w)y t (w)dw (16)

[0070] Let α=pπ / 2, when α≠Zπ (Z is an integer), the kernel function K p (u, w) is:

[0071]

[0072] When α = Zπ, the kernel function is δ(wu), and when α = (2Z+1)π, the kernel function is δ(w+u); where t represents the number of frames, and the value range is t∈[1, T]; w represents the frequency point, and the value range is w∈[1, W]. The R orders extracted in the experiment are [0, 0.1, 0.2, ..., 1], totaling 11 orders.

[0073] Step 3: Extract training targets. Calculate the network training targets based on the Fourier amplitude spectrum obtained in step 2: speech mask and noise mask.

[0074] Calculate the speech mask IRM s The steps are:

[0075]

[0076] Calculate the noise mask IRM n The steps are:

[0077]

[0078] Among them S mag and N mag are the amplitude spectra of speech and noise calculated in step 2 respectively.

[0079] Step 4: Construct a network model for the interaction of speech and noise information based on a dual-path UNET and a loss function based on time domain spectrum, frequency domain spectrum, and speech noise mask:

[0080] Furthermore, the network model based on the interaction of speech and noise information includes a speech channel and a noise channel. The speech channel and the noise channel have the same network structure, including an encoding layer, a residual multi-head self-attention module and a decoding layer. An interaction module is interspersed in the middle to communicate speech and noise information. Finally, the estimation results of the two channels are fused.

[0081] Further, taking the voice channel model as an example:

[0082] The encoder structure of the voice channel is:

[0083]

[0084] in represents the output of the i-layer encoder, Fusion represents the information fusion module of speech and noise, PReLU represents the activation function, BN represents batch normalization, Conv represents the complex convolution layer, the encoding layer channels are set to [8, 16, 32, 64, 128, 128], the kernel is (2, 5), and the stride is set to (1, 2); the noise channel has the same structure as the speech channel, and the output of the noise channel i-layer encoder is recorded as

[0085] The dilated residual self-attention module (DRA) is set after the encoding layer. The DRA module structure is: forward propagation layer FF, residual self-attention module RSA, dilated convolution layer DC, and finally the output feature of this module is obtained. and The specific steps are as follows: the forward propagation layer (FF) contains two linear layers, each followed by an activation layer PReLU and dropout. The Fusion module fuses the resulting features of the speech and noise channels and inputs them as the output of the FF to the next module RSA. The residual self-attention module (RSA) contains two residual modules (Res) and a self-attention module (SA). Each Res includes a complex convolution layer with the same parameter settings as the convolution layer in the encoding layer. The number of input channels is the same as the input feature dimension. The input of Res and the output of the complex convolution layer are added as the output of Res. The Fusion module fuses the outputs of the Res modules of speech and noise as the input of the next module SA. SA combines the outputs of temporal attention (TA) and feature attention (FA), specifically expressed as:

[0086] k=Reshape(Conv((·) Res ))k∈{K,Q,V} (21)

[0087]

[0088] (·) TA / FA =(·) Res +Conv(Reshape * (TA / FA)) (23)

[0089] Among them (·) represents or / means or T represents the time dimension, F represents the feature dimension, C represents the number of channels, which is set to 128. When calculating the TA output feature, Reshape represents When calculating FA, Reshape means Reshape * Represents the reverse operation of Reshape; the attention at the time level and the feature level are combined as the output of SA. The specific operation is:

[0090] (·) SA =PReLU(BN(Conv(Concat{(·) TA , (·) FA , (·) Res}))) (twenty four)

[0091] Concat represents the concatenation operation. The number of input and output channels is (3C, C) respectively. The kernel size and stride of the convolution layer are both set to (1, 1). The output of the RSA module is:

[0092]

[0093] Where FF represents the first forward propagation layer, Indicates element-by-element addition; the dilated convolution layer DC further expands the feature reading range and contains N dilated convolution blocks DB. Each DB contains two convolution layers, and the dilation parameter of the first layer is set to 2 i -1 , the second layer expansion parameter is set to 2 N-i , where i represents the i-th DB, and there are N DBs in total, where N is 8. With the help of the residual structure, the input of the DB and the output of the second convolutional layer are added together to obtain the result which is then fused as the output of the DB. The output of the last block is recorded as the output of the DRA module as (·) DRA , input to the decoder;

[0094] After the DRA module, a decoding layer (Decoder) is set, which includes a deconvolution layer, a gating module (Gated), and a fusion module. The parameter settings of the deconvolution layer are opposite to those of the convolution layer in the encoder. The specific structure is as follows: the deconvolution layer has the opposite structure to the corresponding convolution layer; each convolution layer is connected to the Gated layer, which includes a convolution layer BN and PReLU. Specifically, the output of the deconvolution layer is connected to the features of the corresponding encoding layer. After the convolution layer, BN, and PReLU, the output of the gate module is multiplied element-by-element with the features of the corresponding encoding layer to obtain the output of the gate module. The features of the speech and noise channels are then fused through the information fusion module to obtain the output of the convolution layer (·). dec ;

[0095] Furthermore, the structure of the speech and noise information fusion module Fusion is:

[0096]

[0097] Signal1 can represent the voice channel or noise channel The output of the previous layer, Signal2 is the opposite, i represents the number of layers, fusion represents the output of the fusion module, represents the real part, represents the imaginary part, j represents the sign of the imaginary part;

[0098] Furthermore, the network model is connected to the joint module (Merge) after the Decoder to combine the features of the speech channel and the noise channel and output the predicted value. Specifically, the noise channel spectrum estimation value is recorded as The waveform estimation is recorded as The speech channel spectrum estimation is denoted as The time domain waveform is recorded as The network's speech estimate is set to The parameter α is set to 0.5; and The estimated values ​​of the noise channel and speech channel are output to the network for the loss function calculation in the next part.

[0099] Furthermore, the loss functions of time domain spectrum, frequency domain spectrum and speech noise mask are established. The specific steps are as follows:

[0100]

[0101] The specific definition is:

[0102]

[0103] in ||·|| represents the L2 norm, the same below;

[0104]

[0105]

[0106] The loss function (12) where β, γ, and δ are set to 0.1;

[0107] Step 5: Train the model. Input the score domain features processed in step 2 into the network model for training. Adjust the weights and biases of the neural network according to the loss function back propagation. Finally, the neural network reaches the global optimal solution. The model training is completed and the trained weights are solidified for each algorithm call.

[0108] Step 6: Test results. Extract the score domain features of the speech signal to be enhanced according to step 2, input the features into the network model trained in step 5 for prediction, output the estimated speech mask and noise mask, and reconstruct the predicted speech signal by combining the amplitude spectrum and phase spectrum obtained in step 2. The reconstruction method is the same as the time domain estimation value of the speech channel in step 4. The calculation steps are the same.

Claims

1. A speech enhancement method based on fractional domain information interaction of two-way UNET, characterized by: The following steps are involved: (1) Collect training set speech, including pure speaker speech s and noise data n; (2) Training data and training feature extraction: the collected speech and noise are mixed according to the target signal-to-noise ratio, and the amplitude spectrum features, phase spectrum features of speech, noise, and noisy speech, as well as the fractional domain features of noisy speech, are extracted as input data for the network model; (3) Extracting training targets: Calculating the network’s training targets based on the amplitude spectrum obtained in step (2): speech mask and noise mask; (4) Construct a network model for speech and noise information interaction based on a dual-path UNET and a loss function based on time domain spectrum, frequency domain spectrum, and speech noise mask; (5) Training the model: input the score domain features processed in step (2) into the network model for training, and the obtained network is solidified for subsequent prediction results; (6) Test results: extract the score domain features of the speech signal to be enhanced according to step (2), input the features into the network model trained in step (5) for prediction, output the estimated speech mask and noise mask, and combine the amplitude spectrum and phase spectrum obtained in step (2) to reconstruct the predicted enhanced speech signal; The network model based on speech and noise information interaction constructed in the step (4) includes a speech channel and a noise channel, wherein the speech channel and the noise channel have the same network structure, including an encoding layer, a residual multi-head self-attention module and a decoding layer, and an interaction module is interspersed in the middle to communicate information between speech and noise. Finally, the estimation results of the two channels are fused, and the weights and biases of the neural network are adjusted according to the back propagation of the loss function so that the neural network reaches the global optimal solution. After the training of the neural network model is completed, the trained weights are solidified for each algorithm call.

2. The speech enhancement method based on fractional domain information interaction of two-way UNET according to claim 1 is characterized in that: The specific steps of mixing according to different signal-to-noise ratios in step (2) are: 2.1 Mix speech s and noise n according to the signal-to-noise ratio formula and set the weight factor according to the signal-to-noise ratio snr , update noise , noisy speech is represented as ; 2.2 Using short-time Fourier transform to calculate the amplitude spectrum and phase spectrum of speech, noise, and noisy speech, the audio The number of frames obtained after windowing and framing is , the amplitude spectrum S of the speech is obtained by short-time Fourier transform STFT mag and phase spectrum S phase ; Amplitude spectrum of noise N mag and phase spectrum N phase ; Amplitude spectrum Y of noisy speech mag and phase spectrum Y phase ; 2.3 Extract R orders of fractional complex spectra of noisy speech, and the dimension of each order of fractional spectrum is , where T represents the time frame and W represents the frame length. The R-order fractional spectra are combined into a dimension of One-dimensional vector of noisy speech signal The order is The fractional spectrum of is calculated as follows: ， ， make ,when When for: ， ， when When , the kernel function is ,when When , the kernel function is ;in Indicates the number of frames, the value range is ; Indicates frequency, value range ; is an integer.

3. The speech enhancement method based on fractional domain information interaction of two-way UNET according to claim 1 is characterized in that: The specific formula for calculating the training target speech mask and noise mask in step (3) is: Calculate speech mask The formula is: ， ， Compute noise mask The formula is: ， ， in and are the amplitude spectra of speech and noise calculated in step 2 respectively.

4. The speech enhancement method based on fractional domain information interaction of two-way UNET according to claim 1 is characterized in that: The structure of the speech channel model constructed in step (4): The encoder expression of the voice channel is: ， ， in: Indicates the The output of the layer encoder, represents the information fusion module of speech and noise, represents the activation function, represents batch normalization, Represents a complex convolution layer; the noise channel has the same structure as the speech channel. The output of the layer encoder is recorded as ; The expansion residual self-attention module DRA is set after the encoding layer. The DRA module structure is: forward propagation layer , the residual self-attention module RSA, and then through the dilated convolution layer DC, output the features of this module and , the specific steps are: The forward propagation layer FF contains two linear layers, each followed by an activation layer PReLU and dropout. The fusion module fuses the resulting features of the speech and noise channels and inputs them as the output of FF to the next module RSA. The residual self-attention module RSA consists of two residual modules Res and a self-attention module SA; each Res includes a complex convolution layer with the same parameter settings as the convolution layer in the encoding layer. The number of input channels is the same as the input feature dimension. The input of Res and the output of the complex convolution layer are added as the output of Res. The fusion module fuses the output of the Res module of speech and noise as the input of the next module SA; SA combines the outputs of temporal attention TA and feature attention FA, which is specifically expressed as: ， ， ， ， ， ， in represent or , / means or, , represents the time dimension, represents the feature dimension, Represents the number of channels when calculating TA output features express , when calculating FA, Reshape means , express The reverse operation of ; the attention at the time level and the feature level are combined as the output of SA, the specific operation is: ， ， Concat represents the connection operation, and the number of input and output channels are , the output of module RSA is: ， ， in represents the first forward propagation layer, It means element-by-element addition; The dilated convolution layer DC further expands the feature reading range and contains N dilated convolution blocks DB. Each DB contains two convolution layers. The dilation parameter of the first layer is set to , the second layer expansion parameter is set to ,in Indicates the DB, a total of N DB; with the help of the residual structure, the input of DB and the output of the second convolution layer are added together to obtain the result which is used as the output of DB through fusion, and the output of the last block is recorded as the output of the DRA module , input to the decoder; The decoding layer Decoder is set after the DRA module, which includes the deconvolution layer, the gating module Gated and the Fusion module. The parameter setting of the deconvolution layer is opposite to the convolution layer in the Encoder. The specific structure is: The deconvolution layer has the opposite structure to the corresponding convolution layer. Each convolution layer is connected to Gated, which contains convolution layer BN and PReLU. Specifically, the output of the deconvolution layer is connected to the features of the corresponding coding layer. After the convolution layer, BN and PReLU, the output of the gate module is multiplied element by element with the features of the corresponding coding layer to obtain the output of the gate module. The features of the speech and noise channels are then fused through the information fusion module to obtain the output of the convolution layer. ; After the decoding layer, the output results of the speech channel and the noise channel are merged and the predicted value is output.

5. The speech enhancement method based on fractional domain information interaction of two-way UNET according to claim 1, characterized in that: The structure of the speech and noise information fusion module Fusion is: ， ， in Can represent voice channels or noise channel The output of the previous layer, On the contrary, Indicates the number of layers, represents the output of the fusion module, represents the real part, represents the imaginary part, Indicates the sign of the imaginary part.

6. The speech enhancement method based on fractional domain information interaction of two-way UNET according to claim 1, characterized in that: The decoder is followed by the joint module Merge, which combines the features of the speech channel and the noise channel. The reconstruction method of the speech signal in step 6 is the time domain estimate of the speech channel. The calculation steps are the same, specifically: The estimated value of the noise channel spectrum is recorded as , the waveform estimation is recorded as , the speech channel spectrum estimation is recorded as , the time domain waveform is recorded as , the network's speech estimate is set to ; and The estimated values ​​of the noise channel and speech channel are output to the network for the loss function calculation in the next part.

7. The speech enhancement method based on fractional domain information interaction of two-way UNET according to claim 1, characterized in that: Step (4) establishes the loss function of time domain spectrum, frequency domain spectrum, and speech noise mask. The specific calculation formula is: ， ， The specific definition is: ， ， in , express Norm, the same below; ， ， ， 。

Images