An improved high-performance denoising algorithm based on self-attention mechanism

Abstract

This paper provides an improved, high-performance noise reduction algorithm based on the self-attention mechanism. This algorithm leverages the powerful mapping capabilities of deep neural networks to reduce the computational complexity of the self-attention mechanism. Combined with a spectral masking strategy, it achieves a high-performance noise reduction algorithm for complex noisy environments. Compared to other speech enhancement algorithms, this algorithm has lower computational complexity, effectively captures the time-frequency characteristics of the input speech, and achieves better noise reduction in low signal-to-noise ratio environments.

Description

Technical Field

[0001] The present invention relates to the technical field of speech noise reduction, and in particular to an improved high-performance noise reduction algorithm based on a self-attention mechanism. Background Art

[0002] Voice is widely used in our daily lives, for example, in mobile communications, audio chats, remote conferencing, and voice control. With the continuous improvement of people's quality of life, they have higher expectations for voice quality and clarity. However, voice can be subject to numerous noise artifacts, severely impacting the listener's experience. Currently, the mainstream voice enhancement solution is a frequency domain method based on neural networks. This method operates in the frequency domain, focusing on more features and information. Furthermore, it can improve system robustness and reduce computational costs to a certain extent. However, these algorithms still suffer from high computational complexity and poor performance in complex environments. Summary of the Invention

[0003] In order to solve the problems in the prior art, the present invention utilizes the powerful mapping capability of deep neural networks to reduce the computational complexity of self-attention, and combines the strategy of spectrum masking to realize a high-performance noise reduction algorithm for complex noise environments.

[0004] The technical solution for achieving the purpose of the present invention is:

[0005] This paper addresses some defects of the traditional self-attention algorithm, proposes a corresponding improved algorithm, and builds a complete noise reduction network model to achieve excellent noise reduction effect.

[0006] An improved high-performance denoising algorithm based on the self-attention mechanism includes the following steps:

[0007] 1) The input is processed by layer normalization, then point-wise convolution is used to capture cross-channel information, and then depth-wise convolution is used to extract feature information in the time-frequency dimension. Finally, the three projection matrices Q, K, and V are obtained by segmentation. The calculation formula is:

[0008]

[0009]

[0010]

[0011] in, and Represent the projection matrices in point-by-point convolution and depth-wise convolution respectively;

[0012] 2) Change the dimensions of the three matrices Q, K, and V to R (C,L×F) 、R (L×F,C) and R (C,L×F) ; Where R represents the real number domain, C represents the channel dimension, L represents the number of time frames, and F represents the number of frequency points;

[0013] 3) Calculate the dot product of the Q and K matrices to implicitly encode global information across channels. After the normalized exponential function (softmax) calculation, the size of the channel attention map is R (C,C) , do a dot product between the channel attention map and the V matrix to get the self-attention;

[0014] Channel attention calculation formula:

[0015]

[0016] Among them, μ is a learnable scale factor used to adjust the dot product result of Q and K.

[0017] Self-attention calculation formula:

[0018] Output=W p ChannelAttention(Q,K,V)+Input.

[0019] Where W p Represents the projection matrix for point-wise convolution.

[0020] Furthermore, for the entire network, cMSE is used as the loss function, and the calculation formula is:

[0021]

[0022] Among them S cRI and S cMag Represent the complex-valued compressed spectrum and amplitude-compressed spectrum of clean speech, represents the estimated speech spectrum, α and β represent the rescaling factors, which are 0.3 and 0.7 respectively.

[0023] Among them S cRI and S cMag Specifically, it can be expressed as:

[0024] S cMag =|S Mag | c

[0025]

[0026] Where c represents the compression coefficient, and its value is 0.3.

[0027] The beneficial effects of the present invention are:

[0028] 1. Compared with other speech enhancement algorithms, the algorithm of the present invention has lower computational complexity.

[0029] 2. The algorithm of the present invention can effectively capture the time-frequency characteristics of the input speech.

[0030] 3. The algorithm of the present invention can achieve better noise reduction effect in low signal-to-noise ratio environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is the overall structure diagram of traditional self-attention;

[0032] Figure 2 This is the overall structure diagram of the improved self-attention;

[0033] Figure 3 This is a performance comparison chart of enhanced speech and unprocessed noisy speech. DETAILED DESCRIPTION

[0034] Figure 1-Figure 3 To explain the present invention, but the present invention is not limited to Figure 1-Figure 3 within the range shown.

[0035] Traditional self-attention algorithms such as Figure 1 shown.

[0036] Among them, Q, K, and V are three matrices obtained based on the input vector, representing Query, Key, and Value respectively. Therefore, the overall calculation process of self-attention is shown in formula (1):

[0037]

[0038] in, Represents the scale transformation factor, used to scale QK T Otherwise, the gradient will disappear after softmax. Self-attention has a large receptive field and can capture long-term feature information. However, using self-attention will increase the complexity and computational complexity of the neural network. For example, when calculating the self-attention graph of the speech spectrum Y(L,F), its time complexity is O(L 2 F 2 ).

[0039] To alleviate this problem, we propose an improved self-attention that can capture long-term feature information under the condition of linear complexity. Figure 2 shown.

[0040] We first process the input through Layer Normalization, then use point-wise convolution to capture cross-channel information, then use depth-wise convolution to extract feature information in the time-frequency dimension, and finally segment to obtain the three projection matrices Q, K, and V. The calculation process of this process is shown in formula (2):

[0041]

[0042]

[0043]

[0044] in, and Represent the projection matrices in point-wise convolution and depth-wise convolution respectively. The introduction of point-wise convolution and depth-wise convolution layers can make full use of the features of different channels at the same time-frequency position, so that the network can focus on local information.

[0045] In order to perform subsequent calculations, we change the dimensions of the three matrices Q, K, and V to R (C,L×F) 、R (L×F,C) and R (C,L×F) , where R represents the real number domain, C represents the channel dimension, L represents the number of time frames, and F represents the number of frequency points;

[0046] After that, we calculated the dot product of the Q and K matrices to implicitly encode global information across channels. After softmax, the size of the channel attention map is R (C,C) Finally, we use the channel attention map to do a dot product with the V matrix to get the final self-attention. The calculation process of channel attention is shown in formula (3):

[0047]

[0048] Among them, μ is a learnable scale factor used to adjust the dot product result of Q and K. Therefore, the complete self-attention calculation result is shown in formula (4):

[0049] Output=W pChannelAttention(Q,K,V)+Input

[0050] Where W p Represents the projection matrix for point-wise convolution.

[0051] Furthermore, we perform multi-head processing in the channel dimension so that the improved self-attention can compute attention in parallel and enhance the expressiveness of the model.

[0052] For the entire network, we use cMSE as the loss function, which can take into account the influence of amplitude and phase at the same time. The specific calculation process is shown in formula (5):

[0053]

[0054] Among them S cRI and S cMag They represent the complex-valued compressed spectrum and amplitude-compressed spectrum of clean speech respectively. S represents the estimated speech spectrum, α and β represent the scale adjustment factors, which are 0.3 and 0.7 respectively. cRI and S cMag Specifically, it can be expressed as:

[0055] S cMag =|S Mag | c

[0056]

[0057] Where c represents the compression coefficient, and its value is 0.3.

[0058] In order to verify the noise reduction effect of the improved self-attention, we use the dataset provided by the first Deep NoiseSuppression Challenge to generate our dataset. For the training set, we generated a total of 100 hours of noisy speech and clean speech pairs. The reverberation time RT of the training set is 60 The range of the noise level was 0.3s to 1.4s, and the signal-to-noise ratio (SNR) ranged from -5dB to 20dB. For the validation set, all conditions were identical to those for the training set, except that the duration was 10 hours. For the test set, we generated five datasets with a step size of 5dB and a range from -5dB to 15dB. Each dataset was 1 hour long, and all speech was reverberation-free.

[0059] In order to test the noise reduction effect, we selected WB-PESQ, DNSMOS, STOI and SI-SDR as evaluation indicators. Figure 3The performance comparison chart shows performance compared to unprocessed noisy speech. It can be seen that the improved self-attention system significantly improves all metrics compared to unprocessed noisy speech, significantly enhancing speech clarity and intelligibility. Furthermore, subjective testing revealed that even at low signal-to-noise ratios, residual noise was virtually inaudible.

[0060] The above embodiments are only specific embodiments of the present invention, but are not limited to the embodiments. Any equivalent modifications and prior art additions made in accordance with this application without departing from the concept of the present invention are considered to be within the technical scope of the present invention.

Claims

1. An improved high-performance denoising algorithm based on self-attention mechanism, characterized by: The steps include: 1) The input is normalized by layers, then subjected to point-by-point convolution to capture cross-channel information. Next, depthwise convolution is used to extract feature information in the time-frequency dimension. Finally, the three projection matrices Q, K, and V are obtained by segmentation. The calculation formula is: Among them, Q represents Query, K represents Key, and V represents Value. and Represent the projection matrices in point-by-point convolution and depth-wise convolution respectively; 2) Change the dimensions of the three matrices Q, K, and V to R (C,L×F) 、R (L×F,C) and R (C,L×F) , where R represents the real number domain, C represents the channel dimension, L represents the number of time frames, and F represents the number of frequency points; 3) Calculate the dot product of the Q and K matrices to implicitly encode global information across channels. After calculation by the normalized exponential function, the size of the channel attention map obtained is R (C,C) , do the dot product between the channel attention map and the V matrix to get the self-attention; Channel attention calculation formula: Among them, μ is a learnable scale factor used to adjust the dot product result of Q and K. Self-attention calculation formula: Output=W p ChannelAttention(Q,K,V)+Input Where W p Represents the projection matrix for point-wise convolution.

2. The improved high-performance denoising algorithm based on the self-attention mechanism according to claim 1, characterized in that: For the entire network using the self-attention mechanism, the loss function uses cMSE, and the calculation formula is: Among them S cRI and S cMag Represent the complex-valued compressed spectrum and amplitude-compressed spectrum of clean speech, represents the estimated speech spectrum, α and β represent the scale adjustment factors, which are 0.3 and 0.7 respectively. Among them S cRI and S cMag Specifically, it can be expressed as: S cMag =|S Mag | c Where c represents the compression coefficient, and its value is 0.3.

Images