Wind noise suppression method, device, apparatus, and computer-readable storage medium

Abstract

The present invention relates to the field of noise reduction technology, and in particular to a wind noise suppression method, apparatus, device, and computer-readable storage medium, wherein the method comprises: upon determining the presence of wind noise in a microphone signal, performing noise elimination processing on a low-frequency signal in the microphone signal using a preset deep neural network combined with an attention mechanism to obtain a first processed signal, wherein the low-frequency signal is a signal having a frequency less than or equal to a preset frequency, and performing weighted processing on the low-frequency signal features extracted by the deep neural network based on attention weights through the attention mechanism; performing noise elimination processing on a high-frequency signal in the microphone signal using a preset non-neural network algorithm to obtain a second processed signal, wherein the high-frequency signal is a signal having a frequency greater than a preset frequency; and fusing the first processed signal and the second processed signal to obtain a wind noise suppression result. The present invention improves the wind noise suppression effect of a device without increasing the hardware cost and design difficulty of the device.

Description

Technical Field

[0001] The present invention relates to the field of noise reduction technology, and in particular to a wind noise suppression method, device, equipment, and computer-readable storage medium. Background Art

[0002] Wind noise is the most common type of noise encountered in outdoor sound pickup, severely impacting the quality of outdoor sound pickup systems such as mobile phones and hearing aids. Wind noise is caused by the interaction between airflow and obstacles (such as buildings, the human body, and the microphone cavity). Therefore, the characteristics of wind noise caused by different obstacles vary. In many cases, wind noise sound pressure levels can reach 80dB SPL, completely masking speech signals, significantly reducing speech intelligibility and causing auditory discomfort.

[0003] Traditional methods for dealing with wind noise include windscreens, commonly found on handheld microphones and professional shotgun microphones. These come in a variety of materials, including sponge, artificial fur, and iron mesh. Their principle is to reduce the air velocity near the microphone diaphragm, breaking up the airflow and reducing turbulence. Bone conduction sensors (VPUs) designed for voice pickup can capture speech by collecting vibration signals from the human mandible. Since wind noise only occurs in air-conducted sound and not bone-conducted sound, bone conduction sensors can directly mitigate wind noise when picking up speech. However, wind noise suppression solutions based on windscreens and VPUs increase equipment cost and structural design complexity. Summary of the Invention

[0004] The main purpose of the present invention is to provide a wind noise suppression method, device, equipment and computer-readable storage medium, aiming to improve the wind noise suppression effect of the equipment without increasing the hardware cost and design difficulty of the equipment.

[0005] To achieve the above object, the present invention provides a method for suppressing wind noise, the method comprising the following steps:

[0006] When it is determined that wind noise exists in the microphone signal, a low-frequency signal in the microphone signal is subjected to noise elimination processing using a preset deep neural network combined with an attention mechanism to obtain a first processed signal, wherein the low-frequency signal is a signal with a frequency less than or equal to a preset frequency; in the process of performing noise elimination processing on the low-frequency signal using the deep neural network, a low-frequency signal feature extracted by the deep neural network is weighted based on an attention weight using the attention mechanism, wherein the higher the similarity between the low-frequency signal feature and the denoised low-frequency signal feature, the greater the attention weight;

[0007] Performing noise elimination processing on a high-frequency signal in the microphone signal using a preset non-neural network algorithm to obtain a second processed signal, wherein the high-frequency signal is a signal with a frequency greater than the preset frequency;

[0008] The first processed signal and the second processed signal are fused to obtain a wind noise suppression result.

[0009] Optionally, the deep neural network includes an encoder, a recurrent neural network module, an attention mechanism module, a decoder, and a fully connected layer. The step of performing noise elimination processing on the low-frequency signal in the microphone signal using a preset deep neural network combined with the attention mechanism to obtain the first processed signal includes:

[0010] Inputting the low-frequency signal in the microphone signal into the encoder for processing to obtain a first processing result;

[0011] Inputting the first processing result into the recurrent neural network module for processing to obtain a second processing result, wherein, when the first processing result is processed by the recurrent neural network module, a result obtained by processing a previous frame processing result by the recurrent neural network module is used, and the previous frame processing result is a result obtained by the encoder processing a previous frame signal of the microphone signal;

[0012] Inputting the second processing result into the decoder for processing to obtain a third processing result;

[0013] Inputting the first processing result and the third processing result into the attention mechanism module for processing to obtain a fourth processing result;

[0014] The third processing result and the fourth processing result are connected, and the connected result is input into the fully connected layer for processing to obtain a first processed signal.

[0015] Optionally, the attention mechanism module includes a weight calculation block and a weighted processing block, and the step of inputting the first processing result and the third processing result into the attention mechanism module for processing to obtain the fourth processing result includes:

[0016] Inputting the first processing result and the third processing result into the weight calculation block for processing to obtain an attention weight matrix;

[0017] The attention weight matrix and the first processing result are input into the weighted processing block for processing to obtain a fourth processing result.

[0018] Optionally, the weight calculation block includes three convolution layers, one dot product layer, and two activation functions. The step of inputting the first processing result and the third processing result into the weight calculation block for processing to obtain the attention weight matrix includes:

[0019] Inputting the first processing result into a first convolutional layer for performing a convolution operation to obtain a first convolution result, and inputting the third processing result into a second convolutional layer for performing a convolution operation to obtain a second convolution result;

[0020] Inputting the first convolution result and the second convolution result into a first dot product layer to calculate a first dot product result, and inputting the first dot product result into a first activation function for processing to obtain a first activation result;

[0021] The first activation result is input into the third convolutional layer for convolution operation to obtain a third convolution result, and the third convolution result is input into the second activation function for processing to obtain an attention weight matrix.

[0022] Optionally, the weighted processing block includes a convolution layer and a dot product layer, and the step of inputting the attention weight matrix and the first processing result into the weighted processing block for processing to obtain the fourth processing result includes:

[0023] Inputting the first processing result into a fourth convolutional layer for performing a convolution operation to obtain a fourth convolution result;

[0024] The fourth convolution result and the attention weight matrix are input into the second dot product layer to calculate the fourth processing result.

[0025] Optionally, before the step of performing noise elimination processing on the low-frequency signal in the microphone signal using a preset deep neural network combined with an attention mechanism to obtain the first processed signal when determining that wind noise exists in the microphone signal, the method further includes:

[0026] performing a wind noise analysis on the microphone signal, and determining whether wind noise exists in the microphone signal according to the wind noise analysis result;

[0027] After the step of performing wind noise analysis on the microphone signal and determining whether wind noise exists in the microphone signal according to the wind noise analysis result, the method further includes:

[0028] When it is determined according to the wind noise analysis result that no wind noise exists in the microphone signal, the microphone signal is subjected to noise elimination processing using the non-neural network algorithm to obtain a microphone signal after suppressing wind noise.

[0029] Optionally, the step of performing wind noise analysis on the microphone signal and determining whether wind noise exists in the microphone signal according to the wind noise analysis result includes:

[0030] When there are two or more microphone signals, calculating the target correlation between the microphone signals;

[0031] According to the target correlation and the corresponding relationship between the preset correlation and the wind speed, a wind noise analysis result is obtained by matching;

[0032] Or, calculating the target low-frequency energy of a signal having a frequency less than a second preset frequency in any microphone signal;

[0033] According to the target low-frequency energy and the corresponding relationship between the preset low-frequency energy and the wind speed, a wind noise analysis result is obtained by matching.

[0034] To achieve the above object, the present invention further provides a wind noise suppression device, comprising:

[0035] a noise cancellation module for, when determining that wind noise exists in a microphone signal, performing noise cancellation processing on a low-frequency signal in the microphone signal using a preset deep neural network combined with an attention mechanism to obtain a first processed signal, wherein the low-frequency signal is a signal with a frequency less than or equal to a preset frequency; and in the process of performing noise cancellation processing on the low-frequency signal using the deep neural network, performing weighted processing on low-frequency signal features extracted by the deep neural network based on an attention weight using the attention mechanism, wherein the higher the similarity between the low-frequency signal features and the denoised low-frequency signal features, the greater the attention weight;

[0036] The noise elimination module is further configured to perform noise elimination processing on a high-frequency signal in the microphone signal using a preset non-neural network algorithm to obtain a second processed signal, wherein the high-frequency signal is a signal with a frequency greater than the preset frequency;

[0037] A fusion module is used to fuse the first processed signal and the second processed signal to obtain a wind noise suppression result.

[0038] To achieve the above-mentioned objectives, the present invention also provides a wind noise suppression device, which includes: a memory, a processor, and a wind noise suppression program stored in the memory and runnable on the processor. When the wind noise suppression program is executed by the processor, the steps of the wind noise suppression method described above are implemented.

[0039] In addition, to achieve the above objectives, the present invention also proposes a computer-readable storage medium, on which a wind noise suppression program is stored. When the wind noise suppression program is executed by a processor, the steps of the wind noise suppression method described above are implemented.

[0040] In the present invention, when it is determined that there is wind noise in the microphone signal, a preset deep neural network combined with an attention mechanism is used to perform noise elimination processing on the low-frequency signal in the microphone signal to obtain a first processed signal, wherein the low-frequency signal is a signal with a frequency less than or equal to the preset frequency; in the process of performing noise elimination processing on the low-frequency signal through the deep neural network, the low-frequency signal features extracted by the deep neural network are weighted based on the attention weight through the attention mechanism, and the higher the similarity between the low-frequency signal features and the denoised low-frequency signal features, the greater the attention weight; a preset non-neural network algorithm is used to perform noise elimination processing on the high-frequency signal in the microphone signal to obtain a second processed signal, wherein the high-frequency signal is a signal with a frequency greater than the preset frequency; the first processed signal and the second processed signal are fused to obtain a wind noise suppression result.

[0041] The present invention introduces an attention mechanism into a deep neural network, and performs weighted processing on the low-frequency signal features extracted by the deep neural network based on the attention weight through the attention mechanism. The higher the similarity between the low-frequency signal features and the denoised low-frequency signal features, the greater the attention weight. In this way, the deep neural network can focus on important information in the low-frequency signal, that is, the speech signal and the important frequency bands in the speech signal, through the weighted processing of the attention mechanism during the processing of the low-frequency signal, thereby making the deep neural network more accurate in eliminating noise of the low-frequency signal, thereby improving the wind noise suppression effect of the device without increasing the hardware cost and design difficulty of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the hardware operating environment involved in an embodiment of the present invention;

[0043] Figure 2 This is a flow chart of an embodiment of a method for suppressing wind noise according to the present invention;

[0044] Figure 3 A structural diagram of a deep neural network involved in an embodiment of the present invention;

[0045] Figure 4 This is a structural diagram of an attention mechanism module involved in an embodiment of the present invention;

[0046] Figure 5 A structural diagram of a recurrent neural network layer involved in an embodiment of the present invention;

[0047] Figure 6 The figure is a schematic diagram of a wind noise suppression process involved in an embodiment of the present invention.

[0048] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0050] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0051] It should be noted that the wind noise suppression device in the embodiment of the present invention can be a headset, a smart phone, a personal computer, a server and other devices, and is not specifically limited here.

[0052] like Figure 1 As shown, the wind noise suppression device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0053] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation on the wind noise suppression device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0054] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a wind noise suppression program. The operating system is a program that manages and controls the hardware and software resources of the device and supports the operation of the wind noise suppression program and other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used to communicate data with the client; the network interface 1004 is mainly used to establish a communication connection with the server; and the processor 1001 can be used to call the wind noise suppression program stored in the memory 1005 and perform the following operations:

[0055] When it is determined that wind noise exists in the microphone signal, a low-frequency signal in the microphone signal is subjected to noise elimination processing using a preset deep neural network combined with an attention mechanism to obtain a first processed signal, wherein the low-frequency signal is a signal with a frequency less than or equal to a preset frequency; in the process of performing noise elimination processing on the low-frequency signal using the deep neural network, a low-frequency signal feature extracted by the deep neural network is weighted based on an attention weight using the attention mechanism, wherein the higher the similarity between the low-frequency signal feature and the denoised low-frequency signal feature, the greater the attention weight, and the attention mechanism is used to perform weighted processing on the features extracted by the deep neural network;

[0056] Performing noise elimination processing on a high-frequency signal in the microphone signal using a preset non-neural network algorithm to obtain a second processed signal, wherein the high-frequency signal is a signal with a frequency greater than the preset frequency;

[0057] The first processed signal and the second processed signal are fused to obtain a wind noise suppression result.

[0058] Furthermore, the deep neural network includes an encoder, a recurrent neural network module, an attention mechanism module, a decoder, and a fully connected layer. The operation of performing noise elimination processing on the low-frequency signal in the microphone signal using a preset deep neural network combined with the attention mechanism to obtain a first processed signal includes:

[0059] Inputting the low-frequency signal in the microphone signal into the encoder for processing to obtain a first processing result;

[0060] Inputting the first processing result into the recurrent neural network module for processing to obtain a second processing result, wherein, when the first processing result is processed by the recurrent neural network module, a result obtained by processing a previous frame processing result by the recurrent neural network module is used, and the previous frame processing result is a result obtained by the encoder processing a previous frame signal of the microphone signal;

[0061] Inputting the second processing result into the decoder for processing to obtain a third processing result;

[0062] Inputting the first processing result and the third processing result into the attention mechanism module for processing to obtain a fourth processing result;

[0063] The third processing result and the fourth processing result are connected, and the connected result is input into the fully connected layer for processing to obtain a first processed signal.

[0064] Furthermore, the attention mechanism module includes a weight calculation block and a weighted processing block, and the operation of inputting the first processing result and the third processing result into the attention mechanism module for processing to obtain the fourth processing result includes:

[0065] Inputting the first processing result and the third processing result into the weight calculation block for processing to obtain an attention weight matrix;

[0066] The attention weight matrix and the first processing result are input into the weighted processing block for processing to obtain a fourth processing result.

[0067] Furthermore, the weight calculation block includes three convolution layers, one dot product layer, and two activation functions. The operation of inputting the first processing result and the third processing result into the weight calculation block for processing to obtain the attention weight matrix includes:

[0068] Inputting the first processing result into a first convolutional layer for performing a convolution operation to obtain a first convolution result, and inputting the third processing result into a second convolutional layer for performing a convolution operation to obtain a second convolution result;

[0069] Inputting the first convolution result and the second convolution result into a first dot product layer to calculate a first dot product result, and inputting the first dot product result into a first activation function for processing to obtain a first activation result;

[0070] The first activation result is input into the third convolutional layer for convolution operation to obtain a third convolution result, and the third convolution result is input into the second activation function for processing to obtain an attention weight matrix.

[0071] Furthermore, the weighted processing block includes a convolution layer and a dot product layer, and the operation of inputting the attention weight matrix and the first processing result into the weighted processing block for processing to obtain the fourth processing result includes:

[0072] Inputting the first processing result into a fourth convolutional layer for performing a convolution operation to obtain a fourth convolution result;

[0073] The fourth convolution result and the attention weight matrix are input into the second dot product layer to calculate the fourth processing result.

[0074] Furthermore, when it is determined that wind noise exists in the microphone signal, the low-frequency signal in the microphone signal is subjected to noise elimination processing using a preset deep neural network combined with an attention mechanism to obtain a first processed signal. Before the operation, the processor 1001 may also be used to call a wind noise suppression program stored in the memory 1005 and perform the following operations:

[0075] performing a wind noise analysis on the microphone signal, and determining whether wind noise exists in the microphone signal according to the wind noise analysis result;

[0076] After performing wind noise analysis on the microphone signal and determining whether wind noise exists in the microphone signal according to the wind noise analysis result, the method further includes:

[0077] When it is determined according to the wind noise analysis result that no wind noise exists in the microphone signal, the microphone signal is subjected to noise elimination processing using the non-neural network algorithm to obtain a microphone signal after suppressing wind noise.

[0078] Furthermore, the operation of performing wind noise analysis on the microphone signal and determining whether wind noise exists in the microphone signal according to the wind noise analysis result includes:

[0079] When there are two or more microphone signals, calculating the target correlation between the microphone signals;

[0080] According to the target correlation and the corresponding relationship between the preset correlation and the wind speed, a wind noise analysis result is obtained by matching;

[0081] Or, calculating the target low-frequency energy of a signal having a frequency less than a second preset frequency in any microphone signal;

[0082] According to the target low-frequency energy and the corresponding relationship between the preset low-frequency energy and the wind speed, a wind noise analysis result is obtained by matching.

[0083] Reference Figure 2 , Figure 2 FIG. 4 is a flow chart of an embodiment of a method for suppressing wind noise according to the present invention.

[0084] The embodiment of the present invention provides an embodiment of the wind noise suppression method. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than here. In this embodiment, the execution subject of the wind noise suppression method can be a headset, a smart phone, a personal computer, a server and other devices, which are not limited in this embodiment. In this embodiment, for the convenience of description, each embodiment is described with the wind noise suppression device as the execution subject. In this embodiment, the wind noise suppression method includes the following steps:

[0085] Step S10: When it is determined that wind noise exists in the microphone signal, a low-frequency signal in the microphone signal is subjected to noise elimination processing using a preset deep neural network combined with an attention mechanism to obtain a first processed signal, wherein the low-frequency signal is a signal with a frequency less than or equal to a preset frequency; in the process of performing noise elimination processing on the low-frequency signal using the deep neural network, a low-frequency signal feature extracted by the deep neural network is weighted based on an attention weight using the attention mechanism, wherein the higher the similarity between the low-frequency signal feature and the denoised low-frequency signal feature, the greater the attention weight;

[0086] The wind noise suppression device can obtain one or more microphone signals and perform wind noise analysis on the obtained microphone signals to determine whether wind noise exists in the microphone signals or determine the intensity of the wind noise in the microphone signals.

[0087] In this embodiment, when it is determined that there is wind noise in the microphone signal, the microphone signal is subjected to noise elimination to suppress the wind noise. By analyzing the spectrum diagram of the audio signal with wind noise, it is found that wind noise mainly affects the low-frequency band. Therefore, when the microphone signal is subjected to noise elimination processing, the microphone signal can be divided into a low-frequency signal and a high-frequency signal according to a preset frequency, and the low-frequency signal is subjected to noise elimination processing using a deep neural network combined with an attention mechanism. The result obtained is called the first signal for distinction. In this embodiment, the low-frequency signal features extracted by the deep neural network are weighted based on the attention weight through the attention mechanism. The higher the similarity between the low-frequency signal features and the denoised low-frequency signal features, the greater the attention weight. This allows the deep neural network to focus on important information in the low-frequency signal, that is, the speech signal and the important frequency bands in the speech signal, through the weighted processing of the attention mechanism during the processing of the low-frequency signal, thereby making the deep neural network more accurate in noise elimination of the low-frequency signal.

[0088] Specifically, in this embodiment, the low-frequency signal can be a signal in the microphone signal that is less than or equal to the preset frequency, and the high-frequency signal can be a microphone signal in the microphone signal that is higher than the preset frequency. The preset frequency can be set as needed and is not limited in this embodiment.

[0089] In a specific embodiment, the wind noise suppression device can perform frame processing on the acquired microphone signal, and perform wind noise suppression processing on each frame of microphone signal in sequence according to the frame sequence. In this embodiment, the microphone signal of a frame currently being processed is used as the microphone signal. It should be noted that, in a specific embodiment, the wind noise suppression device can perform wind noise suppression processing on the acquired offline microphone signal, or it can perform wind noise suppression processing on the microphone signal acquired by the microphone in real time. For example, in a feasible embodiment, the wind noise suppression device can be a headphone device, in which at least one microphone is provided, and the microphone signal is collected by the microphone. The wind noise suppression device obtains the microphone signal collected by the microphone in real time, performs wind noise suppression processing on the microphone signal, and then outputs the processing result through the speaker in the headphone device or sends it to other devices.

[0090] Step S20, performing noise elimination processing on the high-frequency signal in the microphone signal using a preset non-neural network algorithm to obtain a second processed signal, wherein the high-frequency signal is a signal having a frequency greater than the preset frequency;

[0091] Since wind noise mainly affects the low frequency band, in this embodiment, a preset non-neural network algorithm is used to perform noise elimination processing on the high frequency signal in the microphone signal, and the obtained result is called the second processed signal for distinction.

[0092] In this embodiment, high-frequency signals are processed by a non-neural network algorithm, which can reduce the overall amount of computation required by the wind noise suppression device during wind noise suppression processing, thereby reducing the requirements for the hardware computing power of the wind noise suppression device.

[0093] Step S30: fusing the first processed signal and the second processed signal to obtain a wind noise suppression result.

[0094] After separately processing the microphone signals to obtain a first processed signal and a second processed signal, the wind noise suppression device can fuse the first processed signal and the second processed signal to obtain a wind noise suppression result. The fusion can be performed by superposition or weighted fusion. The weights of the weighted fusion can be set as needed and are not limited in this embodiment.

[0095] In a specific embodiment, the wind noise suppression device may fuse the first processed signal and the second processed signal in the time domain to obtain a fused signal in the time domain. This signal is the signal after the microphone signal has undergone wind noise suppression, and the wind noise suppression device uses this signal as the wind noise suppression result. The wind noise suppression device may output the wind noise suppressed signal in the time domain, or further process the signal before outputting it, for example, performing dynamic range control (DRC) on the signal before outputting it.

[0096] In a specific embodiment, when there are multiple microphone signals, the wind noise suppression device may process the multiple microphone signals into a single signal, and then perform noise cancellation on the single signal. For example, the wind noise suppression device may perform beamforming on the multiple microphone signals, and based on the beamforming results, perform noise cancellation on each microphone signal in a preset direction to obtain a directional noise suppression signal, and then perform noise cancellation on the directional noise suppression signal.

[0097] In a specific embodiment, the wind noise suppression device can copy one signal (one microphone signal or one signal obtained by processing multiple microphone signals) into two signals, hereinafter referred to as signal 1 and signal 2. In one feasible embodiment, the wind noise suppression device can use a deep neural network to perform full-band noise elimination processing on signal 1, and then low-pass filter the result after the noise elimination processing, with the upper cutoff frequency of the low-pass filter being a preset frequency, and use the filtered signal as the first processed signal; use a non-neural network algorithm to perform noise elimination processing on signal 2, and then high-pass filter the result after the noise elimination processing, with the lower cutoff frequency of the high-pass filter being a preset frequency, and use the filtered signal as the second processed signal. In another feasible embodiment, the wind noise suppression device can low-pass filter signal 1, with the upper cutoff frequency of the low-pass filter being a preset frequency, and use a deep neural network to perform full-band noise elimination processing on the filtered signal, and use the processed signal as the first processed signal; use a high-pass filter signal 2, with the lower cutoff frequency of the high-pass filter being a preset frequency, and use a non-neural network algorithm to perform full-band noise elimination processing on the filtered signal, and use the processed signal as the second processed signal. In a specific implementation, high-pass filtering and low-pass filtering can be implemented by respectively using a high-pass filter and a low-pass filter formed by connecting five biquad filters in series.

[0098] Furthermore, in a feasible implementation manner, when there are two or more microphone signals, before step S20, the method further includes:

[0099] The remote signal is used to perform echo cancellation on each microphone signal to obtain an echo-cancelled signal; for example, there are two microphone signals represented by microphone signal 1 and microphone signal 2 respectively. The wind noise suppression device uses the remote signal to perform echo cancellation on microphone signal 1 to obtain echo-cancelled signal 1, and uses the remote signal to perform echo cancellation on microphone signal 2 to obtain echo-cancelled signal 2.

[0100] Performing beamforming on each of the echo cancellation signals, and performing noise suppression in a preset direction on each of the echo cancellation signals based on a result of the beamforming to obtain a directional noise suppression signal;

[0101] In this embodiment, a preset deep neural network is used to perform noise elimination processing on the low-frequency signal in the directional noise suppression signal to obtain a first processed signal, and a preset non-neural network algorithm is used to perform noise elimination processing on the high-frequency signal in the directional noise suppression signal to obtain a second processed signal.

[0102] The wind noise suppression device may use a mature algorithm to implement echo cancellation and beamforming on the microphone signal, which is not limited in this embodiment.

[0103] Furthermore, in one feasible implementation, step S10: when it is determined that wind noise exists in the microphone signal, performing noise elimination processing on the low-frequency signal in the microphone signal using a preset deep neural network combined with an attention mechanism, before obtaining the first processed signal, further includes:

[0104] Step S40, performing wind noise analysis on the microphone signal, and determining whether wind noise exists in the microphone signal according to the wind noise analysis result;

[0105] In this embodiment, wind noise analysis is performed on the microphone signal, and whether wind noise exists in the microphone signal is determined based on the wind noise analysis result. In this embodiment, the specific method of wind noise analysis is not limited here.

[0106] In this embodiment, before step S40: performing wind noise analysis on the microphone signal and determining whether wind noise exists in the microphone signal according to the wind noise analysis result, the following steps are further included:

[0107] Step S50 : When it is determined according to the wind noise analysis result that there is no wind noise in the microphone signal, the microphone signal is subjected to noise elimination processing using the non-neural network algorithm to obtain a microphone signal after suppressing wind noise.

[0108] When it is determined based on the wind noise analysis results that there is no wind noise in the microphone signal, the microphone signal can be processed using a non-neural network algorithm to eliminate noise, which can reduce the amount of computation required by the wind noise suppression device and thus reduce the requirements for the hardware computing power of the wind noise suppression device.

[0109] Furthermore, in a feasible implementation manner, the step S40: performing wind noise analysis on the microphone signal includes:

[0110] Step S401, when there are two or more microphone signals, calculating the target correlation between the microphone signals;

[0111] When there are two or more microphone signals, the wind noise suppression device can use the correlation between the microphone signals to perform wind noise analysis. In a specific embodiment, when there are two microphone signals, the wind noise suppression device can directly calculate the correlation between the two microphone signals and use the correlation as the target correlation. When there are more than two microphone signals, the wind noise suppression device can calculate the correlation between each pair of microphone signals, average the correlations (other fusion methods such as addition may also be used) to obtain the target correlation, or directly use all the correlations as the target correlation.

[0112] Specifically, in one feasible implementation, the process of calculating the correlation between the two microphone signals can be: Fourier transform can be performed on the two time-domain microphone signals. For example, after the Fourier transform calculation, the 8 kHz bandwidth is divided into 128 sub-bands. Y1(K) and Y2(K) represent the Fourier transforms of microphone signal 1 and microphone signal 2, respectively. The coherence coefficient within the specified bandwidth is calculated and used as the correlation between microphone signal 1 and microphone signal 2. The calculation formula of the coherence coefficient can be specifically:

[0113]

[0114] In another feasible embodiment, the process of calculating the correlation between the two microphone signals may be: respectively calculating the number of sampling points where the signals are negative in the two microphone signals; and calculating the target correlation between the two microphone signals based on the number of sampling points. For example, in one feasible embodiment, the definition based on x 2 The correlation function of :

[0115]

[0116] Among them, 12 、o 22 are the elements of the following matrix:

[0117]

[0118] in, Indicates the number of sampling points where the microphone signal 1 is positive at time 0 to k. It represents the number of points where the microphone signal 2 has negative values ​​from time 0 to time k, where k is the length of one frame of the microphone signal, N=2k.

[0119] Step S402, obtaining a wind noise analysis result by matching the target correlation and the correspondence between the preset correlation and the wind speed;

[0120] Based on the experimental test results, the correspondence between the correlation between the microphone signals and the wind speed (which can represent the wind noise intensity) can be set in advance in the wind noise suppression device. The correspondence is expressed as follows: when there is wind noise in the microphone signal, or the wind speed is greater, the correlation between the microphone signals is smaller. After the wind noise suppression device calculates the target correlation, it can match the wind noise analysis result according to the preset correspondence between the correlation and the wind speed. For example, when the wind noise analysis result is a result that characterizes whether there is wind noise in the microphone signal, the wind noise suppression device can match the wind speed corresponding to the target correlation according to the correspondence. When the wind speed is greater than a certain wind speed, it is obtained that there is wind noise in the microphone signal.

[0121] In one feasible implementation, when there are more than two microphone signals and multiple target correlations, the wind noise suppression device can also match the wind speed corresponding to each target correlation separately, then calculate the average of each wind speed, and then obtain the wind noise analysis result based on the calculation result.

[0122] In this embodiment, another feasible wind noise analysis implementation is proposed. Step S40: performing wind noise analysis on the microphone signal includes:

[0123] Step S403, calculating the target low-frequency energy of the signal with a frequency lower than the second preset frequency in any microphone signal;

[0124] In this embodiment, when there is only one microphone signal, the wind noise suppression device performs wind noise analysis based on the microphone signal. When there are two or more microphone signals, the wind noise suppression device can select any one microphone signal from the microphone signals for wind noise analysis.

[0125] For one microphone signal, the wind noise suppression device calculates the low-frequency energy (hereinafter referred to as the target low-frequency energy for distinction) of the signal in the microphone signal whose frequency is less than a preset frequency (hereinafter referred to as the second frequency for distinction). The second frequency can be preset as needed, for example, set to 1500HZ. There are many ways to calculate the target low-frequency energy, which are not limited in this embodiment. For example, in one feasible embodiment, the wind noise suppression device can first perform low-pass filtering on the microphone signal, and the upper cutoff frequency of the low-pass filtering is the second frequency. The low-pass filtering can be implemented by, but is not limited to, an IIR (Infinite Impulse Response) filter. Suppose a frame signal after filtering is x1 LP , target low-frequency energy P low It can be calculated as follows:

[0126]

[0127] Where k represents the frame length of a frame of microphone signal.

[0128] Step S404 : According to the target low-frequency energy and the preset correspondence between the low-frequency energy and the wind speed, a wind noise analysis result is obtained by matching.

[0129] The correspondence between low-frequency energy and wind speed (which can represent the intensity of wind noise) can be set in advance in the wind noise suppression device based on experimental test results. The correspondence is expressed as follows: when there is wind noise in the microphone signal, or the wind speed is greater, the low-frequency energy of the signal with a frequency less than the second preset frequency in the microphone signal is greater. After calculating the target low-frequency energy, the wind noise suppression device can match the wind noise analysis result based on the preset correspondence between low-frequency energy and wind speed. For example, when the wind noise analysis result is a result that characterizes whether there is wind noise in the microphone signal, the wind noise suppression device can match the wind speed corresponding to the target low-frequency energy based on the correspondence, and when the wind speed is greater than a certain wind speed, the result of the presence of wind noise in the microphone signal is obtained.

[0130] In this embodiment, when it is determined that there is wind noise in the microphone signal, the low-frequency signal in the microphone signal is subjected to noise elimination processing using a preset deep neural network combined with an attention mechanism to obtain a first processed signal, wherein the low-frequency signal is a signal with a frequency less than or equal to the preset frequency, and the attention mechanism is used to perform weighted processing on the features extracted by the deep neural network; the high-frequency signal in the microphone signal is subjected to noise elimination processing using a preset non-neural network algorithm to obtain a second processed signal, wherein the high-frequency signal is a signal with a frequency greater than the preset frequency; the first processed signal and the second processed signal are fused to obtain a wind noise suppression result.

[0131] This embodiment introduces an attention mechanism into the deep neural network, and performs weighted processing on the low-frequency signal features extracted by the deep neural network based on the attention weight through the attention mechanism. The higher the similarity between the low-frequency signal features and the denoised low-frequency signal features, the greater the attention weight. In this way, the deep neural network can focus on important information in the low-frequency signal, that is, the speech signal and important frequency bands in the speech signal, through the weighted processing of the attention mechanism during the processing of the low-frequency signal, thereby making the deep neural network more accurate in eliminating noise of the low-frequency signal, thereby improving the wind noise suppression effect of the device without increasing the hardware cost and design difficulty of the device.

[0132] Furthermore, based on the above-mentioned first embodiment, a second embodiment of the wind noise suppression method of the present invention is proposed. In this embodiment, the deep neural network includes an encoder, a recurrent neural network module, an attention mechanism module, a decoder and a fully connected layer. The steps of step S20 include:

[0133] Step S201: inputting the low-frequency signal in the microphone signal into the encoder for processing to obtain a first processing result;

[0134] In this embodiment, the preset deep neural network may include an encoder, a recurrent neural network module, an attention mechanism module, a decoder and a fully connected layer. Among them, the encoder layer is used to extract data features and downsample the input microphone signal; the recurrent neural network module is used to process the results output by the encoder layer, and in the processing process, the results of processing the microphone signal of the previous frame will be used, so as to use the information of the historical frame to eliminate noise in the current frame and improve the wind noise suppression effect; the attention mechanism module is used to focus on the key signals in the microphone signal, that is, the voice signal, and the key frequency bands in the voice signal; the decoder is used to upsample the results output by the recurrent neural network; the fully connected layer is used to process the results output by the decoder and output the signal after noise elimination. The deep neural network can be trained in advance through a training data set, and the training method can adopt a conventional neural network training method, which will not be described here.

[0135] In one feasible implementation, the encoder and decoder may refer to the encoder-decoder structure in the U-net network, that is, the decoder is used to implement spanning and upsampling of data features.

[0136] The wind noise suppression device inputs the low-frequency signal in the microphone signal into the encoder for processing to obtain a processing result corresponding to the target microphone signal (hereinafter referred to as the first processing result for distinction).

[0137] Step S202: Input the first processing result into the recurrent neural network module for processing to obtain a second processing result, wherein, when the first processing result is processed by the recurrent neural network module, the result obtained by processing the previous frame processing result by the recurrent neural network module is obtained by processing the previous frame signal of the microphone signal by the encoder;

[0138] When performing noise elimination, the wind noise suppression device processes the received microphone signal in a time sequence. In this embodiment, the previous frame signal of the currently processed microphone signal is called the previous frame signal, and the processing result of the encoder on the previous frame signal is called the previous frame processing result.

[0139] In this embodiment, the first processing result is input into the recurrent neural network module for processing to obtain the second processing result. It can be understood that when the first processing result is processed by the recurrent neural network module, the result is obtained by using the recurrent neural network module to process the previous frame processing result.

[0140] Step S203: inputting the second processing result into the decoder for processing to obtain a third processing result;

[0141] The second processing result is input to the decoder for processing, and the obtained result is called the third processing result for distinction.

[0142] Step S204: input the first processing result and the third processing result into the attention mechanism module for processing to obtain a fourth processing result;

[0143] The first processing result and the third processing result are input into the attention mechanism module for processing, and the obtained result is called the fourth processing result for distinction.

[0144] In a specific embodiment, an attention weight matrix representing the degree of attention to information in the first processing result can be calculated based on the first processing result and the third processing result, and the first processing result can be weighted by the attention weight matrix to obtain the fourth processing result.

[0145] The attention mechanism module performs weighted processing on the first processing result to achieve weight distribution of features in the first processing result, thereby screening out important information in the first processing result and enhancing important information, so that the deep neural network focuses on key information during the decoding process, that is, the speech signal and the key frequency bands of the speech signal, thereby improving the accuracy of noise elimination.

[0146] Step S205 , connecting the third processing result and the fourth processing result, and inputting the connected result into the fully connected layer for processing to obtain a first processed signal.

[0147] The third processing result and the fourth processing result are connected, and the connected result is input into the fully connected layer for processing to obtain a first processed signal.

[0148] In this embodiment, an attention mechanism module is introduced to perform weighted processing on the first processing result, so as to realize weight distribution of features in the first processing result, thereby screening important information in the first processing result and enhancing important information, so that the deep neural network pays attention to key information during the decoding process, that is, the speech signal and the key frequency bands of the speech signal, thereby improving the accuracy of noise elimination.

[0149] Furthermore, in one feasible implementation, before the microphone signal is input into the deep neural network, the microphone signal is subjected to time-frequency transformation to obtain a frequency domain signal, which includes a real part signal (i.e. Figure 3 The REAL signal (shown) and the imaginary signal (i.e. Figure 3IMAGE shown); the real signal and imaginary signal are respectively input into their respective corresponding deep neural networks. It should be noted that the deep neural networks corresponding to the real signal and the imaginary signal have the same structure. The real signal and the imaginary signal processed by the deep neural network undergo an inverse time-frequency transform, and the real signal and the imaginary signal after the inverse time-frequency transform are combined to obtain a complex signal. This complex signal is the microphone signal after wind noise suppression, that is, the wind noise suppression result.

[0150] like Figure 3 As shown in FIG, a schematic diagram of the structure of the deep neural network in this embodiment is drawn, in which R_rnn represents a recurrent neural network, and the encoder (enconde) may include multiple encoding layers (such as Figure 3 The decoder can include multiple decoding layers ( Figure 3 Three decoding layers are drawn in the figure). The decoding layer can be used for deconvolution. The number of attention mechanism modules is the same as the number of encoder layers (that is, the number of decoder layers). Each layer of the encoder and decoder is connected to the attention mechanism module respectively. The output data of the attention mechanism module is connected to the output data of the decoder as the input data of the next layer.

[0151] Among them, the encoding layer can use one-dimensional convolution (1D-conv, that is, Figure 3 conv) + downsampling + activation function as shown in , downsampling can use 2*2 pooling layer, and the activation function can use LeakyRelu, which is defined as follows:

[0152]

[0153] In a specific implementation, the recurrent neural network can be implemented using models such as LSTM and GRU with gate structures to have a stronger ability to suppress the gradient vanishing problem and to more effectively learn causal relationships in data that are far apart in time.

[0154] Furthermore, in one feasible implementation, the attention mechanism module includes a weight calculation block and a weighted processing block, and step S204: inputting the first processing result and the third processing result into the attention mechanism module for processing to obtain a fourth processing result includes:

[0155] Step S2041: Input the first processing result and the third processing result into the weight calculation block for processing to obtain an attention weight matrix;

[0156] In this embodiment, the first processing result and the third processing result are input into the weight calculation block for processing to obtain the attention weight matrix. The specific method of calculating the attention weight matrix is ​​not limited in this embodiment.

[0157] Step S2042: Input the attention weight matrix and the first processing result into the weighted processing block for processing to obtain a fourth processing result.

[0158] The attention weight matrix and the first processing result are input into the weighted processing block for processing to obtain a fourth processing result.

[0159] Furthermore, in one feasible implementation, the weight calculation block includes three convolutional layers, one dot product layer, and two activation function layers. Step S2041: inputting the first processing result and the third processing result into the weight calculation block for processing to obtain an attention weight matrix includes:

[0160] Step S20411: Input the first processing result to a first convolution layer for performing a convolution operation to obtain a first convolution result, and input the third processing result to a second convolution layer for performing a convolution operation to obtain a second convolution result;

[0161] In this embodiment, the first processing result is input into the first convolution layer, and a convolution operation is performed based on the first convolution weight. The obtained result is referred to as the first convolution result for distinction.

[0162] The third processing result is input into the second convolution layer, and a convolution operation is performed based on the second convolution weight. The obtained result is called the second convolution result for distinction.

[0163] Step S20412: Input the first convolution result and the second convolution result into a first dot product layer to calculate a first dot product result, and input the first dot product result into a first activation function for processing to obtain a first activation result;

[0164] The first convolution result and the second convolution result are input to the dot product layer (hereinafter referred to as the first dot product layer for distinction) for calculation, and the result obtained is called the first dot product result for distinction. The first dot product result is input to the first activation function for processing, and the result obtained is called the first activation result for distinction.

[0165] Step S20413: Input the first activation result into the third convolutional layer for convolution operation to obtain a third convolution result, and input the third convolution result into the second activation function for processing to obtain an attention weight matrix.

[0166] The first activation result is input to the third convolution layer, and a convolution operation is performed based on the third convolution weight. The result obtained is called the third convolution result for distinction. The third convolution result is input to the second activation function for processing to obtain the attention weight matrix.

[0167] It should be noted that the first convolution weight, the second convolution weight, and the third convolution weight can be obtained during the model training phase, and the training process is not described in detail here. In this embodiment, feature enhancement is performed through convolution operations, and nonlinear factors are added through activation functions to improve the expressive power of the deep neural network for the model.

[0168] Furthermore, in one feasible implementation, the weighted processing block includes a convolution layer and a dot product layer, and step S2042: inputting the attention weight matrix and the first processing result into the weighted processing block for processing to obtain a fourth processing result includes:

[0169] Step S20421: input the first processing result into a fourth convolutional layer for performing a convolution operation to obtain a fourth convolution result;

[0170] In this embodiment, the first processing result is input to the fourth convolution layer, and a convolution operation is performed based on the fourth convolution weight. The obtained result is referred to as the fourth convolution result for distinction. The fourth convolution weight can be obtained during the model training phase, and the training process is not described in detail here.

[0171] Step S20422: Input the fourth convolution result and the attention weight matrix into the second dot product layer to calculate and obtain a fourth processing result.

[0172] The fourth convolution result and the attention weight matrix are input into the dot product layer (hereinafter referred to as the second convolution layer for distinction) to calculate the fourth processing result.

[0173] Furthermore, in one feasible implementation, the processing of the attention mechanism module can refer to Figure 4 ,like Figure 4 As shown, in this embodiment, a convolution operation is performed on the first processing result e1 based on the first convolution weight W2 to obtain a first convolution result, and a convolution operation is performed on the third processing result d1 based on the second convolution weight W2 to obtain a second convolution result; the dot product of the first convolution result and the second convolution result is calculated to obtain a first dot product result; the first activation function Sigmoid function (that is, Figure 4S) shown in , the first dot product result is processed to obtain a first activation result; based on the third convolution weight W4, a convolution operation is performed on the first activation result to obtain a third convolution result; the third convolution result is processed by the second activation function Relu function to obtain an attention weight matrix; based on the fourth convolution weight W3, a convolution operation is performed on the first processing result e1 to obtain a fourth convolution result; the dot product of the attention weight matrix and the fourth convolution result is calculated to obtain the output data Out1 of the attention mechanism module, that is, the fourth processing result.

[0174] The specific formula of the attention mechanism module's processing process is:

[0175]

[0176] in, Denotes convolution, ⊙ denotes dot product, W1, W2, W3, and W4 are all convolution weights obtained by training.

[0177] Furthermore, in one feasible embodiment, the recurrent neural network module includes at least one recurrent neural network layer connected in series, the recurrent neural network layer including a reset gate and a new memory gate. In this embodiment, the decoder processing result corresponding to the previous frame signal of the currently processed microphone signal is referred to as the previous frame processing result. In this embodiment, step S202: inputting the first processing result into the recurrent neural network module for processing to obtain a second processing result includes:

[0178] The first signal processing result is input into the recurrent neural network module, and after the series connection of each recurrent neural network layer, a second signal processing result corresponding to the first signal processing result is obtained. In this embodiment, the recurrent neural network module includes at least one recurrent neural network layer connected in series, for example, two recurrent neural network layers. Then, the wind noise suppression device inputs the first signal processing result into the first recurrent neural network layer for processing, and the result output by the first recurrent neural network layer is input into the second recurrent neural network layer for processing to obtain a second signal processing result corresponding to the first signal processing result.

[0179] It should be noted that, in each recurrent neural network layer, the input data of the first recurrent neural network layer is the first signal processing result; the input data of recurrent neural network layers other than the first recurrent neural network layer is the output data of the previous recurrent neural network layer. In this embodiment, the term "input data corresponding to the first signal processing result in the recurrent neural network layer" refers to the input data of the recurrent neural network layer when the wind noise suppression device uses the recurrent neural network module to process the first signal processing result. For example, assuming that the recurrent neural network module includes two recurrent neural network layers, the input data corresponding to the first signal processing result in the first recurrent neural network layer is the first signal processing result, and the input data corresponding to the first signal processing result in the second recurrent neural network layer is the result obtained by processing the first signal processing result by the first recurrent neural network layer. Similarly, in this embodiment, the term "output data corresponding to the first signal processing result in the recurrent neural network layer" refers to the output data of the recurrent neural network layer when the wind noise suppression device uses the recurrent neural network module to process the first signal processing result.

[0180] The following is an example of the processing process of a layer of recurrent neural network, and this layer of recurrent neural network is called the target recurrent neural network layer for distinction, and the input data corresponding to the first signal processing result in the target recurrent neural network layer is called the target input data for distinction.

[0181] In the process of processing the first signal processing result in series through each layer of the recurrent neural network layer, the target input data corresponding to the first signal processing result in the target recurrent neural network layer is input into the target recurrent neural network layer for processing, and the step of obtaining the target output data corresponding to the first signal processing result in the target recurrent neural network layer includes:

[0182] The target input data and the output data corresponding to the target recurrent neural network layer of the previous frame processing result are input into the reset gate of the target recurrent neural network layer to obtain the reset gate processing result corresponding to the target input data. Figure 5 For the recurrent neural network layer shown in Figure 1, the expression of the reset gate can be:

[0183] A1(t)=sigmoid(X(t)*W1+Y(t-1)*V1+B1).

[0184] Among them, the symbol * represents matrix multiplication, A1(t) represents the reset gate processing result corresponding to the target input data, X(t) represents the target input data, Y(t-1) represents the output data corresponding to the previous frame processing result in the target recurrent neural network layer, W1, V1, and B1 are the parameters in the reset gate, which can be obtained during the model training phase.

[0185] The target input data and the reset gate processing result corresponding to the target input data, as well as the output data corresponding to the target recurrent neural network layer of the previous frame processing result, are input into the new memory gate of the target recurrent neural network layer to obtain the new memory gate processing result corresponding to the target input data. In a feasible implementation, when adopting the following method, Figure 5 When the recurrent neural network layer is shown, the expression of the new memory gate can be:

[0186]

[0187] Among them, the symbol represents the multiplication of corresponding elements. W2, V2, and B2 are the parameters in the new memory gate, which can be obtained during the model training phase.

[0188] The target output data is calculated based on the new memory gate processing result and the reset gate processing result corresponding to the target input data, as well as the output data corresponding to the target recurrent neural network layer of the previous frame processing result. Figure 5 When the recurrent neural network layer is shown, the target output result is represented as Y(t), which can be calculated using the following expression:

[0189] Y(t)=(1-A1(t))*Y(t-1)+A1(t)*A2(t).

[0190] In one possible implementation, when using Figure 5 During the training of a deep neural network with a recurrent neural network layer as shown, backpropagation can be used to calculate the gradients of each parameter and update each parameter based on the gradients. The gradients of each parameter can be calculated using the following method.

[0191] 1) For the new memory gate:

[0192]

[0193]

[0194]

[0195] Here is w 2，k Do gradient calculation

[0196]

[0197] Similarly

[0198]

[0199]

[0200] in

[0201]

[0202] 2) For the reset gate:

[0203]

[0204]

[0205]

[0206] Here is w 1，k Do gradient calculation

[0207]

[0208] Similarly

[0209]

[0210]

[0211] in

[0212]

[0213] In this embodiment, an attention mechanism module is introduced to perform weighted processing on the first processing result, so as to realize weight distribution of features in the first processing result, thereby screening important information in the first processing result and enhancing important information, so that the deep neural network pays attention to key information during the decoding process, that is, the speech signal and the key frequency bands of the speech signal, thereby improving the accuracy of noise elimination.

[0214] In one possible embodiment, the wind noise suppression device can be configured as follows: Figure 6 The flow shown here performs wind noise suppression.

[0215] 1. The input signals are time domain microphone signal 1 (y1) and time domain microphone signal 2 (y2). The microphone signals can be one or multiple. Here, two signals are used as an example.

[0216] 2. Perform a time-frequency transform on the input time-domain microphone signal. Here, FFT is used to obtain frequency-domain signals Y1(K) and Y2(K). Based on the far-end signal (speaker signal), echo cancellation is performed on the two signals.

[0217] 3. Beamform the two microphone signals to suppress noise other than directivity;

[0218] 4. Determine whether the currently processed signal frame contains wind noise or not based on the two microphone signals;

[0219] 5. If the current frame is determined to be a non-wind noise frame, perform traditional noise removal on the microphone signal;

[0220] 6. If the current frame is determined to be a wind noise frame, DNN-based noise removal is performed on the low-frequency signal and traditional noise processing is performed on the high-frequency signal;

[0221] 7. High-pass filter the traditional noise-processed time-domain microphone signal to obtain the output signal out1;

[0222] 8. Low-pass filter the noise-processed signal of the DNN to obtain the output signal out2;

[0223] 9. The fused signal out = k1*out1+k2*out2, where k1 and k2 are weights pre-set as needed;

[0224] 10. Dynamic range control (DRC) is performed on signals under both wind noise and non-wind noise conditions;

[0225] 11. Output the final time domain signal out.

[0226] In addition, an embodiment of the present invention further provides a wind noise suppression device, the device comprising:

[0227] a noise cancellation module for, when determining that wind noise exists in a microphone signal, performing noise cancellation processing on a low-frequency signal in the microphone signal using a preset deep neural network combined with an attention mechanism to obtain a first processed signal, wherein the low-frequency signal is a signal with a frequency less than or equal to a preset frequency; and in the process of performing noise cancellation processing on the low-frequency signal using the deep neural network, performing weighted processing on low-frequency signal features extracted by the deep neural network based on an attention weight using the attention mechanism, wherein the higher the similarity between the low-frequency signal features and the denoised low-frequency signal features, the greater the attention weight;

[0228] The noise elimination module is further configured to perform noise elimination processing on a high-frequency signal in the microphone signal using a preset non-neural network algorithm to obtain a second processed signal, wherein the high-frequency signal is a signal with a frequency greater than the preset frequency;

[0229] A fusion module is used to fuse the first processed signal and the second processed signal to obtain a wind noise suppression result.

[0230] Furthermore, the deep neural network includes an encoder, a recurrent neural network module, an attention mechanism module, a decoder and a fully connected layer, and the noise elimination module is further used to:

[0231] Inputting the low-frequency signal in the microphone signal into the encoder for processing to obtain a first processing result;

[0232] Inputting the first processing result into the recurrent neural network module for processing to obtain a second processing result, wherein, when the first processing result is processed by the recurrent neural network module, a result obtained by processing a previous frame processing result by the recurrent neural network module is used, and the previous frame processing result is a result obtained by the encoder processing a previous frame signal of the microphone signal;

[0233] Inputting the second processing result into the decoder for processing to obtain a third processing result;

[0234] Inputting the first processing result and the third processing result into the attention mechanism module for processing to obtain a fourth processing result;

[0235] The third processing result and the fourth processing result are connected, and the connected result is input into the fully connected layer for processing to obtain a first processed signal.

[0236] Furthermore, the attention mechanism module includes a weight calculation block and a weighted processing block, and the noise elimination module is further used to:

[0237] Inputting the first processing result and the third processing result into the weight calculation block for processing to obtain an attention weight matrix;

[0238] The attention weight matrix and the first processing result are input into the weighted processing block for processing to obtain a fourth processing result.

[0239] Furthermore, the weight calculation block includes three convolution layers, one dot product layer and two activation functions, and the noise elimination module is further used to:

[0240] Inputting the first processing result into a first convolutional layer for performing a convolution operation to obtain a first convolution result, and inputting the third processing result into a second convolutional layer for performing a convolution operation to obtain a second convolution result;

[0241] Inputting the first convolution result and the second convolution result into a first dot product layer to calculate a first dot product result, and inputting the first dot product result into a first activation function for processing to obtain a first activation result;

[0242] The first activation result is input into the third convolutional layer for convolution operation to obtain a third convolution result, and the third convolution result is input into the second activation function for processing to obtain an attention weight matrix.

[0243] Furthermore, the weighted processing block includes a convolution layer and a dot product layer, and the noise elimination module is further configured to:

[0244] Inputting the first processing result into a fourth convolutional layer for performing a convolution operation to obtain a fourth convolution result;

[0245] The fourth convolution result and the attention weight matrix are input into the second dot product layer to calculate the fourth processing result.

[0246] Furthermore, the wind noise suppression module further includes a wind noise analysis module, which is configured to:

[0247] performing a wind noise analysis on the microphone signal, and determining whether wind noise exists in the microphone signal according to the wind noise analysis result;

[0248] The noise cancellation module is further configured to:

[0249] When it is determined according to the wind noise analysis result that no wind noise exists in the microphone signal, the microphone signal is subjected to noise elimination processing using the non-neural network algorithm to obtain a microphone signal after suppressing wind noise.

[0250] Furthermore, the wind noise analysis module is also used to:

[0251] When there are two or more microphone signals, calculating the target correlation between the microphone signals;

[0252] According to the target correlation and the corresponding relationship between the preset correlation and the wind speed, a wind noise analysis result is obtained by matching;

[0253] Or, calculating the target low-frequency energy of a signal having a frequency less than a second preset frequency in any microphone signal;

[0254] According to the target low-frequency energy and the corresponding relationship between the preset low-frequency energy and the wind speed, a wind noise analysis result is obtained by matching.

[0255] The various embodiments of the wind noise suppression device of the present invention can refer to the various embodiments of the wind noise suppression method of the present invention, and will not be repeated here.

[0256] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a wind noise suppression program is stored. When the wind noise suppression program is executed by a processor, the steps of the wind noise suppression method described below are implemented.

[0257] The various embodiments of the wind noise suppression device and the computer-readable storage medium of the present invention may refer to the various embodiments of the wind noise suppression method of the present invention, and will not be described in detail here.

[0258] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0259] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0260] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0261] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for suppressing wind noise, characterized in that: The wind noise suppression method comprises the following steps: When it is determined that wind noise exists in the microphone signal, a low-frequency signal in the microphone signal is subjected to noise elimination processing using a preset deep neural network combined with an attention mechanism to obtain a first processed signal, wherein the low-frequency signal is a signal with a frequency less than or equal to a preset frequency; in the process of performing noise elimination processing on the low-frequency signal using the deep neural network, a low-frequency signal feature extracted by the deep neural network is weighted based on an attention weight using the attention mechanism, wherein the higher the similarity between the low-frequency signal feature and the denoised low-frequency signal feature, the greater the attention weight; Performing noise elimination processing on a high-frequency signal in the microphone signal using a preset non-neural network algorithm to obtain a second processed signal, wherein the high-frequency signal is a signal with a frequency greater than the preset frequency; The first processed signal and the second processed signal are fused to obtain a wind noise suppression result.

2. The wind noise suppression method according to claim 1, wherein: The deep neural network includes an encoder, a recurrent neural network module, an attention mechanism module, a decoder, and a fully connected layer. The step of performing noise elimination processing on the low-frequency signal in the microphone signal using a preset deep neural network combined with an attention mechanism to obtain a first processed signal includes: Inputting the low-frequency signal in the microphone signal into the encoder for processing to obtain a first processing result; Inputting the first processing result into the recurrent neural network module for processing to obtain a second processing result, wherein, when the first processing result is processed by the recurrent neural network module, a result obtained by processing a previous frame processing result by the recurrent neural network module is used, and the previous frame processing result is a result obtained by the encoder processing a previous frame signal of the microphone signal; Inputting the second processing result into the decoder for processing to obtain a third processing result; Inputting the first processing result and the third processing result into the attention mechanism module for processing to obtain a fourth processing result; The third processing result and the fourth processing result are connected, and the connected result is input into the fully connected layer for processing to obtain a first processed signal.

3. The wind noise suppression method according to claim 2, wherein: The attention mechanism module includes a weight calculation block and a weighted processing block, and the step of inputting the first processing result and the third processing result into the attention mechanism module for processing to obtain the fourth processing result includes: Inputting the first processing result and the third processing result into the weight calculation block for processing to obtain an attention weight matrix; The attention weight matrix and the first processing result are input into the weighted processing block for processing to obtain a fourth processing result.

4. The wind noise suppression method according to claim 3, wherein: The weight calculation block includes three convolution layers, one dot product layer, and two activation functions. The step of inputting the first processing result and the third processing result into the weight calculation block for processing to obtain the attention weight matrix includes: Inputting the first processing result into a first convolutional layer for performing a convolution operation to obtain a first convolution result, and inputting the third processing result into a second convolutional layer for performing a convolution operation to obtain a second convolution result; Inputting the first convolution result and the second convolution result into a first dot product layer to calculate a first dot product result, and inputting the first dot product result into a first activation function for processing to obtain a first activation result; The first activation result is input into the third convolutional layer for convolution operation to obtain a third convolution result, and the third convolution result is input into the second activation function for processing to obtain an attention weight matrix.

5. The wind noise suppression method according to claim 3, wherein: The weighted processing block includes a convolution layer and a dot product layer, and the step of inputting the attention weight matrix and the first processing result into the weighted processing block for processing to obtain a fourth processing result includes: Inputting the first processing result into a fourth convolutional layer for performing a convolution operation to obtain a fourth convolution result; The fourth convolution result and the attention weight matrix are input into the second dot product layer to calculate the fourth processing result.

6. The wind noise suppression method according to any one of claims 1 to 5, characterized in that: Before the step of performing noise elimination processing on the low-frequency signal in the microphone signal using a preset deep neural network combined with an attention mechanism to obtain a first processed signal when it is determined that wind noise exists in the microphone signal, the method further includes: performing a wind noise analysis on the microphone signal, and determining whether wind noise exists in the microphone signal according to the wind noise analysis result; After the step of performing wind noise analysis on the microphone signal and determining whether wind noise exists in the microphone signal according to the wind noise analysis result, the method further includes: When it is determined according to the wind noise analysis result that no wind noise exists in the microphone signal, the microphone signal is subjected to noise elimination processing using the non-neural network algorithm to obtain a microphone signal after suppressing wind noise.

7. The wind noise suppression method according to claim 6, wherein: The step of performing wind noise analysis on the microphone signal and determining whether wind noise exists in the microphone signal according to the wind noise analysis result comprises: When there are two or more microphone signals, calculating the target correlation between the microphone signals; According to the target correlation and the corresponding relationship between the preset correlation and the wind speed, a wind noise analysis result is obtained by matching; Or, calculating the target low-frequency energy of a signal having a frequency less than a second preset frequency in any microphone signal; According to the target low-frequency energy and the corresponding relationship between the preset low-frequency energy and the wind speed, a wind noise analysis result is obtained by matching.

8. A wind noise suppression device, characterized in that: The device comprises: a noise cancellation module for, when determining that wind noise exists in a microphone signal, performing noise cancellation processing on a low-frequency signal in the microphone signal using a preset deep neural network combined with an attention mechanism to obtain a first processed signal, wherein the low-frequency signal is a signal with a frequency less than or equal to a preset frequency; and in the process of performing noise cancellation processing on the low-frequency signal using the deep neural network, performing weighted processing on low-frequency signal features extracted by the deep neural network based on an attention weight using the attention mechanism, wherein the higher the similarity between the low-frequency signal features and the denoised low-frequency signal features, the greater the attention weight; The noise elimination module is further configured to perform noise elimination processing on a high-frequency signal in the microphone signal using a preset non-neural network algorithm to obtain a second processed signal, wherein the high-frequency signal is a signal with a frequency greater than the preset frequency; A fusion module is used to fuse the first processed signal and the second processed signal to obtain a wind noise suppression result.

9. A wind noise suppression device, characterized in that: The wind noise suppression device includes: a memory, a processor, and a wind noise suppression program stored in the memory and executable on the processor. When the wind noise suppression program is executed by the processor, the steps of the wind noise suppression method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a wind noise suppression program, which, when executed by a processor, implements the steps of the wind noise suppression method according to any one of claims 1 to 7.

Images