Wind noise pollution range estimation method and suppression method, device, medium, and terminal

Abstract

A method for estimating and suppressing wind noise pollution range, a device, a medium, and a terminal. The wind noise pollution range estimation method includes: obtaining multiple frames of collected signals, each frame of collected signals including signals collected by each microphone in the multi-microphone device; performing wind noise detection on the signals collected by each microphone, and filtering out wind noise frames from the multiple frames of collected signals; for the wind noise frames, estimating the wind noise pollution range of each microphone based on the low-frequency energy of the signals collected by each microphone, a set wind noise boundary threshold, and a conversion coefficient, wherein the wind noise pollution range is used to indicate the frequency band contaminated by wind noise, and the conversion coefficient is used to characterize the relationship between the wind noise pollution range and the wind noise energy. The above scheme can accurately estimate the wind noise pollution range of each wind noise frame, thereby improving the rationality and accuracy of the wind noise pollution range determination. This can also improve the wind noise suppression effect when wind noise suppression is performed on the wind noise frames.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of signal processing technology, and in particular to a method for estimating and suppressing wind noise pollution range, a device, a medium, and a terminal. Background Art

[0002] In voice calls and hearing aid applications, when the microphone is exposed to wind, airflow disturbances can cause non-steady-state vibrations of the microphone diaphragm, resulting in a loud and subjectively unpleasant noise. This noise is generally referred to as wind noise (also known as wind noise). Wind noise has a significant negative impact on call quality and can reduce speech intelligibility.

[0003] Currently, a fixed cutoff frequency is usually set, and wind noise suppression is performed within the fixed cutoff frequency range, or full-band processing is performed.

[0004] However, using a fixed cutoff frequency can leave some high-frequency components of strong wind noise. Using full-band processing to suppress wind noise in areas with no wind noise but with speech can result in loss of high-frequency information in the speech. In summary, existing noise reduction methods for wind noise suppression have limited rationality and accuracy in the frequency bands used. Summary of the Invention

[0005] An object of the embodiments of the present invention is to provide a method for estimating the wind noise pollution range of a multi-microphone device, which can improve the rationality and accuracy of determining the wind noise pollution range.

[0006] To achieve the above-mentioned purpose, an embodiment of the present invention provides a method for estimating the wind noise pollution range of a multi-microphone device, comprising: obtaining multiple frames of acquisition signals, each frame of acquisition signals including signals collected by each microphone in the multi-microphone device; performing wind noise detection on the signals collected by each microphone, and screening out wind noise frames from the multiple frames of acquisition signals; for the wind noise frames, estimating the wind noise pollution range of each microphone based on the low-frequency energy of the signals collected by each microphone, a set wind noise boundary threshold and a conversion coefficient, the wind noise pollution range is used to indicate the frequency band contaminated by wind noise, and the conversion coefficient is used to characterize the relationship between the wind noise pollution range and the wind noise energy.

[0007] Optionally, the conversion coefficient is determined in the following manner: obtaining the type of each microphone in the multi-microphone device; and determining the conversion coefficient corresponding to each microphone according to the type of each microphone.

[0008] Optionally, the conversion coefficient corresponding to each microphone is determined according to the type of each microphone: for each type of microphone, the low-frequency energy of the signal collected by the microphone at different wind speeds is tested, and the frequency range corresponding to the low-frequency energy is obtained; the low-frequency energy of the signal collected at each wind speed and the corresponding frequency range are fitted to obtain the conversion coefficient.

[0009] Optionally, the wind noise pollution range of each microphone is determined based on the low-frequency energy of the signal collected by each microphone, the set wind noise boundary threshold and the conversion coefficient, including: for each microphone, calculating the difference between the low-frequency energy of the signal collected by each microphone and the set wind noise boundary threshold; calculating the quotient of the difference and the conversion coefficient, and determining the wind noise pollution range based on the obtained quotient.

[0010] Optionally, the wind noise pollution range estimation method of the multi-microphone device also includes: for each microphone, based on the signal collected by each microphone, determining the energy of all noises and the energy of other noise signals included in the signal collected by each microphone, the other noise refers to the noise other than wind noise in all the noises; according to the ratio of the energy of the other noise to the energy of the all the noises, the wind noise boundary threshold is corrected.

[0011] An embodiment of the present invention also provides a wind noise suppression method for a multi-microphone device, comprising: using any of the above-mentioned wind noise pollution range estimation methods for a multi-microphone device to estimate the wind noise pollution range of each microphone; for the wind noise frame, selecting one or more microphones from all microphones as target microphones based on the wind noise pollution range of each microphone; and performing wind noise suppression processing on the signal collected by the target microphone within the wind noise pollution range of the target microphone.

[0012] Optionally, for the wind noise frame, one or more microphones are selected from all microphones as target microphones according to the wind noise pollution range of each microphone, including: for the wind noise frame, one or more microphones are selected as target microphones in order from small to large according to the wind noise pollution range of each microphone.

[0013] An embodiment of the present invention also provides a wind noise pollution range estimation device for a multi-microphone device, including: an acquisition unit, used to acquire multiple frames of acquisition signals, each frame of acquisition signals including signals collected by each microphone in the multi-microphone device; a wind noise detection unit, used to perform wind noise detection on the signals collected by each microphone, and filter out wind noise frames from the multiple frames of acquisition signals; a wind noise pollution range estimation unit, used to estimate the wind noise pollution range of each microphone for the wind noise frame based on the low-frequency energy of the signals collected by each microphone, a set wind noise boundary threshold and a conversion coefficient, the wind noise pollution range being used to indicate the frequency band contaminated by wind noise, and the conversion coefficient being used to characterize the relationship between the wind noise pollution range and the wind noise energy.

[0014] An embodiment of the present invention also provides a wind noise suppression device for a multi-microphone device, comprising: a wind noise pollution range estimation device for any of the above-mentioned multi-microphone devices; a wind noise suppression unit, configured to perform wind noise suppression processing on the signals collected by each microphone using a wind noise suppression gain within the estimated wind noise pollution range of each microphone.

[0015] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes the steps of any of the above-mentioned methods for estimating the wind noise pollution range of a multi-microphone device, or the steps of any of the above-mentioned methods for suppressing wind noise of a multi-microphone device.

[0016] An embodiment of the present invention also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor runs the computer program, it executes the steps of any of the above-mentioned methods for estimating the wind noise pollution range of a multi-microphone device, or the steps of any of the above-mentioned methods for suppressing wind noise of a multi-microphone device.

[0017] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0018] The wind noise contamination range estimation method for a multi-microphone device provided in an embodiment of the present invention determines a wind noise frame by performing wind noise detection on the signals collected by each microphone in a multi-frame collected signal. For each wind noise frame, the contamination range of each microphone is estimated based on the low-frequency energy of the signal collected by each microphone, a set wind noise boundary threshold, and a conversion coefficient, thereby determining the frequency band contaminated by wind noise in the wind noise frame. Because the wind noise contamination range of each microphone is determined for each wind noise frame based on the low-frequency energy of the signal collected by each microphone corresponding to each wind noise frame, the contamination range of the microphone corresponding to each wind noise frame is adapted to the actual wind noise impact of each wind noise frame, thereby improving the rationality and accuracy of the determination of the wind noise contamination range of each wind noise frame.

[0019] In addition, when wind noise suppression is subsequently performed on the wind noise frame based on the wind noise pollution range of each microphone, the wind noise suppression effect during noise reduction can be improved.

[0020] Furthermore, since the wind noise pollution range can be accurately determined for each wind noise frame, when the wind noise suppression process is performed on the wind noise frame, both transient wind noise and continuous wind noise can be suppressed effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a method for estimating wind noise pollution range of a multi-microphone device in an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of a wind noise pollution range in an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of an estimated wind noise pollution range in an embodiment of the present invention;

[0024] Figure 4 This is a flow chart of a method for suppressing wind noise using a multi-microphone device according to an embodiment of the present invention.

[0025] Figure 5 It is a schematic diagram of the mapping relationship between the coherence coefficient and the wind noise suppression gain;

[0026] Figure 6 This is another diagram of the mapping relationship between the coherence coefficient and the wind noise suppression gain;

[0027] Figure 7 1 is a schematic structural diagram of a device for estimating wind noise pollution range using a multi-microphone device according to an embodiment of the present invention;

[0028] Figure 8 4 is a schematic structural diagram of a wind noise suppression device of a multi-microphone device in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] As mentioned above, currently, a fixed cutoff frequency is usually set, and wind noise suppression is performed within the fixed cutoff frequency range, or full-band processing is performed. However, due to the limited pollution range of wind noise, the use of a fixed cutoff frequency will result in residual high-frequency parts of wind noise for some very strong wind noise. When full-band processing is used, wind noise suppression in places where there is little or no wind noise and there is speech will cause loss of high-frequency information of the speech. In summary, when existing noise reduction methods are used to suppress wind noise, the wind noise suppression effect obtained is poor.

[0030] To solve the above problems, an embodiment of the present invention provides a method for estimating the wind noise pollution range of a multi-microphone device. By performing wind noise detection on the signals collected by each microphone in the multi-frame collected signal, a wind noise frame is determined. For the wind noise frame, the pollution range of each microphone is estimated based on the low-frequency energy of the signal collected by each microphone, the set wind noise boundary threshold and the conversion coefficient, so as to determine the frequency band contaminated by wind noise in the wind noise frame. Since the wind noise pollution range of each microphone is determined according to the low-frequency energy of the signal collected by each microphone corresponding to each wind noise frame for each wind noise frame, the pollution range of the microphone corresponding to each wind noise frame obtained is adapted to the actual wind noise impact of each wind noise frame, thereby improving the rationality and accuracy of the determination of the wind noise pollution range of each wind noise frame. Therefore, when wind noise suppression is subsequently performed on the wind noise frame based on the wind noise pollution range of each microphone, the wind noise suppression effect during noise reduction can be improved.

[0031] In order to make the above-mentioned objects, features and beneficial effects of the embodiments of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0032] An embodiment of the present invention provides a method for estimating the wind noise pollution range of a multi-microphone device. The method can be executed by a terminal, or by a chip or chip module in the terminal that has a wind noise pollution range estimation function, or by a chip or chip module in the terminal that has a data processing function, or by a baseband chip in the terminal. The terminal can be a multi-microphone device, or other terminal device such as a mobile phone, computer, tablet computer, server, cloud platform, etc. for controlling a multi-microphone device. The multi-microphone device can include a communication device, headset, hearing aid, vehicle-mounted terminal, or other device with multiple microphones.

[0033] Reference Figure 1 , a flowchart of a method for estimating the wind noise pollution range of a multi-microphone device in an embodiment of the present invention is provided. The method for estimating the wind noise pollution range may specifically include the following steps:

[0034] Step 11, obtaining multiple frames of acquisition signals, each frame of the acquisition signal including a signal respectively acquired by each microphone in the multi-microphone device;

[0035] Step 12: Perform wind noise detection on the signals collected by each microphone, and filter out wind noise frames from the multiple frames of collected signals;

[0036] Step 13: For the wind noise frame, estimate the wind noise contamination range of each microphone based on the low-frequency energy of the signal collected by each microphone, the set wind noise boundary threshold, and the conversion coefficient. The wind noise contamination range is used to indicate the frequency band contaminated by wind noise, and the conversion coefficient is used to characterize the relationship between the wind noise contamination range and the wind noise energy.

[0037] The wind noise pollution range is positively correlated with the wind noise energy. Generally, the greater the wind noise energy, the larger the wind noise pollution range.

[0038] Wind noise comes from the vortices and turbulence of airflow. It is non-steady and unpredictable, and is different from any other sound sources in the surrounding area. It is an independent event. Other human voices or noises in the surrounding area are all generated by the vibrations of a specific sound source. Research has found that for multi-microphone equipment, this coherence of "generated by the same sound source" and the incoherence of wind noise "generated independently" can be used to distinguish wind noise from other noises. Then, different signal processing methods are used to reduce noise in the presence and absence of wind noise.

[0039] Based on the mechanism of wind noise generation, for each frame of acquired signals, the signal collected by the first microphone in that frame is recorded as the first signal, and the signal collected by the second microphone is recorded as the second signal. For a frame of acquired signals, if wind noise is present, the coherence between the first and second signals in that frame of acquired signals is poor. Correspondingly, if wind noise is absent, the coherence between the first and second signals in that frame of acquired signals is good.

[0040] In some embodiments of the present invention, in step 12, wind noise can be detected using amplitude-squared coherence. Specifically, wind noise is detected using amplitude-squared coherence, and a corresponding coherence coefficient is obtained. A larger coherence coefficient indicates that the sampled signal is less affected by wind noise, that is, the sampled signal contains less signal corresponding to wind noise. A smaller coherence coefficient indicates that the sampled signal is more affected by wind noise, that is, the sampled signal contains more signal corresponding to wind noise.

[0041] For ease of understanding, the specific implementation of step 12 is described below by taking a multi-microphone device including two microphones as an example.

[0042] Taking the mth frame acquisition signal as an example, the mth frame acquisition signal includes a first signal and a second signal. The first signal and the second signal are converted from the time domain to the frequency domain to obtain a first frequency domain signal corresponding to the first signal and a second frequency domain signal corresponding to the second signal. The amplitude square coherence method is used to perform coherence detection based on the first frequency domain signal and the second frequency domain signal to obtain the coherence coefficient of the mth frame acquisition signal within a set low-frequency range. The coherence coefficient within the low-frequency range is used to characterize the significant frequency domain coherence of the first signal and the second signal within the low-frequency range. The coherence coefficient can be used to determine whether the acquisition signal includes wind noise, thereby determining whether each frame acquisition signal is a wind noise frame, thereby filtering out wind noise frames from each frame acquisition signal. The low-frequency range refers to frequencies below the set low-frequency cutoff frequency. As a preferred embodiment, the set low-frequency cutoff frequency is 120 Hz. It is understood that other frequencies can also be selected as the low-frequency cutoff frequency based on actual needs.

[0043] Each frame of acquired signal typically refers to a signal within a set duration. When converting the acquired signal from the time domain to the frequency domain, multiple frequency points are set. For example, the coherence coefficients of the first and second frequency domain signals are calculated for each frequency point in each frame. Based on the coherence coefficients of each frequency point in each frame, the coherence coefficient of the acquired signal for each frame is obtained.

[0044] When calculating the coherence coefficient of the first frequency domain signal and the second frequency domain signal, the first frequency domain signal and the second frequency domain signal can be processed to obtain a first complex spectrum corresponding to the first frequency domain signal and a second complex spectrum corresponding to the second frequency domain signal. The coherence coefficient of the first complex spectrum and the second complex spectrum is calculated. It will be appreciated that the coherence coefficient of the first frequency domain signal and the second frequency domain signal can also be calculated based on other types of frequency domain signals.

[0045] In some non-limiting embodiments, for each frame of acquired signal, the average value of the coherence coefficients of each frequency point of each frame of acquired signal may be used as the coherence coefficient of each frame of acquired signal.

[0046] In some non-limiting embodiments, continuing to take the mth frame acquisition signal as an example, a first complex spectrum X1 is obtained after the first frequency domain signal is processed, and a second complex spectrum X2 is obtained after the second frequency domain signal is processed. The coherence coefficient of the first signal and the second signal is calculated based on the first complex spectrum X1 and the second complex spectrum X2.

[0047] For example, the squared amplitude coherence is used to detect wind noise and calculate the coherence coefficient. Specifically, the coherence coefficient can be calculated using the following formulas (1) to (5), where the coherence coefficient of each frame of the sampling signal is the average of the coherence coefficients corresponding to multiple frequency points in each frame of the sampling signal.

[0048]

[0049]

[0050]

[0051]

[0052]

[0053] Among them, coh_mean(m) is the coherence coefficient of the mth frame; Φ 11 is the autospectral density of the first microphone; Φ 11 (m,k) is the autospectral density of the kth frequency point in the mth frame of the first microphone; α is the smoothing coefficient, which can be in the range of [0.5, 0.95]; X1(m,k) is the complex spectrum of the kth frequency point in the mth frame of the first microphone; Φ 12 is the cross-spectral density between the first microphone and the second microphone; Φ 12 (m,k) is the cross-spectral density between the first microphone and the second microphone at the kth frequency point in the mth frame; Φ 12 (m-1,k) is the cross-spectral density between the kth frequency point of the m-1th frame of the first microphone and the second microphone; Φ 22 is the autospectral density of the second microphone; Φ 22 (m,k) is the autospectral density of the kth frequency point in the mth frame of the second microphone; Φ 22 (m-1,k) is the autospectral density of the kth frequency point in the m-1th frame of the second microphone; X2(m,k) is the complex spectrum of the kth frequency point in the mth frame of the second microphone; coh(m,k) is the coherence coefficient of the kth frequency point in the mth frame; coh_mean(m) is the mean coherence coefficient in the low-frequency range of the mth frame; N Low is the set low-frequency cutoff frequency; [1,N Low ] is the set low frequency range; To calculate the [1,N Low ] the mean value of coh(m,k) at frequency point k on ; is the conjugate variable corresponding to X1(m,k); is the conjugate variable corresponding to X2(m,k); is Φ 12 The conjugate variable corresponding to (m,k), where m is a positive integer greater than 1 and k is a positive integer.

[0054] In a specific implementation, after obtaining the coherence coefficient of each frame of the acquired signal, it is possible to determine whether each frame of the acquired signal is a wind noise frame based on the relationship between the coherence coefficient of each frame of the acquired signal and a set coherence coefficient threshold. For example, if the coherence coefficient of the acquired signal is less than the set coherence coefficient threshold, the acquired signal frame is determined to be a wind noise frame.

[0055] In a specific implementation, the coherence coefficient threshold can be configured according to the requirements for noise reduction processing accuracy, which is not limited here.

[0056] Research has found that, given the microphone type, wind speed is correlated with the extent of wind noise pollution. The magnitude of wind speed can be reflected in low-frequency energy. Low-frequency energy refers to energy within the low-frequency range. Therefore, the extent of wind noise pollution can be determined based on the low-frequency energy of the collected signal within this range.

[0057] In some embodiments, low-frequency energy can be characterized by spectral energy or spectral amplitude. When spectral energy is used to characterize low-frequency energy, the set wind noise boundary threshold is the set spectral energy threshold. When spectral amplitude is used to characterize low-frequency energy, the set wind noise boundary threshold is the set spectral amplitude threshold.

[0058] In some non-limiting embodiments, the spectrum amplitude may be characterized by a low-frequency amplitude spectrum.

[0059] Furthermore, the spectrum amplitude uses the low-frequency average amplitude spectrum, which is the average of the low-frequency amplitude spectra at each frequency point in each frame within the set low-frequency range. Using the low-frequency average amplitude spectrum can reduce errors caused by fluctuations, improve the accuracy of low-frequency energy estimation, and thus help improve the accuracy of determining the wind noise pollution range.

[0060] In some embodiments, the conversion coefficient is determined in the following manner: Specifically, the type of each microphone in the multi-microphone device is obtained; and the conversion coefficient corresponding to each microphone is determined according to the type of each microphone.

[0061] Furthermore, for various types of microphones, the low-frequency energy of the signal collected by the microphone at different wind speeds is tested, and the frequency range corresponding to the low-frequency energy is obtained; the low-frequency energy of the signal collected at each wind speed and the corresponding frequency range are fitted to obtain the conversion coefficient. The conversion coefficient can be used to characterize the mapping relationship between the wind noise pollution range and the wind noise energy (low-frequency energy) of the signal collected at each wind speed. The mapping relationship can be a measured curve or an approximate equation obtained by fitting the measured curve. For the convenience of calculation, the approximate equation is often expressed as a polynomial relationship, which can accurately estimate the wind noise pollution range. Among them, the wind noise pollution range can be a spectrum pollution range.

[0062] Wind noise energy is typically concentrated in low frequencies. Since the wind noise contamination range is positively correlated with wind noise energy, the higher the wind speed, the higher the low-frequency energy, the larger the wind noise contamination range (the range of contaminated frequencies), and the greater the probability of high-frequency contamination. By fitting the low-frequency energy of the signal collected at various wind speeds and the corresponding frequency range, the resulting conversion coefficient can be used to characterize the wind noise contamination range generated by a microphone of that type under different wind noise energies.

[0063] The frequency range corresponding to the low-frequency energy can be determined based on a set wind noise boundary threshold (such as a spectral energy threshold or a spectral amplitude threshold), or can be determined manually based on a spectrogram related to the low-frequency energy. The specific method for determining the frequency range corresponding to the low-frequency energy is not limited herein.

[0064] In some non-limiting implementations, for some types of microphones, the mapping relationship between wind noise pollution range and wind noise energy (low-frequency energy) at different wind speeds does not change much. In this case, the conversion coefficient can be simplified to a constant value.

[0065] Taking the first microphone Mic1 as an example, on the logarithmic spectrum of the first microphone Mic1, the contamination range of wind noise can be approximately linear. Figure 2 , a schematic diagram of the wind noise pollution range in an embodiment of the present invention is provided. Assuming that the slope of straight line L is k(m), the slope k(m) is related to the characteristics of the first microphone itself and the wind speed. When determining the selection of the first microphone, it can be approximately considered that the slope is only related to the wind speed. The magnitude of the wind speed can be reflected in the low-frequency energy, and the magnitude of the wind speed is related to the wind noise pollution range. Therefore, the wind noise pollution range of each microphone can be estimated based on the low-frequency energy. Among them, the gray area 0-W is the wind noise pollution range.

[0066] In a specific implementation, taking the mth frame signal as an example, the low-frequency average amplitude spectrum of the mth frame signal collected by the first microphone can be calculated using the following formula (6), and the low-frequency average amplitude spectrum of the mth frame signal collected by the second microphone can be calculated using the following formula (7).

[0067]

[0068]

[0069] Among them, E Low1 (m) is the low-frequency average amplitude spectrum of the mth frame of the first microphone, which reflects the energy of wind noise at low frequencies; X1(m,k) is the complex spectrum of the kth frequency point of the mth frame of the first microphone; E Low2 (m) is the low-frequency average amplitude spectrum of the mth frame of the second microphone; X2(m,k) is the complex spectrum of the kth frequency point of the mth frame of the second microphone; NLow1 N is the low-frequency cutoff frequency point of the low-frequency range when calculating the low-frequency average amplitude spectrum of the first microphone; Low2 is the low-frequency cutoff frequency point of the low-frequency range when calculating the low-frequency average amplitude spectrum of the second microphone; |X2(m,k)| is the absolute value of X2(m,k); |X1(m,k)| is the absolute value of X1(m,k).

[0070] It should be noted that the low-frequency average amplitude spectrum is defined on the logarithmic spectrum. The logarithm can be based on the natural logarithm e, 10, or other numbers. Numbers with different bases are related to each other through the base conversion formula. The above formulas (6) and (7) use base 2 as an example and use the mean method to obtain the low-frequency average amplitude spectrum, reducing the error caused by fluctuations.

[0071] Taking the first microphone as an example, when calculating the low-frequency average amplitude spectrum, the average of the low-frequency amplitude spectra corresponding to the first several frequency points is generally taken. The above formula takes 1~N Low1 The average value of the low-frequency amplitude spectrum of the frequency points within the range. In practice, it can also be taken as 2~N Low1 The mean of the low-frequency amplitude spectrum of the frequency points within the range.

[0072] Taking the first microphone as an example, yes Figure 2 A point on the black line L, E Low1 (m) reflects the energy of wind noise at low frequencies, which is represented by the low-frequency average amplitude spectrum. In fact, it is very small. For the convenience of calculation, we take E Low1 (m) can also be approximately considered as the intercept of the black line on the ordinate.

[0073] In a specific implementation of step 13, for each microphone, the difference between the low-frequency energy of the signal collected by each microphone and the set wind noise boundary threshold is calculated; the quotient of the difference and the conversion coefficient is calculated, and the wind noise pollution range is determined based on the obtained quotient.

[0074] In some non-limiting embodiments, the wind noise pollution range can be estimated using the following formula (8):

[0075]

[0076] Among them, f R (m) is the right boundary of the wind noise contamination range of the first microphone; E Low1 (m) is the low-frequency average amplitude spectrum of the first microphone; thr2 is the wind noise boundary threshold; k(m) is the conversion coefficient.

[0077] In some embodiments, after obtaining the right boundary of the wind noise pollution range, the wind noise pollution range may be [0, f R (m)].

[0078] When determining the wind noise contamination range of the first microphone, thr2 is the wind noise boundary threshold corresponding to the first microphone, and k(m) is the conversion coefficient corresponding to the first microphone.

[0079] Accordingly, when determining the wind noise pollution range of the second microphone, E Low1 (m) is replaced by the low-frequency average amplitude spectrum E of the second microphone Low2 (m), thr2 is the wind noise boundary threshold corresponding to the second microphone, and k(m) is the conversion coefficient corresponding to the second microphone.

[0080] Reference Figure 3 , a schematic diagram of an estimated wind noise pollution range in an embodiment of the present invention is given. Among them, the white boundary f R Represents the right boundary of the wind noise pollution range.

[0081] When spectral energy is used to characterize low-frequency energy, the estimation of the wind noise pollution range can refer to the description given in the above embodiment using the low-frequency average amplitude spectrum as an example. It is only necessary to adaptively replace the low-frequency average amplitude spectrum with spectral energy and take the wind noise boundary threshold as the spectral energy threshold. No further details will be given here.

[0082] Furthermore, the wind noise boundary threshold can be determined in the following manner: for each microphone, based on the signal collected by each microphone, determine the energy of all noises and the energy of other noise signals included in the signal collected by each microphone, where the other noise refers to the noise other than wind noise in all the noises; and based on the ratio of the energy of the other noise to the energy of all the noises, correct the wind noise boundary threshold.

[0083] In some embodiments, modifying the wind noise boundary threshold may refer to increasing or decreasing the preset wind noise boundary threshold according to the ratio of the low-frequency average energy of the other noises to the low-frequency average energy of all noises.

[0084] For example, if other noises are large, such as the ratio of the low-frequency average energy of other noises to the low-frequency average energy of all noises is greater than the set ratio threshold, the preset wind noise boundary threshold can be increased and corrected, that is, the corrected wind noise boundary threshold is greater than the preset wind noise boundary threshold. For another example, if other noises are small, such as the ratio of the low-frequency average energy of other noises to the low-frequency average energy of all noises is less than the set ratio threshold, the preset wind noise boundary threshold can be decreased and corrected, that is, the corrected wind noise boundary threshold is less than the preset wind noise boundary threshold. For another example, if the ratio of the low-frequency average energy of other noises to the low-frequency average energy of all noises is equal to the set ratio threshold, the preset wind noise boundary threshold is not corrected.

[0085] From the above, it can be seen that the wind noise pollution range estimation method of the multi-microphone device provided by the embodiment of the present invention determines the wind noise frame by performing wind noise detection on the signal collected by each microphone in the multi-frame collection signal. For the wind noise frame, the pollution range of each microphone is estimated according to the low-frequency energy of the signal collected by each microphone, the set wind noise boundary threshold and the conversion coefficient, so as to determine the frequency band contaminated by wind noise in the wind noise frame. Since the wind noise pollution range of each microphone is determined according to the low-frequency energy of the signal collected by each microphone corresponding to each wind noise frame for each wind noise frame, the pollution range of the microphone corresponding to each wind noise frame obtained is adapted to the actual wind noise influence of each wind noise frame, thereby improving the rationality and accuracy of the wind noise pollution range determination. Furthermore, when wind noise suppression is subsequently performed on the wind noise frame based on the wind noise pollution range of each microphone, the wind noise suppression effect during noise reduction can be improved.

[0086] An embodiment of the present invention also provides a method for suppressing wind noise in a multi-microphone device. The method can be executed by a terminal, a chip or chip module in the terminal that has a wind noise suppression function, a chip or chip module in the terminal that has a data processing function, or a baseband chip in the terminal. The terminal can be a multi-microphone device, or other terminal device such as a mobile phone, computer, tablet computer, server, cloud platform, etc. for controlling the multi-microphone device. The multi-microphone device can include devices with multiple microphones, such as communication devices, headphones, vehicle-mounted devices, and hearing aids.

[0087] Reference Figure 4 , a flowchart of a method for suppressing wind noise of a multi-microphone device in an embodiment of the present invention is provided. The method for suppressing wind noise of a multi-microphone device may specifically include the following steps:

[0088] Step 41: Estimate the wind noise pollution range of each microphone.

[0089] Step 42 , for the wind noise frame, select one or more microphones from all microphones as target microphones according to the wind noise pollution range of each microphone;

[0090] Step 43 : performing wind noise suppression processing on the signal collected by the target microphone within the wind noise pollution range of the target microphone.

[0091] In a specific implementation, step 41 may use the wind noise pollution range estimation method for a multi-microphone device provided in any of the above embodiments to estimate the wind noise pollution range of each microphone.

[0092] In a specific implementation of step 42 , for the wind noise frame, one or more microphones are selected as target microphones in order from small to large according to the wind noise pollution range of each microphone.

[0093] The number of target microphones is related to the number of channels of voice data actually output. For example, for single-channel voice data, the number of target microphones is one. For another example, for dual-channel voice data, the number of target microphones is two. For another example, for triple-channel voice data, the number of target microphones is three.

[0094] When the number of the target microphone is one, for the wind noise frame, the microphone with the smallest wind noise pollution range is selected as the target microphone.

[0095] When there are two target microphones, for the wind noise frame, the wind noise pollution ranges of the microphones are sorted from small to large, and the microphones with the smallest and second smallest wind noise pollution ranges are selected as target microphones in order of wind noise pollution ranges.

[0096] It should be noted that the target microphones selected for different wind noise frames may be the same or different. The selection is based on the calculated wind noise contamination range of each microphone in each wind noise frame. For example, if there is only one target microphone, for the first wind noise frame, the target microphone selected may be microphone 1. For the second wind noise frame, the target microphone selected may be microphone 2.

[0097] In a specific implementation of step 43 , when there is only one target microphone, wind noise suppression processing is performed on the signal collected by the target microphone within the wind noise pollution range of the target microphone.

[0098] When there are multiple target microphones, wind noise suppression processing is performed on the signal collected by each target microphone within the wind noise pollution range of each target microphone.

[0099] As can be seen above, for the wind noise frame, one or more microphones are selected from all microphones as target microphones based on the wind noise contamination range of each microphone. Wind noise suppression processing is then performed on the signal collected by the target microphone within the wind noise contamination range of the target microphone. Wind noise suppression can be performed on the signal collected by the target microphone that is less affected by wind noise and within the estimated wind noise contamination range, thereby providing a better wind noise suppression effect and achieving a better speech processing effect.

[0100] In a specific implementation, in step 43, a wind noise suppression gain can be obtained based on a coherent weight method. That is, the wind noise suppression gain is obtained based on the coherence coefficient mapping. The wind noise suppression gain can be used to perform wind noise suppression processing on the signal collected by the target microphone. An artificial intelligence (AI) wind noise suppression algorithm can also be used to perform wind noise suppression processing on the signal collected by the target microphone within the wind noise pollution range of the target microphone. Other algorithms can also be used for wind noise suppression processing, and examples are not given here one by one.

[0101] Reference Figure 5 , a schematic diagram of the mapping relationship between the coherence coefficient and the wind noise suppression gain is given. Figure 5 In some embodiments, a gradient function may be used to characterize the mapping relationship between the coherence coefficient and the wind noise suppression gain. For example, when the coherence coefficient is in the range of [0, a], the wind noise suppression gain is A. When the coherence coefficient is in the range of [a, b], the wind noise suppression gain is Where B is the wind noise suppression gain B corresponding to the coherence coefficient b, and n is any coherence coefficient between [a, b]. When the coherence coefficient is between [b, 1], the wind noise suppression gain is B.

[0102] Reference Figure 6 , provides another schematic diagram of the mapping relationship between the coherence coefficient and the wind noise suppression gain. In other embodiments, a curve function may be used to represent the mapping relationship between the coherence coefficient and the wind noise suppression gain.

[0103] In some further embodiments, the coherence coefficient of each frame of sampling signals is used as the wind noise suppression gain of each frame of sampling signals.

[0104] It should be noted that other types of functions may also be used to represent the mapping relationship between the coherence coefficient and the wind noise suppression gain, and examples are not given here one by one.

[0105] In some embodiments, the wind noise suppression gain may be in the range of [0, 1] and the coherence coefficient may be in the range of [0, 1].

[0106] In some non-limiting embodiments, the wind noise suppression gain corresponding to each frame of the sampling signal can be obtained according to the coherence coefficient of the sampling signal of the frame. The coherence coefficient is positively correlated with the wind noise suppression gain. The larger the coherence coefficient, the larger the wind noise suppression gain, the smaller the amount of wind noise suppression, the higher the coherence between the first signal and the second signal in the sampling signal, the smaller the wind noise, and the smaller the intensity of denoising the sampling signal. Correspondingly, the smaller the coherence coefficient, the smaller the wind noise suppression gain, the greater the amount of wind noise suppression, the lower the coherence between the first signal and the second signal in the sampling signal, the greater the wind noise, and the greater the intensity of denoising the sampling signal.

[0107] In a specific implementation, for each frame of sampling signal, the wind noise suppression gain corresponding to each frame of sampling signal can be used to perform a gain on the sampling signal to obtain a signal after wind noise processing. The signal after wind noise processing is converted from the frequency domain to the time domain to obtain a signal after wind noise suppression, thereby achieving wind noise suppression.

[0108] In some embodiments, the wind noise suppression gain of each frequency point is obtained based on the coherence coefficient mapping of each frequency point in each frame of the sampling signal. For each frame of the sampling signal, the wind noise suppression gain of each frequency point is used to perform a gain on the frequency domain signal (such as the complex spectrum) of each frequency point.

[0109] Implementing gain on the sampled signal by using the wind noise suppression gain corresponding to each frame of the sampled signal may be performing a multiplication operation on the wind noise suppression gain and the frequency domain signal.

[0110] Furthermore, for each frequency point of each frame of the sampling signal, the wind noise suppression gain corresponding to each microphone at each frequency point can be used to implement gain on the frequency domain signal at each frequency point.

[0111] For example, taking the frequency spectrum position of the first microphone at the frequency point k of the mth frame as an example, the following formula (9) can be used to implement gain on the signal of the frequency point k of the mth frame.

[0112] X'1(m,k)=X1(m,k)·gain(m,k); (9)

[0113] Wherein, X'1(m,k) is the frequency domain result after wind noise suppression; X1(m,k) is the frequency domain signal of the first microphone at frequency point k in the mth frame; gain(m,k) is the wind noise suppression gain at frequency point k in the mth frame.

[0114] Since the wind noise pollution range can be accurately determined for each wind noise frame, when the wind noise frame is subjected to wind noise suppression processing, both transient wind noise and continuous wind noise can be suppressed effectively.

[0115] The wind noise pollution range estimation method of a multi-microphone device and the wind noise suppression method of a multi-microphone device provided in the embodiments of the present invention can be applied to hearing aids, headphones, mobile phone communications, and vehicle-mounted scenarios.

[0116] The embodiment of the present invention further provides a device for estimating the wind noise pollution range of a multi-microphone device. The device for estimating the wind noise pollution range of a multi-microphone device can be used to implement the method for estimating the wind noise pollution range of a multi-microphone device provided in the above embodiment. Figure 7 , provides a wind noise pollution range estimation device for a multi-microphone device in an embodiment of the present invention, the wind noise pollution range estimation device 70 for a multi-microphone device includes:

[0117] an acquisition unit 71, configured to acquire multiple frames of acquisition signals, each frame of the acquisition signal including signals acquired by each microphone in the multi-microphone device;

[0118] A wind noise detection unit 72 is configured to perform wind noise detection on the signals collected by each microphone, and filter out wind noise frames from the multiple frames of collected signals;

[0119] The wind noise pollution range estimation unit 73 is used to estimate the wind noise pollution range of each microphone for the wind noise frame based on the low-frequency energy of the signal collected by each microphone, the set wind noise boundary threshold and the conversion coefficient. The wind noise pollution range is used to indicate the frequency band contaminated by wind noise, and the conversion coefficient is used to characterize the relationship between the wind noise pollution range and the wind noise energy.

[0120] In a specific implementation, the specific working principle and workflow of the wind noise pollution range estimation device 70 of the multi-microphone device can be found in the description of the wind noise pollution range estimation method of the multi-microphone device provided in the above embodiment, which will not be repeated here.

[0121] In a specific implementation, the wind noise pollution range estimation device 70 of the above-mentioned multi-microphone device can correspond to a chip with a wind noise pollution range estimation function in the multi-microphone device, such as an SOC (System-On-a-Chip), a baseband chip, etc.; or correspond to a chip module with a wind noise pollution range estimation function in the multi-microphone device; or correspond to a chip module with a data processing function chip, or correspond to a multi-microphone device.

[0122] The embodiment of the present invention further provides a wind noise suppression device for a multi-microphone device, which can be used to implement the wind noise suppression method for a multi-microphone device provided in any of the above embodiments. Figure 8 , a schematic structural diagram of a wind noise suppression device for a multi-microphone device according to an embodiment of the present invention is provided. The wind noise suppression device 80 for a multi-microphone device includes:

[0123] The wind noise pollution range estimation device 70 of the multi-microphone device provided by any of the above embodiments;

[0124] a selection unit 81 for selecting one or more microphones as target microphones from all microphones according to the wind noise pollution range of each microphone for the wind noise frame;

[0125] The wind noise suppression unit 82 is configured to perform wind noise suppression processing on the signal collected by the target microphone within the wind noise pollution range of the target microphone.

[0126] In a specific implementation, for more information on the specific working principle and workflow of the wind noise suppression device 80 of the multi-microphone device, please refer to the relevant descriptions of the wind noise pollution range estimation method and wind noise suppression method of the multi-microphone device provided in any of the above embodiments, which will not be repeated here.

[0127] In a specific implementation, the wind noise suppression device 80 of the above-mentioned multi-microphone device can correspond to a chip with wind noise suppression function in the multi-microphone device, such as SOC (System-On-a-Chip), baseband chip, etc.; or correspond to a chip module with wind noise suppression function in the multi-microphone device; or correspond to a chip module with a data processing function chip, or correspond to a multi-microphone device.

[0128] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes the steps of the method for estimating the wind noise pollution range of a multi-microphone device provided in any of the above embodiments of the present invention, or executes the steps of the method for suppressing wind noise of a multi-microphone device provided in any of the embodiments.

[0129] The computer-readable storage medium may include a non-volatile memory or a non-transitory memory, and may also include an optical disk, a mechanical hard disk, a solid-state drive, etc.

[0130] Specifically, in the embodiment of the present invention, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0131] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0132] An embodiment of the present invention also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor runs the computer program, it executes the steps of the wind noise pollution range estimation method of the multi-microphone device provided in any of the above embodiments, or executes the steps of the wind noise suppression method of the multi-microphone device provided in any embodiment.

[0133] The memory is coupled to the processor, and the memory may be located inside or outside the terminal. The memory and the processor may be connected via a communication bus.

[0134] The terminal may include but is not limited to earphones, hearing aids, vehicle-mounted terminals, mobile phones, computers, tablet computers and other terminal devices, and may also be a server, cloud platform, etc.

[0135] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is merely a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0137] In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units. For example, for various devices and products applied to or integrated into a chip, the various modules / units contained therein may all be implemented in the form of hardware such as circuits, or at least some of the modules / units may be implemented in the form of software programs, which run on the processor integrated inside the chip, and the remaining (if any) modules / units may be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated into a chip module, the various modules / units contained therein may all be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as a chip, circuit module, etc.) or different components of the chip module, or at least some of the modules / units may be implemented in the form of hardware such as circuits. The element can be implemented in the form of a software program, which runs on the processor integrated inside the chip module, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in the terminal, the various modules / units contained therein can be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or different components in the terminal, or, at least some modules / units can be implemented in the form of a software program, which runs on the processor integrated inside the terminal, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits.

[0138] It should be understood that the term "and / or" as used herein simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " as used herein indicates that the related objects are in an "or" relationship.

[0139] The term "plurality" used in the embodiments of the present application refers to two or more.

[0140] The first, second, third, etc. descriptions appearing in the embodiments of this application are only for illustration and distinction of the description objects. There is no order, nor does it indicate any special limitation on the number of devices in the embodiments of this application, and cannot constitute any limitation on the embodiments of this application.

[0141] It should be noted that the serial numbers of the steps in this embodiment do not limit the execution order of the steps.

[0142] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A method for estimating wind noise pollution range of a multi-microphone device, characterized in that: include: Acquire multiple frames of acquisition signals, each frame of the acquisition signal including a signal acquired by each microphone in the multi-microphone device; Performing wind noise detection on the signals collected by each microphone, and filtering out wind noise frames from the multiple frames of collected signals; For the wind noise frame, estimate the wind noise contamination range of each microphone based on the low-frequency energy of the signal collected by each microphone, the set wind noise boundary threshold, and the conversion coefficient, where the wind noise contamination range is used to indicate the frequency band contaminated by wind noise, and the conversion coefficient is used to characterize the relationship between the wind noise contamination range and the wind noise energy; The step of determining the wind noise pollution range of each microphone according to the low-frequency energy of the signal collected by each microphone, the set wind noise boundary threshold, and the conversion coefficient includes: For each microphone, calculating the difference between the low-frequency energy of the signal collected by each microphone and the set wind noise boundary threshold; A quotient of the difference and the conversion coefficient is calculated, and the wind noise pollution range is determined according to the obtained quotient.

2. The method for estimating wind noise pollution range of a multi-microphone device according to claim 1, characterized in that: The conversion coefficient is determined as follows: Obtaining the type of each microphone in the multi-microphone device; The conversion coefficient corresponding to each microphone is determined according to the type of each microphone.

3. The method for estimating wind noise pollution range of a multi-microphone device according to claim 2, wherein: The conversion coefficient corresponding to each microphone is determined according to the type of each microphone: For various types of microphones, test the low-frequency energy of the signals collected by the microphones at different wind speeds, and obtain the frequency range corresponding to the low-frequency energy; The low-frequency energy of the signal collected at each wind speed and the corresponding frequency range are fitted to obtain the conversion coefficient.

4. The method for estimating wind noise pollution range of a multi-microphone device according to claim 1, wherein: Also includes: For each microphone, determine, based on the signal collected by each microphone, the energy of all noises and the energy of other noise signals included in the signal collected by each microphone, where the other noise refers to noise other than wind noise in the all noises; The wind noise boundary threshold is modified according to a ratio of the energy of the other noises to the energy of all the noises.

5. A method for suppressing wind noise in a multi-microphone device, characterized in that: include: The wind noise pollution range of each microphone is estimated by using the wind noise pollution range estimation method of the multi-microphone device according to any one of claims 1 to 4; For the wind noise frame, one or more microphones are selected from all microphones as target microphones according to the wind noise pollution range of each microphone; Within the wind noise pollution range of the target microphone, wind noise suppression processing is performed on the signal collected by the target microphone.

6. The method for suppressing wind noise of a multi-microphone device according to claim 5, wherein: The method of selecting one or more microphones as target microphones from all microphones according to the wind noise pollution range of each microphone for the wind noise frame includes: For the wind noise frame, one or more microphones are selected as target microphones in order from small to large according to the wind noise pollution range of each microphone.

7. A device for estimating wind noise pollution range of a multi-microphone device, characterized in that: include: an acquisition unit, configured to acquire multiple frames of acquisition signals, each frame of the acquisition signal including signals respectively acquired by each microphone in the multi-microphone device; a wind noise detection unit, configured to perform wind noise detection on the signals collected by each microphone and filter out wind noise frames from the multiple frames of collected signals; a wind noise contamination range estimation unit, configured to estimate, for the wind noise frame, the wind noise contamination range of each microphone based on the low-frequency energy of the signal collected by each microphone, a set wind noise boundary threshold, and a conversion coefficient, wherein the wind noise contamination range is used to indicate the frequency band contaminated by wind noise, and the conversion coefficient is used to characterize the relationship between the wind noise contamination range and the wind noise energy; The wind noise pollution range estimation unit is used to calculate, for each microphone, the difference between the low-frequency energy of the signal collected by each microphone and the set wind noise boundary threshold; A quotient of the difference and the conversion coefficient is calculated, and the wind noise pollution range is determined according to the obtained quotient.

8. A wind noise suppression device for a multi-microphone device, characterized in that: include: The wind noise pollution range estimation device of the multi-microphone device according to claim 7; a selection unit, configured to select one or more microphones as target microphones from all microphones according to the wind noise pollution range of each microphone for the wind noise frame; The wind noise suppression unit is used to perform wind noise suppression processing on the signal collected by the target microphone within the wind noise pollution range of the target microphone.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program executes the steps of the method for estimating the wind noise pollution range of a multi-microphone device according to any one of claims 1 to 4, or executes the steps of the method for suppressing wind noise of a multi-microphone device according to claim 5 or 6.

10. A terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the processor executes the steps of the method for estimating the wind noise pollution range of a multi-microphone device according to any one of claims 1 to 4, or executes the steps of the method for suppressing wind noise of a multi-microphone device according to claim 5 or 6.

Images