Wind noise pollution degree estimation method, wind noise suppression method, medium and terminal

Abstract

A wind noise pollution degree estimation method, a wind noise suppression method, a medium and a terminal, the wind noise pollution degree estimation method comprises: acquiring multiple frames of collected signals, each frame of collected signal comprising signals collected by each microphone in the multi-microphone device, at least comprising a first signal collected by a first microphone and a second signal collected by a second microphone; performing wind noise detection on the signals collected by each microphone, and screening wind noise frames from the multiple frames of collected signals; for the wind noise frames, estimating the wind noise pollution degree of the first microphone according to the relationship between the low-frequency energy of the first signal collected by the first microphone and the low-frequency energy of the second signal collected by the second microphone, the first microphone being a main microphone, and the wind noise pollution degree being used to indicate the degree of influence of the microphone on wind noise. By adopting the above scheme, the wind noise suppression degree of the wind noise frame can be determined based on the wind noise pollution degree, so as to improve the wind noise suppression effect.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of speech processing, and particularly relate to a wind noise pollution degree estimation method, a wind noise suppression method, a medium and a terminal. BACKGROUND

[0002] In a speech communication scenario and an application of a hearing aid, when a microphone is in a wind condition, airflow disturbance can cause non-steady vibration of a microphone diaphragm, causing noise with a relatively large volume and extremely uncomfortable in subjective listening, which is generally referred to as wind noise (may also be referred to as wind noise). Wind noise has a great negative impact on communication quality and can reduce speech intelligibility.

[0003] Currently, taking a dual-microphone device as an example, a method of amplitude square coherence is usually used to calculate wind noise suppression gain of each frequency point in a frequency spectrum under a dual-microphone condition. The wind noise suppression gain is used to perform wind noise suppression processing on signals collected by each microphone. However, for a main microphone, if the collected signal has speech and no wind noise, and a signal collected by an auxiliary microphone has wind noise, the wind noise suppression gain obtained by using the method of coherence is used to process the signal of the main microphone, which can easily cause speech loss, resulting in poor wind noise suppression effect. SUMMARY

[0004] Embodiments of the present application solve the technical problem of poor wind noise suppression effect.

[0005] To solve the above technical problem, embodiments of the present application provide a wind noise pollution degree estimation method of a multi-microphone device, comprising: obtaining multiple frames of collected signals, each frame of collected signal comprising signals collected by each microphone in the multi-microphone device, at least comprising a first signal collected by a first microphone and a second signal collected by a second microphone; performing wind noise detection on the signals collected by each microphone, and screening wind noise frames from the multiple frames of collected signals; and estimating a wind noise pollution degree of the first microphone according to a relationship between low-frequency energy of the first signal collected by the first microphone and low-frequency energy of the second signal collected by the second microphone for the wind noise frames, the first microphone being a main microphone, and the wind noise pollution degree being used to indicate a degree of influence of wind noise on the microphone.

[0006] Optionally, the estimating the wind noise pollution degree of the first microphone according to the relationship between the low-frequency energy of the first signal collected by the first microphone and the low-frequency energy of the second signal collected by the second microphone comprises: determining a reference value according to a maximum value of the low-frequency energy of the first signal and the low-frequency energy of the second signal; and estimating the wind noise pollution degree of the first microphone according to a ratio of the low-frequency energy of the first signal to the reference value.

[0007] Optionally, the determining the reference value according to the maximum of the low-frequency energy of the first signal and the low-frequency energy of the second signal comprises: obtaining a first adjustment coefficient, correcting the maximum by using the first adjustment coefficient, and obtaining the reference value according to the corrected value.

[0008] Optionally, the obtaining the reference value according to the corrected value comprises: obtaining a second adjustment coefficient, correcting the corrected value by using the second adjustment coefficient to obtain the reference value.

[0009] Optionally, the estimating the wind noise pollution degree of the first microphone according to the ratio of the low-frequency energy of the first signal and the reference value comprises: obtaining a third adjustment coefficient, correcting the low-frequency energy of the first signal by using the third adjustment coefficient to obtain the corrected low-frequency energy of the first signal; and estimating the wind noise pollution degree of the first microphone according to the ratio of the corrected low-frequency energy of the first signal and the reference value.

[0010] Optionally, the wind noise pollution degree of the first microphone is estimated by using the following formula: wherein P1(m) is the wind noise pollution degree of the first microphone in the mth frame; E Low1 (m) is the low-frequency energy of the first signal of the first microphone in the low-frequency range defined by the low-frequency cutoff point N Low1 ; e1 is the third adjustment coefficient; β is the first adjustment coefficient; e2 is the second adjustment coefficient; max(E Low1 (m), E Low2 (m)) is the maximum of E Low1 (m) and E Low2 (m); E Low2 (m) is the low-frequency energy of the second signal of the second microphone in the low-frequency range defined by the low-frequency cutoff point N Low2 .

[0011] Optionally, the wind noise pollution degree estimation method of the multi-microphone device further comprises: estimating the wind noise pollution degree of the second microphone according to the ratio of the low-frequency energy of the second signal and the reference value.

[0012] Optionally, the estimating the wind noise pollution degree of the first microphone according to the relationship between the low-frequency energy of the first signal collected by the first microphone and the low-frequency energy of the second signal collected by the second microphone for the wind noise frame comprises: calculating the total energy of the low-frequency energy of the first signal and the low-frequency energy of the second signal; calculating the energy proportion of the low-frequency energy of the first signal in the total energy; and estimating the wind noise pollution degree of the first microphone according to the energy proportion.

[0013] Optionally, the estimating the wind noise pollution degree of the first microphone according to the energy proportion includes: when the energy proportion is less than a proportion threshold, determining the wind noise pollution degree of the first microphone according to a preset positive correlation coefficient; and when the energy proportion is greater than or equal to the proportion threshold, taking a set wind noise pollution degree as the wind noise pollution degree of the first microphone.

[0014] Optionally, the estimating the wind noise pollution degree of the first microphone according to the energy proportion includes: calculating the wind noise pollution degree of the first microphone according to the energy proportion by using the following formula: wherein, E Low1 (m) is the low-frequency energy of the first signal; E Low2 (m) is the low-frequency energy of the second signal; γ1(m) is the energy proportion; P1(m) is the wind noise pollution degree of the first microphone; η, σ and b are adjustment coefficients.

[0015] Optionally, the wind noise pollution degree estimation method of the multi-microphone device further includes: when the number of microphones included in the multi-microphone device is greater than or equal to three, taking the microphone with the minimum low-frequency energy of the signals collected by the respective microphones as the first microphone; combining the other microphones in the multi-microphone device with the first microphone respectively, and estimating the wind noise pollution degrees of the other microphones, the other microphones being the microphones in the multi-microphone device except the first microphone; wherein the second microphone is the microphone with the minimum wind noise pollution degree among the other microphones.

[0016] The embodiment of the application further provides a wind noise suppression method of a multi-microphone device, including: estimating the wind noise pollution degree of the first microphone by using any wind noise pollution degree estimation method of a multi-microphone device as described above; determining a wind noise suppression gain according to the wind noise pollution degree of the first microphone for each wind noise frame; and performing wind noise suppression processing on the first signal collected by the first microphone by using the wind noise suppression gain for each wind noise frame.

[0017] Optionally, the determining a wind noise suppression gain according to the wind noise pollution degree of the first microphone includes: calculating an initial wind noise suppression gain of each wind noise frame according to the coherence coefficient of the first microphone and the second microphone for each wind noise frame; and correcting the initial wind noise suppression gain by using the wind noise pollution degree of the first microphone, and taking the corrected wind noise suppression gain as the wind noise suppression gain.

[0018] Optionally, the initial wind noise suppression gain is modified by using the wind noise pollution degree of the first microphone, and the modified wind noise suppression gain of the first microphone is used as the wind noise suppression gain, including: calculating a difference between 1 and the initial wind noise suppression gain for each wind noise frame; performing multiplication operation on the wind noise pollution degree of the first microphone and the difference to obtain a multiplication result; and using the difference between 1 and the multiplication result as the modified wind noise suppression gain of the first microphone.

[0019] Optionally, the wind noise suppression processing is performed on the signal collected by the second microphone by using the modified wind noise suppression gain corresponding to the second microphone.

[0020] Optionally, the wind noise suppression method of the multi-microphone device further includes: estimating a wind noise pollution range of the first microphone according to the low-frequency energy of the first signal collected by the first microphone, a set wind noise boundary threshold, and a conversion coefficient for each wind noise frame, the wind noise pollution range being used to indicate a frequency band polluted by wind noise, and the conversion coefficient being used to represent the relationship between wind noise energy and the wind noise pollution range.

[0021] Optionally, the wind noise suppression processing is performed on the first signal collected by the first microphone by using the wind noise suppression gain for each wind noise frame, including: performing the wind noise suppression processing on the first signal collected by the first microphone by using the wind noise suppression gain corresponding to the first microphone in the wind noise pollution range of the first microphone.

[0022] The embodiment of the present application further provides a wind noise pollution degree estimation device of a multi-microphone device, including: an acquisition unit, configured to acquire a plurality of frames of collected signals, each frame of collected signal including signals collected by each microphone in the multi-microphone device respectively, and at least including a first signal collected by a first microphone and a second signal collected by a second microphone; a wind noise detection unit, configured to perform wind noise detection on the signals collected by each microphone, and screen wind noise frames from the plurality of frames of collected signals; and a wind noise pollution degree estimation unit, configured to estimate a wind noise pollution degree of the first microphone according to the relationship between the low-frequency energy of the first signal collected by the first microphone and the low-frequency energy of the second signal collected by the second microphone for each wind noise frame, the first microphone being a main microphone, and the wind noise pollution degree being used to indicate the influence degree of wind noise on the microphone.

[0023] The embodiment of the present application also provides a wind noise suppression device of a multi-microphone device, which comprises the wind noise pollution degree estimation device of the multi-microphone device, a calculation unit configured to calculate, for each wind noise frame, a wind noise suppression gain of each wind noise frame, a wind noise suppression gain determination unit configured to determine, for each wind noise frame, the wind noise suppression gain according to the wind noise pollution degree of the first microphone, and a wind noise suppression processing unit configured to perform, for each wind noise frame, wind noise suppression processing on the first signal collected by the first microphone by using the wind noise suppression gain.

[0024] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is configured to perform the steps of any one of the wind noise pollution degree estimation methods of the multi-microphone device or the steps of any one of the wind noise suppression methods of the multi-microphone device when the computer program is executed by a processor.

[0025] The embodiment of the present application also provides a terminal, which comprises a memory and a processor, and the memory stores a computer program capable of being executed by the processor, and the processor is configured to perform the steps of any one of the wind noise pollution degree estimation methods of the multi-microphone device or the steps of any one of the wind noise suppression methods of the multi-microphone device when the computer program is executed.

[0026] Compared with the prior art, the technical scheme of the embodiment of the present application has the following beneficial effects:

[0027] In the wind noise pollution degree estimation method of the multi-microphone device provided by the embodiment of the present application, wind noise frames are determined by performing wind noise detection on signals collected by each microphone in multiple frames of collected signals. Each frame of collected signal comprises at least a first signal collected by a first microphone and a second signal collected by a second microphone. For the wind noise frames, the wind noise pollution degree of the first microphone is estimated according to the relationship between the low-frequency energy of the first signal collected by the first microphone and the low-frequency energy of the second signal collected by the second microphone. The wind speed can be reflected in the low-frequency energy, the low-frequency energy can reflect the energy of wind noise, and the energy of wind noise can represent the pollution degree of wind noise. The first microphone is a main microphone, and the wind noise pollution degree of the first microphone can be determined by the relationship between the low-frequency energy of the first signal and the low-frequency energy of the second signal. When wind noise suppression is performed on the wind noise frames subsequently, the wind noise suppression degree of the wind noise frames can be determined based on the wind noise pollution degree, so as to improve the wind noise suppression effect. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a flowchart of the wind noise pollution degree estimation method of the multi-microphone device in the embodiment of the present application;

[0029] Figure 2is a flow chart of a wind noise suppression method of a multi-microphone device in an embodiment of the present application;

[0030] Figure 3 is a mapping relationship diagram of a coherence coefficient and an initial wind noise suppression gain;

[0031] Figure 4 is a mapping relationship diagram of a coherence coefficient and an initial wind noise suppression gain;

[0032] Figure 5 is a structural diagram of a wind noise pollution degree estimation device of a multi-microphone device in an embodiment of the present application;

[0033] Figure 6 is a structural diagram of a wind noise suppression device of a multi-microphone device in an embodiment of the present application. DETAILED DESCRIPTION

[0034] As described above, taking a dual-microphone device as an example, at present, the amplitude square coherence method is usually used to calculate the wind noise suppression gain of each frequency point on the frequency spectrum under the dual-microphone condition. The wind noise suppression gain is used to perform wind noise suppression processing on the signals collected by each microphone. Since the coherence coefficient size only reflects the coherence degree of the two microphone signals, it does not reflect which microphone is more affected by wind noise. Therefore, for the same frequency point of the same frame, the wind noise suppression gains of the two microphones are the same. However, for the main microphone, if the collected signal has speech but no wind noise, and the signal collected by the auxiliary microphone has wind noise, the wind noise suppression gain obtained by using the coherence method is used to process the signal of the main microphone, which is easy to cause speech loss, resulting in poor wind noise suppression effect.

[0035] To solve the above problem, in an embodiment of the present application, wind noise detection is performed on the signals collected by each microphone in multiple frames of collected signals to determine a wind noise frame. Each frame of sampled signal at least includes a first signal collected by a first microphone and a second signal collected by a second microphone. For the wind noise frame, the wind noise pollution degree of the first microphone is estimated according to the relationship between the low-frequency energy of the first signal collected by the first microphone and the low-frequency energy of the second signal collected by the second microphone. The wind speed size can be reflected on the low-frequency energy, the low-frequency energy can reflect the energy size of wind noise, and the energy of wind noise can represent the pollution degree of wind noise. The first microphone is the main microphone, and the wind noise pollution degree of the first microphone can be determined through the relationship between the low-frequency energy of the first signal and the low-frequency energy of the second signal. When wind noise suppression is performed on the wind noise frame subsequently, the wind noise suppression degree of the wind noise frame can be determined based on the wind noise pollution degree to improve the wind noise suppression effect.

[0036] In order to make the above-mentioned purposes, features and benefits of the embodiments of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings.

[0037] The wind noise pollution degree estimation method of the multi-microphone device can be executed by a terminal, a chip or a chip module with wind noise pollution degree estimation function in the terminal, a chip or a chip module with data processing function in the terminal, or a baseband chip in the terminal. The terminal can be a multi-microphone device, or a mobile phone, a computer, a tablet computer, a server, a cloud platform or other terminal device for controlling the multi-microphone device. The multi-microphone device can include a communication device, a vehicle-mounted terminal, a headset, a hearing aid and other devices with multiple microphones.

[0038] Reference Figure 1 The wind noise pollution degree estimation method of the multi-microphone device can include the following steps:

[0039] Step 11, obtaining multiple frames of collected signals, each frame of collected signal including signals collected by each microphone in the multi-microphone device, at least including a first signal collected by a first microphone and a second signal collected by a second microphone;

[0040] Step 12, performing wind noise detection on the signals collected by each microphone, and screening wind noise frames from the multiple frames of collected signals;

[0041] Step 13, for the wind noise frames, estimating the wind noise pollution degree of the first microphone according to the relationship between the low-frequency energy of the first signal collected by the first microphone and the low-frequency energy of the second signal collected by the second microphone, the first microphone being a main microphone, and the wind noise pollution degree being used to indicate the degree of influence (pollution degree) of the microphone on the wind noise.

[0042] The greater the wind noise pollution degree, the greater the degree of influence on the wind noise.

[0043] Wind noise is the vortex and turbulence from airflow, which is non-steady and unpredictable, and is different from any other sound source around it, and is an independent event. The surrounding other human voice or noise is generated by the vibration of a specific sound source. Research has found that for a multi-microphone device, the coherence of "the same sound source" and the incoherence of "independent generation" of wind noise can be used to distinguish wind noise from other noise. Then in the presence of wind noise and the absence of wind noise, different signal processing methods are used for noise reduction.

[0044] Based on the mechanism of wind noise generation, for a certain frame of collected signals, for each frame of collected signals, the signal collected by the first microphone in the frame of collected signals is recorded as the first signal, and the signal collected by the second microphone is recorded as the second signal. For a certain frame of collected signals, if there is wind noise, the coherence of the first signal and the second signal in the frame of collected signals is poor. Correspondingly, if there is no wind noise, the coherence of the first signal and the second signal in the frame of collected signals is good.

[0045] In some embodiments of the application, in step 12, the amplitude squared coherence method can be used to detect wind noise. Specifically, the amplitude squared coherence method is used to detect wind noise and obtain the corresponding coherence coefficient. The larger the coherence coefficient, the less the sampled signal is affected by wind noise, that is, the more wind noise corresponding signals contained in the sampled signal; if the coherence coefficient is smaller, the sampled signal is more affected by wind noise, that is, the more wind noise corresponding signals contained in the sampled signal.

[0046] For ease of understanding, the specific implementation of step 12 is illustrated below with an example of a multi-microphone device including two microphones.

[0047] Taking the mth frame of collected signals as an example, the mth frame of collected signals includes the first signal and the second signal. The first signal and the second signal are converted from time domain to frequency domain to obtain the first frequency domain signal corresponding to the first signal and the second frequency domain signal corresponding to the second signal. The amplitude squared coherence method is used to detect the coherence of the first frequency domain signal and the second frequency domain signal to obtain the coherence coefficient of the mth frame of collected signals in the set low frequency range. The coherence coefficient in the low frequency range is used to represent the significant frequency domain coherence of the first signal and the second signal in the low frequency range. The coherence coefficient can be used to determine whether the collected signal includes wind noise, and thus whether each frame of collected signals is a wind noise frame, that is, to filter out wind noise frames from each frame of collected signals. The low frequency range is a frequency lower than the set low frequency cutoff frequency. As a preferred embodiment, the frequency of the set low frequency cutoff frequency is 120 Hz. It can be understood that other frequencies can also be selected as the set low frequency cutoff frequency according to actual needs.

[0048] Each frame of collected signals generally refers to a signal within a set time period. When converting the collected signal from time domain to frequency domain, a plurality of frequency points are set. For example, for each frequency point in each frame, the coherence coefficients of the first frequency domain signal and the second frequency domain signal are calculated respectively. According to the coherence coefficients of each frequency point in each frame, the coherence coefficient of each frame of collected signals is obtained.

[0049] In calculating the coherence coefficient of the first frequency domain signal and the second frequency domain signal, the first frequency domain signal and the second frequency domain signal can be processed to obtain a first complex frequency spectrum corresponding to the first frequency domain signal and a second complex frequency spectrum corresponding to the second frequency domain signal. The coherence coefficient of the first complex frequency spectrum and the second complex frequency spectrum is calculated. It can be understood that the coherence coefficient of the first frequency domain signal and the second frequency domain signal can also be calculated based on other types of frequency domain signals.

[0050] In some non-limiting embodiments, for each frame of the collected signal, the average of the coherence coefficients of each frequency point of each frame of the collected signal can be taken as the coherence coefficient of each frame of the collected signal.

[0051] In some non-limiting embodiments, taking the mth frame of the collected signal as an example, the first complex frequency spectrum X1 is obtained after processing the first frequency domain signal, and the second complex frequency spectrum X2 is obtained after processing the second frequency domain signal. The coherence coefficient of the first signal and the second signal is calculated according to the first complex frequency spectrum X1 and the second complex frequency spectrum X2.

[0052] For example, the amplitude square coherence is used to detect wind noise, and the coherence coefficient is calculated. Specifically, the coherence coefficient can be calculated using the following formulas (1) to (5). Here, the coherence coefficient of each frame of the sampled signal is the average of the coherence coefficients corresponding to the multiple frequency points in each frame of the sampled signal.

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] wherein coh_mean(m) is the coherence coefficient of the mth frame; Φ 11 is the self-spectral density of the first microphone; Φ 11 (m, k) is the self-spectral density of the kth frequency point of the mth frame of the first microphone; α is a smoothing coefficient, which can be in the range of [0.5, 0.95]; X1(m, k) is the complex frequency spectrum of the kth frequency point of the mth frame of the first microphone; Φ 12 is the cross-spectral density between the first microphone and the second microphone; Φ 12 (m, k) is the cross-spectral density between the kth frequency point of the mth frame of the first microphone and the second microphone; Φ 12 (m-1, k) is the cross-spectral density between the kth frequency point of the m-1th frame of the first microphone and the second microphone; Φ 22 is the self-spectral density of the second microphone; Φ22 (m, k) is the self-spectrum density of the mth frame and the kth frequency point of the second microphone; Φ 22 (m-1, k) is the self-spectrum density of the m-1th frame and the kth frequency point of the second microphone; X2(m, k) is the complex spectrum of the mth frame and the kth frequency point of the second microphone; coh(m, k) is the coherence coefficient of the mth frame and the kth frequency point; coh_mean(m) is the mean value of the coherence coefficient in the low frequency range of the mth frame; N Low is the set low frequency cutoff frequency point; [1, N Low ] is the set low frequency range; is the mean value of coh(m, k) at the frequency point k on [1, N Low ]; is the conjugate variable corresponding to X1(m, k); is the conjugate variable corresponding to X2(m, k); is the conjugate variable corresponding to Φ 12 (m, k). Wherein, m is a positive integer greater than 1, and k is a positive integer.

[0059] In a specific implementation, after obtaining the coherence coefficient of each frame of the collected signal, whether each frame of the collected signal is a wind noise frame can be determined according to the relationship between the coherence coefficient of each frame of the collected signal and the set coherence coefficient threshold. For example, if the coherence coefficient of the collected signal is less than the set coherence coefficient threshold, it is determined that the frame of the collected signal is a wind noise frame.

[0060] In a specific implementation, the coherence coefficient threshold can be configured according to the requirement for noise reduction processing accuracy, which is not limited here.

[0061] It is found through research that the signal energy of the active noise has little difference at low frequencies. However, the wind noise, as a non-coherent and non-steady noise with main energy concentrated at low frequencies, can reflect the wind speed, which can represent the wind noise pollution degree.

[0062] In step 13, the wind noise pollution degree of the first microphone can be estimated in the following manner. Specifically, a reference value is determined according to the maximum value of the low frequency energy of the first signal and the low frequency energy of the second signal; and the wind noise pollution degree of the first microphone is estimated according to the ratio of the low frequency energy of the first signal to the reference value.

[0063] Further, a first adjustment coefficient is obtained, and the maximum value is modified by using the first adjustment coefficient, and the modified value is taken as the reference value.

[0064] Further, modifying the maximum value by using the first adjustment coefficient includes multiplying the first adjustment coefficient and the maximum value, and obtaining the reference value according to the multiplication result.

[0065] In some embodiments, the first adjustment coefficient is a preset value.

[0066] In other embodiments, the first adjustment coefficient is related to the low-frequency energy of the signal collected by each microphone. The first adjustment coefficient can be adaptively adjusted according to the low-frequency energy of the signal collected by each microphone. In this case, the first adjustment coefficient is inversely related to the low-frequency energy of the signal collected by each microphone.

[0067] For ease of understanding, the low-frequency energy of the first signal collected by the first microphone is taken as an example for illustration. If the low-frequency energy of the first signal is large, it indicates that the wind noise in the first signal is large. In this case, a relatively small first adjustment coefficient can be selected to increase the wind noise ratio, so that a larger wind noise pollution degree can be obtained. When wind noise suppression is performed based on the wind noise pollution degree, the wind noise suppression strength is larger. Correspondingly, if the low-frequency energy of the first signal is small, it indicates that the wind noise in the first signal is relatively small. In this case, a relatively large first adjustment coefficient can be selected to reduce the wind noise ratio, so that a smaller wind noise pollution degree can be obtained. When wind noise suppression is performed based on the wind noise pollution degree, the wind noise suppression strength is smaller, so as to reduce the loss of voice.

[0068] Further, a second adjustment coefficient is obtained, and the second adjustment coefficient is used to modify the modified value to obtain the reference value.

[0069] In one non-limiting embodiment, the modified value is subtracted from the second adjustment coefficient, and the result of the subtraction operation is taken as the reference value. The second adjustment coefficient is used to represent the background noise energy in the signal with large low-frequency energy in the first signal and the second signal. After the reference value is modified by the second adjustment coefficient, the background noise can be modified, so that the reference value obtained can more accurately represent the energy of the wind noise, thereby improving the accuracy of the wind noise pollution degree estimation.

[0070] In some embodiments, the second adjustment coefficient is a preset value.

[0071] In other embodiments, the second adjustment coefficient is adaptively adjusted according to the background average noise level in the signal with large low-frequency energy in the first signal and the second signal. Here, it is assumed that the second signal has large low-frequency energy. Specifically, the background average noise energy of the second signal in the medium-high frequency band is calculated, and the background average noise energy of the second signal in the medium-high frequency band is converted into the corresponding energy e2 in the low-frequency range (such as 1-N Low2 ). The second adjustment coefficient can be the energy e2.

[0072] Further, a third adjustment coefficient is obtained, and the low frequency energy of the first signal is modified by using the third adjustment coefficient to obtain a modified low frequency energy of the first signal; and the wind noise pollution degree of the first microphone is estimated according to a ratio of the modified low frequency energy of the first signal to the reference value. The third adjustment coefficient is used to represent the energy of the background noise in the low frequency energy of the first signal.

[0073] In some non-limiting embodiments, the low frequency energy of the first signal can be subtracted from the third adjustment coefficient, and the result of the subtraction operation is taken as the modified low frequency energy of the first signal. After the low frequency energy of the first signal is modified by using the third adjustment coefficient, the background noise can be modified, so that the modified low frequency energy of the first signal can more accurately represent the energy of the wind noise, thereby improving the accuracy of the estimation of the wind noise pollution degree.

[0074] In some embodiments, the third adjustment coefficient can be a set value.

[0075] In other embodiments, the third adjustment coefficient can be adaptively adjusted according to the background average noise level in the first signal. Specifically, the corresponding background average noise energy of the medium-high frequency band of the first signal is calculated, the corresponding energy e1 in the low frequency range (such as 1-N Low2 ) of the corresponding background average noise energy of the medium-high frequency band of the first signal is converted, and the third adjustment coefficient can be the energy e1.

[0076] Wherein, N Low2 is the low frequency cutoff frequency point of the low frequency range. The medium-high frequency band refers to a frequency range higher than a certain set starting frequency point, and the starting frequency point of the medium-high frequency band is greater than the low frequency cutoff frequency point of the low frequency range.

[0077] As a non-limiting example, the wind noise pollution degree of the first microphone can be estimated by using the following formula (6).

[0078]

[0079] Wherein, P1(m) is the wind noise pollution degree of the mth frame of the first microphone; E Low1 (m) is the low frequency energy of the first signal of the first microphone in the low frequency range defined by the low frequency cutoff frequency point N Low1 ; e1 is the third adjustment coefficient; β is the first adjustment coefficient; e2 is the second adjustment coefficient; max(E Low1 (m), E Low2 (m)) is the maximum value of E Low1 (m) and E Low2 (m); E Low2 (m) is the low frequency energy of the first signal of the first microphone in the low frequency range defined by the low frequency cutoff frequency point N Low2a low frequency energy of the second signal in the defined low frequency range.

[0080] In step 13, the wind noise pollution degree of the first microphone can also be estimated in the following way. Specifically, a total energy of a low frequency energy of the first signal and a low frequency energy of the second signal is calculated; an energy proportion of the low frequency energy of the first signal to the total energy is calculated; and the wind noise pollution degree of the first microphone is estimated according to the energy proportion.

[0081] In some embodiments, when the energy proportion is less than a proportion threshold, the wind noise pollution degree of the first microphone is determined according to a preset positive correlation coefficient; and when the energy proportion is greater than or equal to the proportion threshold, a set wind noise pollution degree is taken as the wind noise pollution degree of the first microphone.

[0082] For example, when the proportion threshold is 0.5, the positive correlation coefficient is 2, and the set wind noise pollution degree is 1. That is, when the energy proportion is less than 0.5, the wind noise pollution degree of the first microphone is twice the energy proportion. When the energy proportion is greater than or equal to 0.5, the wind noise pollution degree of the first microphone is 1. It should be noted that the above proportion threshold of 0.5, the positive correlation coefficient of 2, and the set wind noise pollution degree of 1 are only examples for easy understanding, and other values can also be taken in actuality.

[0083] In other embodiments, the wind noise pollution degree of the first microphone estimated according to the energy proportion is calculated by using the following formulas (7) and (8):

[0084]

[0085]

[0086] wherein E Low1 (m) is the low frequency energy of the first signal; E Low2 (m) is the low frequency energy of the second signal; γ1(m) is the energy proportion; P1(m) is the wind noise pollution degree of the first microphone; η, σ, and b are adjustment coefficients.

[0087] The adjustment coefficients η, σ, and b can be preset. When the adjustment coefficients η, σ, and b take different values, the wind noise pollution degrees obtained are different. For example, if it is desired to have better protection for the voice in the voice data, η can be made smaller, or σ can be made smaller, or b can be made larger, etc. In addition, when γ1(m) is smaller, P1(m) is also smaller.

[0088] In specific implementation, the low frequency energy can be represented by spectral energy or spectral amplitude, etc.

[0089] In some non-limiting embodiments, the spectral amplitude can be characterized using a low-frequency amplitude spectrum.

[0090] Furthermore, the spectral amplitude is calculated using the low-frequency average amplitude spectrum, which is the average of the low-frequency amplitude spectra at each frequency point within each frame of a defined low-frequency range. Using the low-frequency average amplitude spectrum can reduce errors caused by fluctuations, improve the accuracy of low-frequency energy prediction, and thus help improve the accuracy of wind noise pollution estimation.

[0091] In specific implementation, taking the m-th frame acquisition signal as an example, the low-frequency average amplitude spectrum of the m-th frame signal acquired by the first microphone can be calculated using the following formula (9), and the low-frequency average amplitude spectrum of the m-th frame signal acquired by the second microphone can be calculated using the following formula (10).

[0092]

[0093]

[0094] Among them, E Low1 (m) represents the low-frequency average amplitude spectrum of the m-th frame of the first microphone, reflecting the energy of wind noise at low frequencies; X1(m,k) represents the complex spectrum of the k-th frequency point of the m-th frame of the first microphone; E Low2 (m) represents the low-frequency average amplitude spectrum of the m-th frame of the second microphone; X2(m,k) represents the complex spectrum of the k-th frequency point of the m-th frame of the second microphone; N Low1 The low-frequency cutoff point in the low-frequency range when calculating the low-frequency average amplitude spectrum of the first microphone; N Low2 The low-frequency cutoff point is used to calculate the low-frequency average amplitude spectrum of the second microphone; |X2(m,k)| is the absolute value of X2(m,k); |X1(m,k)| is the absolute value of X1(m,k).

[0095] It should be noted that the low-frequency average amplitude spectrum is defined on the logarithmic spectrum. The logarithm can be based on the natural logarithm e, 10, or other numbers. Numbers with different bases are related to each other through a base-changing formula. The above formulas (9) and (10) use 2 as the base as an example and use the method of averaging to obtain the low-frequency average amplitude spectrum, thereby reducing the error caused by fluctuations.

[0096] Taking the first microphone as an example, when calculating the low-frequency average amplitude spectrum, the average of the low-frequency amplitude spectra corresponding to the first few frequency points is generally taken. The above formula takes values ​​from 1 to N. Low1 The mean of the low-frequency amplitude spectrum across a range of frequencies. In practice, values ​​from 2 to N can also be used. Low1 The mean of the low-frequency amplitude spectrum at frequency points within the range. N Low1 The definition is the low-frequency cutoff point of the low-frequency range when calculating the low-frequency average amplitude spectrum of the first microphone. For example, NLow1 The corresponding frequency can be selected at about 200 Hertz (Hz) in general. It is to be noted that N Low1 Other frequencies can also be taken, which are not limited here.

[0097] It can be understood that the low frequency energy under the non-logarithmic spectrum can also be utilized. For example, taking the first microphone as an example, the low frequency average amplitude spectrum of the mth frame of the first microphone is calculated by using the following formula (11).

[0098]

[0099] Wherein, E Low1 (m) is the low frequency average amplitude spectrum of the mth frame of the first microphone, which reflects the energy size of the wind noise at the low frequency; X1(m, k) is the complex frequency spectrum of the kth frequency point of the mth frame of the first microphone; N Low1 is the low frequency cutoff frequency point of the low frequency range when calculating the low frequency average amplitude spectrum of the first microphone.

[0100] From the above scheme, it can be known that the wind noise frame is determined by detecting the wind noise of the signals collected by each microphone in the multiple frames of collected signals. Each frame of sampling signal at least includes the first signal collected by the first microphone and the second signal collected by the second microphone. For the wind noise frame, the wind noise pollution degree of the first microphone is estimated according to the relationship between the low frequency energy of the first signal collected by the first microphone and the low frequency energy of the second signal collected by the second microphone. The wind speed size can be reflected on the low frequency energy, the low frequency energy can reflect the energy size of the wind noise, and the energy of the wind noise can represent the pollution degree of the wind noise. The first microphone is the main microphone, and the wind noise pollution degree of the first microphone can be determined by the relationship between the low frequency energy of the first signal and the low frequency energy of the second signal. When the wind noise frame is suppressed in the subsequent wind noise suppression, the wind noise suppression degree of the wind noise frame can be determined based on the wind noise pollution degree, so as to improve the wind noise suppression effect.

[0101] In a specific implementation, the wind noise pollution degree of the second microphone is estimated according to the ratio of the low frequency energy of the second signal to the reference value.

[0102] Further, the specific estimation scheme of the wind noise pollution degree of the second microphone can be the same as the estimation scheme of the wind noise pollution degree of the first microphone provided in the above embodiment, and specific reference can be made to the related description in the estimation scheme of the wind noise pollution degree of the first microphone in the above embodiment, which will not be described here.

[0103] In a specific implementation, when the number of microphones in the multi-microphone device is greater than or equal to three, a microphone with the lowest low-frequency energy of the signals collected by the respective microphones is selected as the first microphone; the other microphones in the multi-microphone device are combined with the first microphone respectively, and the wind noise pollution degree of the other microphones is estimated, wherein the other microphones are the microphones in the multi-microphone device except the first microphone. The second microphone is the microphone with the lowest wind noise pollution degree among the other microphones, i.e., the microphone with the second-lowest wind noise pollution degree among all the microphones.

[0104] The low-frequency energy of the signals collected by the respective microphones can represent the wind noise pollution degree of the respective microphones, so the microphone with the lowest low-frequency energy of the signals is selected as the first microphone, i.e., the main microphone. At this time, the first microphone is least affected by wind noise, and the collected signal has the best speech intelligibility. Since the first microphone is less affected by wind noise, the wind noise pollution degree of the other microphones can be estimated more accurately by taking the first microphone as a reference, combining the other microphones with the first microphone respectively, and estimating the wind noise pollution degree of the other microphones, and then the wind noise pollution degree of the other microphones is sorted, and the second microphone is the microphone with the lowest wind noise pollution degree among the other microphones, i.e., the microphone with the second-lowest wind noise pollution degree among all the microphones.

[0105] For example, the other microphones include a third microphone and a fourth microphone. The third microphone is combined with the first microphone, and the wind noise pollution degree of the third microphone is estimated according to the relationship between the low-frequency energy of the third signal collected by the third microphone and the low-frequency energy of the first signal collected by the first microphone. The fourth microphone is combined with the first microphone, and the wind noise pollution degree of the fourth microphone is estimated according to the relationship between the low-frequency energy of the fourth signal collected by the fourth microphone and the low-frequency energy of the first signal collected by the first microphone.

[0106] After the first microphone and the second microphone are determined, the wind noise suppression gain of each microphone can be determined according to the following rules.

[0107] When the first microphone and the second microphone are combined during subsequent wind noise suppression, the initial wind noise suppression gain is obtained from the coherence coefficient of the first microphone and the second microphone, and the initial wind noise suppression gain is corrected by using the wind noise pollution degree of the first microphone and the second microphone obtained above. That is, the wind noise suppression gain of the first microphone is obtained by correcting the initial wind noise suppression gain by using the wind noise pollution degree of the first microphone, and the wind noise suppression gain of the second microphone is obtained by correcting the initial wind noise suppression gain by using the wind noise pollution degree of the second microphone.

[0108] When considering a device with more than two microphones, preferably, each of the other microphones is combined with the first microphone, and an initial wind noise suppression gain corresponding to each combination is obtained. For each combination, the initial wind noise suppression gain is corrected according to the wind noise pollution degree of the other microphone relative to the first microphone to obtain the wind noise suppression gain of the other microphone.

[0109] When performing wind noise suppression, the signals collected by each microphone can be subjected to wind noise suppression processing according to the number of output channels required by the system and according to the wind noise suppression gain of each microphone obtained above.

[0110] For example, the input signal is multi-channel, but the number of output signal channels is single, and for a wind noise frame, the first microphone is used as the output channel, and the wind noise suppression gain is the wind noise suppression gain of the first microphone as described above, i.e., the wind noise suppression gain obtained after correcting the initial wind noise suppression gain according to the wind noise pollution degree of the first microphone when the first microphone is combined with the second microphone.

[0111] For example, the input signal is multi-channel, but the number of output signal channels is single, and for a wind noise frame, the first microphone is used as the output channel, and the wind noise suppression gain is the wind noise suppression gain of the first microphone as described above, i.e., the wind noise suppression gain obtained after correcting the initial wind noise suppression gain according to the wind noise pollution degree of the first microphone when the first microphone is combined with the second microphone.

[0112] For example, the input signal is multi-channel, but the number of output signal channels is single, and for a wind noise frame, the first microphone is used as the output channel, and the wind noise suppression gain is the wind noise suppression gain of the first microphone as described above, i.e., the wind noise suppression gain obtained after correcting the initial wind noise suppression gain according to the wind noise pollution degree of the first microphone when the first microphone is combined with the second microphone.

[0113] The embodiment of the present application also provides a wind noise suppression method of a multi-microphone device, which can be executed by a terminal, a chip or chip module with a wind noise suppression function in the terminal, a chip or chip module with a data processing function in the terminal, or a baseband chip in the terminal. The terminal can be a multi-microphone device, or a mobile phone, computer, tablet computer, server, cloud platform or other terminal device for controlling the multi-microphone device. The multi-microphone device can include a communication device, earphone, vehicle-mounted device, hearing aid and other devices with multiple microphones.

[0114] Reference Figure 2 A flowchart of the wind noise suppression method of the multi-microphone device is shown in the embodiment of the present application, which can specifically include the following steps:

[0115] Step 21: estimating the wind noise pollution degree of the first microphone;

[0116] Step 22: determining the wind noise suppression gain according to the wind noise pollution degree of the first microphone for each wind noise frame;

[0117] Step 23: performing wind noise suppression processing on the first signal collected by the first microphone by using the wind noise suppression gain for each wind noise frame.

[0118] In the specific implementation, the wind noise pollution degree of the first microphone can be estimated by using the wind noise pollution degree estimation method of the multi-microphone device provided in any of the above embodiments. The specific implementation scheme of the wind noise pollution degree of the first microphone is described in the above Figure 1 and related embodiments, which will not be described here.

[0119] In the specific implementation, the step 22 can be implemented in the following manner, specifically: calculating the initial wind noise suppression gain of each wind noise frame according to the coherence coefficient of the first microphone and the second microphone for each wind noise frame; and correcting the initial wind noise suppression gain by using the wind noise pollution degree of the first microphone, and taking the corrected wind noise suppression gain as the wind noise suppression gain.

[0120] In some embodiments, the initial wind noise suppression gain of each wind noise frame can be calculated based on the coherence weight method. That is, the initial wind noise suppression gain is obtained based on the coherence coefficient.

[0121] Reference Figure 3 A mapping relationship diagram of the coherence coefficient and the initial wind noise suppression gain is shown. The mapping relationship between the coherence coefficient and the initial wind noise suppression gain is described in the above Figure 3In some embodiments, the mapping relationship between the coherence coefficient and the initial wind noise suppression gain can be represented by a gradient function. For example, when the coherence coefficient is in [0, a], the initial wind noise suppression gain takes A. When the coherence coefficient is in [a, b], the initial wind noise suppression gain takes where B is the initial wind noise suppression gain corresponding to the coherence coefficient b, and n is any coherence coefficient between [a, b]. When the coherence coefficient is in [b, 1], the initial wind noise suppression gain takes B.

[0122] Referring to Figure 4 , another mapping relationship between the coherence coefficient and the initial wind noise suppression gain is given. In other embodiments, the mapping relationship between the coherence coefficient and the initial wind noise suppression gain can be represented by a curve function.

[0123] In yet other embodiments, the coherence coefficient of each frame of the sampled signal is taken as the initial wind noise suppression gain of each frame of the sampled signal.

[0124] It should be noted that other types of functions can also be used to represent the mapping relationship between the coherence coefficient and the initial wind noise suppression gain, which will not be exemplified one by one here.

[0125] In some embodiments, the range of the modified wind noise suppression gain and the initial wind noise suppression gain can both be [0, 1]. The range of the coherence coefficient can be [0, 1].

[0126] In a specific implementation, the initial wind noise suppression gain is modified according to the wind noise pollution degree of the first microphone, and the modified wind noise suppression gain is taken as the wind noise suppression gain. Specifically, it can include: for each wind noise frame, calculating the difference between 1 and the initial wind noise suppression gain; performing a multiplication operation on the wind noise pollution degree of the first microphone and the difference to obtain a multiplication result; and taking the difference between 1 and the multiplication result as the modified wind noise suppression gain corresponding to the first microphone.

[0127] In some non-limiting embodiments, the modified wind noise suppression gain corresponding to the first microphone can be obtained by using the following formula (12).

[0128] gain1(m, k) = 1 - (1 - gain'1(m, k)) * P1(m); (12)

[0129] where gain1(m, k) is the modified wind noise suppression gain corresponding to the first microphone, gain'1(m, k) is the initial wind noise suppression gain, and P1(m) is the wind noise pollution degree of the first microphone.

[0130] It should be noted that the wind noise suppression can be performed on the main microphone, or can be performed on other microphones in the multi-microphone device. When the wind noise suppression needs to be performed on other microphones, the same calculation method as the wind noise suppression gain of the first microphone can be used to obtain the wind noise suppression gain of other microphones.

[0131] In some non-limiting embodiments, the wind noise suppression processing is performed on the signal collected by the second microphone using the modified wind noise suppression gain corresponding to the second microphone.

[0132] For the calculation scheme of the modified wind noise suppression gain corresponding to other microphones, refer to the description of the relevant part of the modified wind noise suppression gain corresponding to the first microphone in the above embodiments, which will not be repeated here.

[0133] In a specific implementation, for each frame of sampling signal, the wind noise suppression gain corresponding to the frame of sampling signal can be used to implement gain on the sampling signal to obtain a wind noise suppression processed sampling signal. The wind noise suppression processed sampling signal is converted from the frequency domain to the time domain to obtain a wind noise suppressed signal, thereby realizing wind noise suppression.

[0134] In some embodiments, taking the first microphone as an example, after obtaining the wind noise suppression gain, the wind noise suppression gain can be used to apply gain to the first signal collected by the first microphone to obtain a wind noise suppression processed result.

[0135] The sampling signal is processed to obtain a frequency domain signal, and the wind noise suppression gain corresponding to each frame of sampling signal is used to implement gain on the frequency domain signal, which can be multiplication of the wind noise suppression gain and the frequency domain signal. The frequency domain signal can be a spectrum signal.

[0136] Using the above scheme, for each wind noise frame, the wind noise suppression gain is determined according to the wind noise pollution degree of the main microphone (the first microphone), and then when the wind noise suppression is performed on the first signal collected by the first microphone based on the wind noise suppression gain, the wind noise suppression can be performed in a corresponding degree in combination with the actual wind noise pollution degree of the first microphone, which better balances the wind noise suppression degree and reduces the voice loss, improves the wind noise suppression effect, and further improves the voice processing effect.

[0137] In one specific implementation of step 23, the wind noise suppression gain can be used to perform wind noise suppression processing on the first signal collected by the first microphone in a fixed cutoff frequency range.

[0138] In another specific implementation of step 23, the wind noise suppression gain can be used to perform wind noise suppression processing on the first signal collected by the first microphone in the full frequency bandwidth.

[0139] In another specific implementation of step 23, the wind noise pollution range of the first microphone can be estimated. Within the wind noise pollution range of the first microphone, the first signal collected by the first microphone is subjected to wind noise suppression processing using the wind noise suppression gain corresponding to the first microphone.

[0140] In a specific implementation, for the wind noise frame, the wind noise pollution range of the first microphone is estimated according to the low-frequency energy of the first signal collected by the first microphone, a set wind noise boundary threshold, and a conversion coefficient, the wind noise pollution range being used to indicate a frequency band polluted by wind noise, and the conversion coefficient being used to represent the relationship between wind noise energy and wind noise pollution range.

[0141] In a specific implementation, for each microphone, the difference between the low-frequency energy of the signal collected by the microphone and the set wind noise boundary threshold can be calculated, and the quotient of the difference and the conversion coefficient can be calculated, and the wind noise pollution range can be determined according to the obtained quotient.

[0142] In some non-limiting embodiments, the wind noise pollution range can be estimated using the following formula (13):

[0143]

[0144] wherein f R (m) is the right boundary of the wind noise pollution range of the first microphone; E Low1 (m) is the low-frequency average amplitude spectrum of the first microphone; thr2 is the wind noise boundary threshold; and k(m) is the conversion coefficient.

[0145] In some embodiments, after the right boundary of the wind noise pollution range is obtained, the wind noise pollution range can be [0, f R (m)].

[0146] When the wind noise pollution range of the first microphone is determined, thr2 takes the wind noise boundary threshold corresponding to the first microphone, and k(m) takes the conversion coefficient corresponding to the first microphone.

[0147] Correspondingly, when the wind noise pollution range of the second microphone is determined, E Low1 (m) is replaced by the low-frequency average amplitude spectrum E Low2 (m) of the second microphone, thr2 takes the wind noise boundary threshold corresponding to the second microphone, and k(m) takes the conversion coefficient corresponding to the second microphone.

[0148] When the number of multi-microphone devices is three or more, the microphone with the minimum low-frequency energy among the signals collected by the microphones can be selected as the first microphone (i.e., the main microphone), and the wind noise pollution levels of the other microphones can be calculated by combining each of the other microphones with the first microphone. For each combination, the initial wind noise suppression gain of the other microphone with respect to the first microphone is calculated, and then the initial wind noise suppression gain is corrected according to the wind noise pollution level of the other microphone to obtain the wind noise suppression gain of the other microphone.

[0149] When wind noise suppression is performed, the signals collected by each microphone can be subjected to wind noise suppression processing according to the number of output channels required by the system and according to the wind noise suppression gain corresponding to each microphone.

[0150] For example, the input signal is multi-channel, but the number of output channels is single, and for a wind noise frame, the first microphone is used as the output channel, and the wind noise suppression gain of the first microphone is used, i.e., the wind noise suppression gain obtained by correcting the initial wind noise suppression gain according to the wind noise pollution level of the first microphone when the first microphone is combined with the second microphone.

[0151] For example, the number of output channels is two, and the two microphones with the minimum wind noise pollution are selected as the first microphone and the second microphone according to the wind noise pollution level from small to large. The first microphone and the second microphone are used as the output channels. For a wind noise frame, the first signal collected by the first microphone is subjected to wind noise suppression processing using the wind noise suppression gain of the first microphone, and the second signal collected by the second microphone is subjected to wind noise suppression processing using the wind noise suppression gain of the second microphone. The second microphone is the microphone with the minimum wind noise pollution among the other microphones.

[0152] For example, the number of output channels is three. The three microphones with the minimum wind noise pollution are selected as the first microphone, the second microphone, and the third microphone according to the wind noise pollution level from small to large. The first microphone, the second microphone, and the third microphone are used as the output channels. The first microphone and the second microphone are combined to calculate the wind noise suppression gain of the first microphone and the wind noise suppression gain of the second microphone. The first microphone and the third microphone are combined to calculate the wind noise suppression gain of the third microphone. For a wind noise frame, the first signal collected by the first microphone is subjected to wind noise suppression processing using the wind noise suppression gain of the first microphone, the second signal collected by the second microphone is subjected to wind noise suppression processing using the wind noise suppression gain of the second microphone, and the third signal collected by the third microphone is subjected to wind noise suppression processing using the wind noise suppression gain of the third microphone.

[0153] Further, the wind noise boundary threshold can be determined in the following manner: for each microphone, according to the signal collected by each microphone, the energy of all noise included in the signal collected by each microphone and the energy of other noise, which is noise other than wind noise in the all noise, are determined; and the wind noise boundary threshold is corrected according to the proportion relationship between the energy of the other noise and the energy of the all noise.

[0154] In some embodiments, the correction of the wind noise boundary threshold can mean that the preset wind noise boundary threshold is increased or decreased according to the proportion relationship between the energy of the other noise and the energy of the all noise.

[0155] For example, if the other noise is large, such as the proportion relationship between the energy of the other noise and the energy of the all noise is greater than a set proportion threshold, the preset wind noise boundary threshold can be increased, that is, the corrected wind noise boundary threshold is greater than the preset wind noise boundary threshold. For another example, if the other noise is small, such as the proportion relationship between the energy of the other noise and the energy of the all noise is less than a set proportion threshold, the preset wind noise boundary threshold can be decreased, that is, the corrected wind noise boundary threshold is less than the preset wind noise boundary threshold. For another example, if the proportion relationship between the energy of the other noise and the energy of the all noise is equal to the set proportion threshold, the preset wind noise boundary threshold is not corrected.

[0156] It should be noted that, on the basis of the above wind noise suppression method, other conventional noise reduction algorithms or artificial intelligence (AI) noise reduction algorithms can be used to further improve the noise reduction effect.

[0157] It should be noted that, in addition to being used to correct the initial wind noise gain, the estimated wind noise pollution degree of the first microphone can also be used as a parameter update basis for a noise reduction or echo module, for example, when the wind noise is greater, the noise reduction degree in post-processing is also increased.

[0158] The embodiment of the present application also provides a wind noise pollution degree estimation device of a multi-microphone device, which can be used to implement the wind noise pollution degree estimation method of the multi-microphone device provided in any of the above embodiments. Figure 5 The structure of the wind noise pollution degree estimation device of the multi-microphone device in the embodiment of the present application is shown in the structure diagram of the wind noise pollution degree estimation device of the multi-microphone device. The wind noise pollution degree estimation device 50 of the multi-microphone device comprises:

[0159] The acquisition unit 51 is configured to acquire a plurality of frames of collected signals, each frame of collected signal comprising signals collected by each microphone in the multi-microphone device, at least comprising a first signal collected by a first microphone and a second signal collected by a second microphone;

[0160] The wind noise detection unit 52 is configured to perform wind noise detection on signals collected by each microphone, and to select wind noise frames from the multiple frames of collected signals;

[0161] The wind noise pollution degree estimation unit 53 is configured to estimate, for the wind noise frames, a wind noise pollution degree of the first microphone, which is a main microphone, according to a relationship between low-frequency energy of the first signal collected by the first microphone and low-frequency energy of the second signal collected by the second microphone, the wind noise pollution degree being used to indicate a degree of influence of wind noise on the microphone.

[0162] In specific implementations, the specific working principle and working process of the wind noise pollution degree estimation apparatus 50 of the multi-microphone device can be referred to the descriptions of the wind noise pollution degree estimation method of the multi-microphone device provided in the above embodiments, which will not be repeated here.

[0163] In specific implementations, the wind noise pollution degree estimation apparatus 50 of the multi-microphone device can correspond to a chip with a wind noise pollution degree estimation function in the multi-microphone device, such as a SOC (System-On-a-Chip), a baseband chip, etc.; or a chip module including a chip with a wind noise pollution degree estimation function; or a chip module with a data processing function, or a multi-microphone device.

[0164] The embodiments of the present application also provide a wind noise suppression apparatus of a multi-microphone device, which can be used to implement the wind noise suppression method of the multi-microphone device provided in any of the above embodiments. Figure 6 FIG. 1 shows a structure diagram of a wind noise suppression apparatus of a multi-microphone device according to an embodiment of the present application, which comprises:

[0165] The wind noise pollution degree estimation apparatus 50 of the multi-microphone device provided in any of the above embodiments;

[0166] The calculation unit 61 is configured to calculate, for each wind noise frame, a wind noise suppression gain of each wind noise frame;

[0167] The wind noise suppression gain determination unit 62 is configured to determine, for each wind noise frame, a wind noise suppression gain according to the wind noise pollution degree of the first microphone;

[0168] The wind noise suppression processing unit 63 is configured to perform, for each wind noise frame, wind noise suppression processing on the first signal collected by the first microphone using the wind noise suppression gain.

[0169] In specific implementation, more details of the working principle and working process of the wind noise suppression device 60 of the multi-microphone device can be found in the above-mentioned multi-microphone device wind noise pollution estimation method and wind noise suppression method, and will not be repeated here.

[0170] In specific implementation, the wind noise suppression device 60 of the multi-microphone device can correspond to a chip with wind noise suppression function in the multi-microphone device, such as SOC (System-On-a-Chip), baseband chip, etc.; or correspond to a chip module including a chip with wind noise suppression function; or correspond to a chip module with data processing function, or correspond to a multi-microphone device.

[0171] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is run by a processor to execute the steps of the multi-microphone device wind noise pollution estimation method provided by any of the above-mentioned embodiments of the present application, or the steps of the multi-microphone device wind noise suppression method provided by any of the above-mentioned embodiments of the present application.

[0172] The computer readable storage medium can include a non-volatile memory or a non-transitory memory, and can also include an optical disc, a mechanical hard disk, a solid state disk, etc.

[0173] Specifically, in the embodiment of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0174] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (Read-Only Memory, ROM for short), a programmable read-only memory (Programmable ROM, PROM for short), an erasable programmable read-only memory (Erasable PROM, EPROM for short), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM for short) or a flash memory. The volatile memory can be a random access memory (Random Access Memory, RAM for short) used as an external cache. By way of example but not limitation, many forms of random access memory (Random Access Memory, RAM for short) are available, such as static random access memory (Static RAM, SRAM for short), dynamic random access memory (DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM for short), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM for short), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM for short), synchronous link dynamic random access memory (Synchlink DRAM, SLDRAM for short) and direct memory bus random access memory (Direct Rambus RAM, DR RAM for short).

[0175] The embodiments of the present application also provide a terminal, comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor executes the steps of the wind noise pollution degree estimation method of the multi-microphone device provided by any of the above embodiments or the steps of the wind noise suppression method of the multi-microphone device provided by any of the above embodiments when running the computer program.

[0176] The memory and the processor are coupled, and the memory can be located in the terminal or outside the terminal. The memory and the processor can be connected through a communication bus.

[0177] The terminal can include, but is not limited to, a headset, a hearing aid, a vehicle terminal, a mobile phone, a computer, a tablet computer and other terminal devices, and can also be a server, a cloud platform, etc.

[0178] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer program can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired or wireless manner.

[0179] In several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the above-described device embodiments are only illustrative; for example, the division of the units is only a logical function division, and actual implementation can have another division manner; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. The units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0180] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically included separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a hardware plus software function unit. For example, for each device or product applied to or integrated in a chip, each module / unit contained therein can be realized in the form of a circuit or hardware, or at least part of the modules / units can be realized in the form of a software program running on a processor integrated in the chip, and the remaining (if any) part of the modules / units can be realized in the form of a circuit or hardware; for each device or product applied to or integrated in a chip module, each module / unit contained therein can be realized in the form of a circuit or hardware, and different modules / units can be located in the same component (for example, a chip, a circuit module, etc.) or different components of the chip module, or at least part of the modules / units can be realized in the form of a software program running on a processor integrated in the chip module, and the remaining (if any) part of the modules / units can be realized in the form of a circuit or hardware; for each device or product applied to or integrated in a terminal, each module / unit contained therein can be realized in the form of a circuit or hardware, and different modules / units can be located in the same component (for example, a chip, a circuit module, etc.) or different components of the terminal, or at least part of the modules / units can be realized in the form of a software program running on a processor integrated in the terminal, and the remaining (if any) part of the modules / units can be realized in the form of a circuit or hardware.

[0181] It should be understood that the term "and / or" herein only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein represents an "or" relationship between the associated objects before and after it.

[0182] The "multiple" appearing in the embodiments of the present application means two or more.

[0183] The first, second, third, etc. description appearing in the embodiments of the present application is only for illustrative and distinguishing description objects, and has no order, and does not represent a special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0184] It should be noted that the serial numbers of the steps in the embodiments do not represent a limitation on the execution order of the steps.

[0185] Although the present application has been disclosed with reference to the above embodiments, the application is not limited to the above embodiments. It will be apparent to those skilled in the art that various modifications and changes can be made thereto without departing from the spirit and scope of the application. The scope of the application should be limited only by the appended claims.

Claims

1. A method for estimating the degree of wind noise pollution in a multi-microphone device, characterized in that, The method comprises the following steps: acquiring a plurality of frames of collected signals, each frame of the collected signals comprising signals collected by each microphone in the multi-microphone device, at least comprising a first signal collected by a first microphone and a second signal collected by a second microphone; performing wind noise detection on the signals collected by each microphone, and screening wind noise frames from the plurality of frames of collected signals; for the wind noise frames, estimating a wind noise pollution degree of the first microphone according to a relationship between a low-frequency energy of the first signal collected by the first microphone and a low-frequency energy of the second signal collected by the second microphone, the first microphone being a main microphone, and the wind noise pollution degree being used to indicate a degree of influence of wind noise on the microphone; wherein the estimating of the wind noise pollution degree of the first microphone according to the relationship between the low-frequency energy of the first signal collected by the first microphone and the low-frequency energy of the second signal collected by the second microphone comprises: acquiring a first adjustment coefficient, correcting a maximum value between the low-frequency energy of the first signal and the low-frequency energy of the second signal by using the first adjustment coefficient, obtaining a reference value according to the corrected value, and the first adjustment coefficient being inversely related to the low-frequency energy of the signal collected by each microphone; estimating the wind noise pollution degree of the first microphone according to a ratio of the low-frequency energy of the first signal to the reference value.

2. The method according to claim 1, wherein the obtaining of the reference value according to the corrected value comprises: acquiring a second adjustment coefficient, correcting the corrected value by using the second adjustment coefficient to obtain the reference value, and the second adjustment coefficient being used to represent a background noise energy in the signal with a larger low-frequency energy between the first signal and the second signal.

3. The method of estimating the wind noise contamination level of a multi-microphone device according to claim 1, wherein, The estimating of the wind noise pollution degree of the first microphone according to the ratio of the low-frequency energy of the first signal to the reference value comprises: acquiring a third adjustment coefficient, correcting the low-frequency energy of the first signal by using the third adjustment coefficient to obtain a corrected low-frequency energy of the first signal, and the third adjustment coefficient being used to represent a background noise energy in the low-frequency energy of the first signal; estimating the wind noise pollution degree of the first microphone according to a ratio of the corrected low-frequency energy of the first signal to the reference value.

4. The method of estimating the degree of wind noise contamination of a multi-microphone device according to claim 2 or 3, wherein The wind noise pollution degree of the first microphone is estimated by using the following formula: ； wherein, is a wind noise pollution degree of the first microphone for the mth frame; is a low frequency energy of the second signal of the second microphone in a low frequency range defined by the low frequency cutoff frequency point is a low frequency energy of the first signal of the first microphone in a low frequency range defined by the low frequency cutoff frequency point is a third adjustment coefficient; is a first adjustment coefficient; is a second adjustment coefficient; is taken as and the maximum value in is a low frequency energy of the second signal of the second microphone in a low frequency range defined by the low frequency cutoff frequency point is a low frequency energy of the first signal of the first microphone in a low frequency range defined by the low frequency cutoff frequency point 5. The method of estimating the wind noise contamination level of a multi-microphone device according to claim 1, wherein, The method further comprises the following steps: estimating a wind noise pollution degree of the second microphone according to a ratio of the low-frequency energy of the second signal to the reference value.

6. The method of estimating the wind noise contamination level of a multi-microphone device according to claim 1, wherein, The method further comprises the following steps: when the number of microphones included in the multi-microphone device is greater than or equal to three, taking a microphone with the smallest low-frequency energy of the signals as the first microphone according to the signals collected by each microphone; combining each of the other microphones in the multi-microphone device with the first microphone, and estimating wind noise pollution degrees of the other microphones, the other microphones being the microphones in the multi-microphone device except the first microphone; wherein the second microphone is a microphone with the smallest wind noise pollution degree among the other microphones.

7. A wind noise suppression method of a multi-microphone device, characterized by, The method comprises the following steps: The wind noise pollution degree of the first microphone is estimated by using the wind noise pollution degree estimation method of the multi-microphone device according to any one of claims 1 to 6; For each wind noise frame, a wind noise suppression gain is determined according to the wind noise pollution degree of the first microphone; For each wind noise frame, the first signal collected by the first microphone is subjected to wind noise suppression processing by using the wind noise suppression gain.

8. The wind noise suppression method of a multi-microphone device according to claim 7, wherein, The determination of the wind noise suppression gain according to the wind noise pollution degree of the first microphone comprises: For each wind noise frame, an initial wind noise suppression gain of each wind noise frame is calculated according to the coherence coefficient of the first microphone and the second microphone; The initial wind noise suppression gain is corrected by using the wind noise pollution degree of the first microphone, and the corrected wind noise suppression gain is taken as the wind noise suppression gain.

9. The wind noise suppression method of a multi-microphone device according to claim 8, wherein, The correction of the initial wind noise suppression gain by using the wind noise pollution degree of the first microphone, and the taking of the corrected wind noise suppression gain as the wind noise suppression gain, comprises: For each wind noise frame, the difference between 1 and the initial wind noise suppression gain is calculated; The wind noise pollution degree of the first microphone is multiplied by the difference to obtain a multiplication result; The difference between 1 and the multiplication result is taken as the corrected wind noise suppression gain corresponding to the first microphone.

10. The wind noise suppression method of a multi-microphone device according to claim 7, wherein, Further comprising: The signal collected by the second microphone is subjected to wind noise suppression processing by using the corrected wind noise suppression gain corresponding to the second microphone. 11.The method of claim 7, wherein, Further comprising: For the wind noise frame, the wind noise pollution range of the first microphone is estimated according to the low-frequency energy of the first signal collected by the first microphone, the set wind noise boundary threshold value, and the conversion coefficient, the wind noise pollution range is used to indicate the frequency band polluted by wind noise, and the conversion coefficient is used to represent the relationship between wind noise energy and wind noise pollution range.

12. The wind noise suppression method of a multi-microphone device according to claim 10, wherein, For each wind noise frame, the first signal collected by the first microphone is subjected to wind noise suppression processing by using the wind noise suppression gain, comprising: Within the wind noise pollution range of the first microphone, the first signal collected by the first microphone is subjected to wind noise suppression processing by using the wind noise suppression gain corresponding to the first microphone.

13. A device for estimating the degree of wind noise pollution in a multi-microphone device, characterized in that, Comprise: An acquisition unit is configured to acquire a plurality of frames of collected signals, each frame of collected signal comprising signals collected by each microphone in the multi-microphone device, at least comprising a first signal collected by a first microphone and a second signal collected by a second microphone; A wind noise detection unit is configured to perform wind noise detection on the signals collected by each microphone, and to screen wind noise frames from the plurality of frames of collected signals; A wind noise pollution degree estimation unit is configured to estimate, for each wind noise frame, a wind noise pollution degree of the first microphone according to a relationship between a low-frequency energy of the first signal collected by the first microphone and a low-frequency energy of the second signal collected by the second microphone, the first microphone being a main microphone, and the wind noise pollution degree being used to indicate an influence degree of wind noise on the microphone. The wind noise pollution degree estimation unit is configured to obtain a first adjustment coefficient, correct a maximum value between a low-frequency energy of the first signal and a low-frequency energy of the second signal by using the first adjustment coefficient, obtain a reference value according to the corrected value, and estimate the wind noise pollution degree of the first microphone according to a ratio between the low-frequency energy of the first signal and the reference value, wherein the first adjustment coefficient is inversely related to the low-frequency energy of the signal collected by each microphone.

14. A wind noise suppression device for a multi-microphone equipment, characterized in that, The wind noise pollution degree estimation apparatus of the multi-microphone device according to claim 13; a calculation unit configured to calculate, for each wind noise frame, a wind noise suppression gain of each wind noise frame; a wind noise suppression gain determination unit configured to determine, for each wind noise frame, a wind noise suppression gain according to the wind noise pollution degree of the first microphone; a wind noise suppression processing unit configured to perform wind noise suppression processing on the first signal collected by the first microphone by using the wind noise suppression gain for each wind noise frame. The computer program is configured to perform the steps of the wind noise pollution degree estimation method of the multi-microphone device according to any one of claims 1 to 6 or the steps of the wind noise suppression method of the multi-microphone device according to any one of claims 7 to 12 when executed by the processor.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, The processor is configured to perform the steps of the wind noise pollution degree estimation method of the multi-microphone device according to any one of claims 1 to 6 or the steps of the wind noise suppression method of the multi-microphone device according to any one of claims 7 to 12 when executing the computer program.

16. A terminal comprising a memory and a processor, said memory having stored thereon a computer program capable of running on said processor, characterized in that, ​

Images