Wind noise suppression method determination method, device, terminal and storage medium

Abstract

The embodiment of the present application provides a method, device, terminal and storage medium for determining a wind noise suppression mode, and relates to the field of wind noise processing technology. The method comprises: collecting ambient noise through a feedforward microphone of an earphone; performing feature extraction processing on the ambient noise to obtain an audio feature vector of the ambient noise; processing the audio feature vector of the ambient noise through a noise recognition model to obtain a recognition result, and the recognition result is used to indicate whether the ambient noise contains wind noise; based on the recognition result, determining the wind noise suppression mode of the earphone. The embodiment of the present application reduces the volume of the earphone.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of wind noise processing, and in particular to a method, device, terminal and storage medium for determining a wind noise suppression mode. Background Art

[0002] When wind blows onto the sound pickup surface of the headset's microphone, obvious wind noise will be generated in headwind or strong airflow conditions, resulting in a poor user experience.

[0003] In the related technology, the wind speed and wind direction of the wind noise signal acting on the microphone are monitored by a wind direction and speed sensor, and a target wind noise prevention parameter matching the monitored wind speed and wind direction is obtained from at least one wind noise prevention parameter, and each wind noise prevention parameter includes at least one of the following: high-pass filter cutoff frequency, signal gain; then, wind noise prevention processing is performed on the signal collected by the microphone according to the target wind noise prevention parameter. Summary of the invention

[0004] The embodiment of the present application provides a method, device, terminal and storage medium for determining a wind noise suppression mode. The technical solution is as follows:

[0005] On the one hand, an embodiment of the present application provides a method for determining a wind noise suppression mode, the method comprising:

[0006] Ambient noise is collected through the headset’s feed-forward microphone;

[0007] Performing feature extraction processing on the environmental noise to obtain an audio feature vector of the environmental noise;

[0008] Processing the audio feature vector of the ambient noise by a noise recognition model to obtain a recognition result, wherein the recognition result is used to indicate whether the ambient noise contains wind noise;

[0009] Based on the recognition result, a wind noise suppression mode of the headset is determined.

[0010] On the other hand, an embodiment of the present application provides a device for determining a wind noise suppression method, the device comprising:

[0011] A noise collection module, used to collect environmental noise through a feedforward microphone of the headset;

[0012] A feature extraction module, used to perform feature extraction processing on the environmental noise to obtain an audio feature vector of the environmental noise;

[0013] A feature recognition module, used for processing the audio feature vector of the ambient noise through a noise recognition model to obtain a recognition result, wherein the recognition result is used to indicate whether the ambient noise contains wind noise;

[0014] A mode determination module is used to determine the wind noise suppression mode of the headset based on the recognition result.

[0015] On the other hand, an embodiment of the present application provides a terminal, which includes a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the method for determining the wind noise suppression mode as described in the above aspect.

[0016] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the method for determining the wind noise suppression mode as described in the above aspects.

[0017] In another aspect, an embodiment of the present application provides a computer program product, the computer program product including computer instructions, the computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for determining the wind noise suppression mode described in the above aspect.

[0018] The technical solution provided in the embodiments of the present application can bring the following beneficial effects:

[0019] By judging whether wind noise exists based on the audio feature vector of the ambient noise and determining the wind noise suppression method of the earphone based on the recognition result, the embodiment of the present application does not need to install a wind direction and speed sensor in the earphone to determine the wind noise suppression method of the earphone, thereby reducing the size of the earphone. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic diagram of an earphone provided by an embodiment of the present application;

[0021] Figure 2 is a hardware system block diagram of a headset provided by an embodiment of the present application;

[0022] Figure 3 is a flow chart of a method for determining a wind noise suppression mode provided by an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a DNN model provided by an embodiment of the present application;

[0024] Figure 5 is a flow chart of a method for determining a wind noise suppression mode provided by another embodiment of the present application;

[0025] Figure 6 is a schematic diagram of a frequency response of a low-pass filter provided by an embodiment of the present application;

[0026] Figure 7 is a flow chart of a method for determining a wind noise suppression mode provided by another embodiment of the present application;

[0027] Figure 8 is a block diagram of a device for determining a wind noise suppression method provided by an embodiment of the present application;

[0028] Fig. 9 This is a structural block diagram of a terminal provided by an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0030] The embodiments of the present application can be applied to headphones. Figure 1 and Figure 2 The earphone includes a speaker (also called a speaker device) 101, a feedforward microphone 102 (also called an external microphone), a feedback microphone 103 (also called an internal microphone), an audio signal processing chip circuit (not shown in the figure), a storage circuit 104, a power supply circuit 105, etc.

[0031] The speaker 101 is used to play anti-phase noise and music signals. The feedforward microphone 102 is used to collect environmental noise, and the collected environmental noise is used for wind noise suppression processing. The feedback microphone 103 is used to collect the internal sound of the headset. The audio signal processing chip circuit is used to execute the wind noise suppression method determination process. The storage circuit 104 is used to store algorithm code, audio signal data, etc. The power supply circuit 105 can supply power to other hardware components, and the power supply source can be a battery built into the headset.

[0032] It should be noted that the structure of the earphones described above is only exemplary. In possible implementations, the embodiments of the present application may also be applied to other earphones different from the earphones described above. Figure 1 In the earphones with a structure of , the embodiments of the present application are not limited to this.

[0033] For example, the embodiment of the present application is described by taking the embodiment of the headset as an example. Figure 2 As shown, the feedforward microphone 101 of the headset collects environmental noise, converts the collected environmental noise into a digital signal through the analog-to-digital converter 106, extracts the audio feature vector, and inputs the audio feature vector into the trained noise recognition model to obtain the recognition result; if there is wind noise, the microphone gain is adjusted based on the strength of the wind noise; if there is no wind noise, the microphone gain is not adjusted. The microphone gain is used to indicate the volume of the microphone. The larger the microphone gain, the higher the volume of the microphone; the smaller the microphone gain, the lower the volume of the microphone.

[0034] Exemplarily, when the embodiment of the present application is executed by headphones, the wind noise suppression method determination process can be executed by an audio signal processing chip circuit inside the headphones.

[0035] For example, for the convenience of introduction and description, the following will take the execution subject of the embodiment of the present application as an example of a terminal for introduction and description, and the terminal may be a headset, a mobile phone, a smart wearable device, etc.

[0036] Please refer to Figure 3 , which shows a flow chart of a method for determining a wind noise suppression mode provided by an embodiment of the present application. The execution subject of the method may be a terminal. The method may include the following steps.

[0037] Step 301: Collect environmental noise through a feedforward microphone of the headset.

[0038] Exemplarily, the feedforward microphone of the headset collects ambient noise in real time.

[0039] In a possible implementation, when the terminal is a headset, after the feedforward microphone of the headset collects the ambient noise, the ambient noise is sent to the audio signal processing chip circuit in the headset, and the audio signal processing chip circuit executes the following wind noise suppression method determination process.

[0040] In a possible implementation, when the terminal is other electronic device than the headset, the terminal establishes a communication connection with the headset, and the terminal and the headset can communicate, for example, the terminal and the headset can communicate through a wired network, or the terminal and the headset can communicate through a wireless network. After the feedforward microphone of the headset collects the ambient noise, the ambient noise is sent to the terminal, and the terminal executes the following process for determining the wind noise suppression method. Exemplarily, the ambient noise collected by the feedforward microphone is in the form of audio data, and the audio data carries the ambient noise collected by the feedforward microphone. The headset sends the audio data carrying the ambient noise to the terminal, so that the terminal can execute the following process for determining the wind noise suppression method.

[0041] Exemplarily, the earphones in the embodiments of the present application may be wired earphones or wireless earphones; the earphones in the embodiments of the present application may be ANC earphones or ordinary earphones, which is not limited in the embodiments of the present application.

[0042] Step 302: Perform feature extraction processing on the ambient noise to obtain an audio feature vector of the ambient noise.

[0043] After the terminal obtains the ambient noise, since it needs to perform noise recognition processing through a noise recognition model, and the noise recognition model cannot directly recognize the ambient noise, the terminal needs to pre-process the ambient noise to obtain digital features that can be recognized by the noise recognition model. Exemplarily, the terminal performs feature extraction processing on the ambient noise to obtain an audio feature vector of the ambient noise. Exemplarily, the audio feature vector of the ambient noise can be the MFCC (Mel-scale Frequency Cepstral Coefficients) vector of the ambient noise. The MFCC vector is a cepstral parameter extracted in the Mel scale frequency domain. The Mel scale describes the nonlinear characteristics of the human ear frequency. For the feature extraction process of the ambient noise, please refer to the following embodiment, which will not be introduced here.

[0044] Step 303: Process the audio feature vector of the environmental noise through the noise recognition model to obtain a recognition result.

[0045] In the embodiment of the present application, the recognition result is used to indicate whether the ambient noise contains wind noise.

[0046] In a possible implementation, the noise recognition model may adopt a DNN (Deep Neural Networks) model. Figure 4 As shown, the DNN model consists of an input layer 410, three hidden layers 420 and an output layer 430. The input layer 410 is the first layer and has 8 neurons; the three hidden layers 420 include: a first hidden layer, a second hidden layer and a third hidden layer, each of which has 9 neurons; and the output layer 430 has 4 neurons.

[0047] In a possible implementation, the training process of the noise recognition model may include the following steps: 1. Acquire training data for training the noise recognition model, the training data including at least one training audio feature vector and a standard recognition result of the training audio feature vector, the standard recognition result refers to a standard result of whether the environmental noise corresponding to the training audio feature vector contains wind noise; 2. Process the training audio feature vector through the noise recognition model to obtain a predicted recognition result, the predicted recognition result refers to a result of predicting whether the environmental noise corresponding to the training audio feature vector contains wind noise; 3. Train the noise recognition model based on the predicted recognition result and the standard recognition result to obtain a noise recognition model that has been trained. Exemplarily, the loss function of the noise recognition model is determined based on the predicted recognition result and the standard recognition result, and the noise recognition model is trained based on the loss function to obtain a noise recognition model that has been trained. Exemplarily, the wind noise contained in the environmental noise corresponding to the at least one training audio feature vector has different strengths. The weights of each layer of neurons of the noise recognition model are continuously trained and updated by forward operation and back propagation through the training audio feature vector, thereby obtaining a noise recognition model for identifying wind noise.

[0048] Step 304: determine the wind noise suppression mode of the earphone based on the recognition result.

[0049] In a possible implementation, when the recognition result is used to indicate that the ambient noise does not contain wind noise, not performing wind noise suppression processing on the earphone is determined as the wind noise suppression mode of the earphone.

[0050] In a possible implementation, when the recognition result indicates that the ambient noise includes wind noise, the wind noise suppression mode of the headset is determined based on the strength of the wind noise. For an introduction to how to determine the wind noise suppression mode of the headset, please refer to the following embodiment, which will not be described here.

[0051] In a possible implementation, when the terminal is a headset, the headset may perform wind noise suppression processing on the ambient noise collected by the feedforward microphone based on the determined wind noise suppression method.

[0052] In a possible implementation, when the terminal is other electronic device than the headset, after the terminal determines the wind noise suppression mode of the headset, it sends the wind noise suppression mode to the headset, so that the headset can perform wind noise suppression processing on the ambient noise collected by the feedforward microphone based on the wind noise suppression mode.

[0053] To sum up, in the technical solution provided in the embodiment of the present application, whether wind noise exists is determined based on the audio feature vector of the ambient noise, and the wind noise suppression method of the earphone is determined based on the recognition result. The embodiment of the present application does not need to install a wind direction and speed sensor in the earphone to determine the wind noise suppression method of the earphone, thereby reducing the size of the earphone.

[0054] In addition, the embodiment of the present application first detects whether there is wind noise and then determines the wind noise suppression method, which will not affect the feedforward wind noise suppression effect when there is no wind noise.

[0055] In addition, the embodiment of the present application realizes wind noise detection through the DNN model algorithm, which reduces the hardware cost. The algorithm of the noise recognition model has a small amount of calculation and low power consumption, which can improve the battery life of the terminal.

[0056] In an exemplary embodiment, Figure 5 As shown, the terminal obtains the audio feature vector of the ambient noise in the following manner:

[0057] Step 501 , performing frame processing on the audio data corresponding to the environmental noise to obtain the i-th audio frame, where i is a positive integer.

[0058] Since the feedforward microphone collects environmental noise in real time, its audio data is not stable as a whole, but its local part can be regarded as stable data. The noise recognition model can only recognize stable data. Therefore, the terminal first divides the environmental noise into frames to obtain the i-th audio frame.

[0059] In a possible implementation, before the terminal performs frame processing on the audio data corresponding to the ambient noise, it first needs to perform analog-to-digital conversion on the audio data to obtain a digital signal; then the analog-to-digital converted audio data is pre-emphasized. The pre-emphasis process uses a high-pass filter, which only allows signal components above a certain frequency to pass through, while suppressing signal components below the frequency, thereby removing unnecessary low-frequency interference such as human conversations, footsteps, and mechanical noise in the audio data, making the frequency spectrum of the audio signal flat. The mathematical expression of the high-pass filter is:

[0060] H(z)=1-az -1 ;

[0061] Among them, a is the correction coefficient, which generally ranges from 0.95 to 0.97, and z is the audio signal.

[0062] The terminal performs frame processing on the audio data after the pre-emphasis processing to obtain an i-th audio frame.

[0063] Step 502: Perform windowing processing on the i-th audio frame to obtain the i-th audio frame after windowing processing.

[0064] Since the audio data after framing needs to be Fourier transformed in the subsequent feature extraction, and a frame of audio data has no obvious periodicity, that is, the left end and the right end of the frame are discontinuous, errors will occur between the original data and the Fourier transform. The more frames there are, the greater the error. Therefore, in order to make the audio data after framing continuous and each frame of audio data exhibits the characteristics of a periodic function, windowing processing is required.

[0065] In a possible implementation, a rectangular window is used to perform windowing on the audio frame. Each frame of data is multiplied by a rectangular window function to increase the continuity of the left and right ends of the frame, and the obtained audio data has obvious periodicity. The function form of the rectangular window w(n) is:

[0066]

[0067] Wherein, n is a positive integer, N is the length of the window (abbreviated as window length), and N is the number of Fourier transform points.

[0068] Step 503: extract features from the i-th windowed audio frame to obtain the i-th audio feature vector of the ambient noise.

[0069] After the audio data is framed and windowed, feature extraction is required to obtain an audio feature vector that can be recognized by the noise recognition model.

[0070] In a possible implementation, this step includes the following sub-steps:

[0071] 1. Perform short-time Fourier transform processing on the i-th windowed audio frame to obtain the i-th first signal.

[0072] Short-Time Fourier Transform (STFT) is a mathematical transformation related to Fourier transform, which is used to determine the frequency and phase of the local sinusoidal wave of a time-varying signal. In a possible implementation, the expression for the short-time Fourier transform of an audio frame is as follows:

[0073]

[0074] Wherein, n is discrete time, ω is continuous frequency, ω=2πk / N, k=0, 1, ... N-1, N is the number of Fourier transform points, and x(i) is the i-th audio frame.

[0075] 2. Perform low-pass filtering on the i-th first signal based on a preset low-pass cutoff frequency to obtain the i-th second signal.

[0076] The preset low-pass cutoff frequency can be adjusted according to actual conditions. For example, the i-th second signal is obtained by performing low-pass filtering on the i-th first signal based on the preset low-pass cutoff frequency through a low-pass filter. The frequency response (abbreviated as frequency response) of the low-pass filter is as follows: Figure 6 As shown, the main energy of wind noise is concentrated in the low-frequency band, so the calculation of wind noise energy mainly considers the signals in the low-frequency band.

[0077] 3. Perform fast Fourier transform processing on the i-th second signal to obtain the i-th energy spectrum.

[0078] Since it is difficult to obtain the signal characteristics of an audio signal from its transformation in the time domain, it is usually necessary to convert the time domain signal into an energy distribution in the frequency domain for processing. Different energy distributions can represent the characteristics of different voices. Therefore, the terminal first performs a Fourier transform on the second signal, and then calculates the energy spectrum based on the second signal after the Fourier transform. The second signal is fast Fourier transformed to obtain the spectrum of the second signal, and the spectrum of the voice signal is modulo squared to obtain the power spectrum of the voice signal, that is, the energy spectrum.

[0079] 4. Process the i-th energy spectrum through the Mel filter bank to obtain the i-th logarithmic energy.

[0080] In order to convert its energy spectrum into a Mel spectrum that conforms to human hearing, the energy spectrum needs to be filtered. Exemplarily, the energy spectrum is processed by a Mel filter bank, and the expression of the frequency response of the Mel filter bank H(k) can be as follows:

[0081]

[0082] in, M represents the total number of filter banks, k represents the number of Fourier transform points, and M is usually 22-26.

[0083] Exemplarily, each logarithmic energy e(m) may be determined by the following formula:

[0084]

[0085] where |X(k)| 2 Represents the energy spectrum.

[0086] 5. Perform discrete cosine transform on the i-th logarithmic energy to obtain the i-th audio feature vector.

[0087] The terminal performs discrete cosine transform (DCT) processing on the logarithmic energy, and the obtained DCT coefficient is the audio feature vector. Exemplarily, the audio feature vector F(n) can be obtained by performing discrete cosine transform processing on the logarithmic energy using the following formula:

[0088]

[0089] Among them, L refers to the MFCC order, usually L is 12-16; M represents the total number of filter banks; N represents the number of Fourier transform points.

[0090] Exemplarily, the audio feature vector in the embodiment of the present application is an MFCC vector. In other possible implementations, the audio feature vector may also be determined in other ways. The embodiment of the present application does not limit the method for determining the audio feature vector.

[0091] In an exemplary embodiment, after the terminal obtains the i-th audio feature vector of the ambient noise, the terminal may determine the wind noise suppression mode of the earphone based on the i-th audio feature vector. Figure 7 As shown, the terminal can determine the wind noise suppression mode of the headset in the following ways:

[0092] Step 701: Process the ith audio feature vector through a noise recognition model to obtain a recognition result of the ith audio feature vector, where the recognition result of the ith audio feature vector is used to indicate whether wind noise exists in the ith audio frame.

[0093] In a possible implementation, each time the terminal obtains an audio feature vector, the terminal inputs the audio feature vector into a noise recognition model, and the noise recognition model processes the audio feature vector to obtain a recognition result of the audio feature vector. The recognition result of the audio feature vector can be used to indicate that wind noise exists in the audio frame corresponding to the audio feature vector, or the recognition result of the audio feature vector is used to indicate that wind noise does not exist in the audio frame corresponding to the audio feature vector.

[0094] Step 702, determining whether the recognition result indicates whether there is wind noise in the i-th audio frame. If the recognition result indicates that there is wind noise in the i-th audio frame, step 703 is executed; if the recognition result indicates that there is no wind noise in the i-th audio frame, step 704 is executed.

[0095] Step 703: performing wind noise suppression processing on the earphone based on the i-th energy spectrum is determined as the wind noise suppression mode.

[0096] When the recognition result of the audio feature vector indicates that there is wind noise in the audio frame corresponding to the audio feature vector, it means that the headset needs to perform wind noise suppression processing. At this time, the terminal can determine the wind noise suppression processing of the headset based on the energy spectrum corresponding to the audio feature vector as the wind noise suppression mode.

[0097] In a possible implementation, this step includes the following sub-steps:

[0098] First, based on the ith energy spectrum, determine the low-frequency energy corresponding to the ith audio frame.

[0099] In a possible implementation, the low-frequency energy Eng(i) corresponding to the i-th audio frame can be determined by the following formula:

[0100]

[0101] Among them, |X(i, k)| 2 represents the i-th energy spectrum, and N represents the number of Fourier transform points.

[0102] Second, in response to the low-frequency energy corresponding to the i-th audio frame being greater than or equal to the wind noise energy threshold, performing wind noise suppression processing on the earphone based on the low-frequency energy corresponding to the i-th audio frame is determined as the wind noise suppression mode.

[0103] In a possible implementation, after obtaining the low-frequency energy corresponding to the i-th audio frame, the low-frequency energy is smoothed to obtain the smoothed low-frequency energy. Exemplarily, the low-frequency energy can be smoothed by the following formula to obtain the smoothed low-frequency energy Eng_smooth(i):

[0104] Eng_smooth(i)=α*Eng_6mooth(i-1)+(l-α)*Eng(i);

[0105] Wherein, 0<α<l, Eng(i) is the low-frequency energy corresponding to the i-th audio frame, and Eng_smooth(i-1) is the low-frequency energy corresponding to the i-1-th audio frame after smoothing.

[0106] It should be noted that when i is 1, no smoothing process is required, that is, the low-frequency energy corresponding to the first audio frame does not need to be smoothed, or it can be said that the low-frequency energy corresponding to the first audio frame after smoothing is itself.

[0107] Exemplarily, the smoothed low-frequency energy corresponding to the i-th audio frame is compared with the wind noise energy threshold. In response to the smoothed low-frequency energy corresponding to the i-th audio frame being greater than or equal to the wind noise energy threshold, the wind noise suppression processing performed on the earphone based on the smoothed low-frequency energy corresponding to the i-th audio frame is determined as the wind noise suppression mode.

[0108] Exemplarily, from the correspondence between at least one stored low-frequency energy range and microphone gain, the target low-frequency energy range to which the low-frequency energy corresponding to the i-th audio frame belongs is determined; the microphone gain corresponding to the target low-frequency energy range in the correspondence is determined as the target microphone gain; and wind noise suppression processing performed on the earphone based on the target microphone gain is determined as the wind noise suppression method.

[0109] Exemplarily, the terminal stores a correspondence between at least one low-frequency energy range and a microphone gain. The terminal can determine the target low-frequency energy range to which the current low-frequency energy belongs based on the current low-frequency energy, and then determine the corresponding microphone gain based on the low-frequency energy range, and determine the microphone gain as the microphone gain for the final wind noise suppression processing.

[0110] Third, in response to the low-frequency energy corresponding to the i-th audio frame being less than the wind noise energy threshold, not performing wind noise suppression processing on the earphone is determined as the wind noise suppression mode.

[0111] When the low-frequency energy corresponding to the i-th audio frame is less than the wind noise energy threshold, it means that the wind noise is not very loud and the earphone does not need to perform wind noise suppression processing on the wind noise.

[0112] Step 704: determining that no wind noise suppression processing is performed on the earphone is the wind noise suppression mode.

[0113] When the recognition result indicates that the i-th audio frame does not include wind noise, it indicates that the headset does not need wind noise suppression processing, so the terminal determines that no wind noise suppression processing is performed on the headset as the wind noise suppression mode. When the headset does not need wind noise suppression processing, the microphone gain of the headset does not need to be changed, and the headset can perform normal active noise reduction.

[0114] To summarize, in the technical solution provided in the embodiment of the present application, wind noise suppression is performed by adjusting the microphone gain based on the strength of the wind noise; when it is detected that the wind noise disappears or the wind noise energy is small, the optimal feedforward noise reduction effect is restored.

[0115] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0116] Please refer to Figure 8 , which shows a block diagram of a wind noise suppression method determination device provided by an embodiment of the present application, the device has the function of implementing the above method example, the function can be implemented by hardware, or by hardware executing corresponding software. The device 800 may include: a noise collection module 810, a feature extraction module 820, a feature recognition module 830 and a method determination module 840.

[0117] A noise collection module 810 is used to collect environmental noise through a feedforward microphone of the headset;

[0118] A feature extraction module 820 is used to perform feature extraction processing on the environmental noise to obtain an audio feature vector of the environmental noise;

[0119] A feature recognition module 830 is used to process the audio feature vector of the ambient noise through a noise recognition model to obtain a recognition result, wherein the recognition result is used to indicate whether the ambient noise contains wind noise;

[0120] The mode determination module 840 is used to determine the wind noise suppression mode of the headset based on the recognition result.

[0121] To sum up, in the technical solution provided in the embodiment of the present application, whether wind noise exists is determined based on the audio feature vector of the ambient noise, and the wind noise suppression method of the earphone is determined based on the recognition result. The embodiment of the present application does not need to install a wind direction and speed sensor in the earphone to determine the wind noise suppression method of the earphone, thereby reducing the size of the earphone.

[0122] In the exemplary embodiment, the feature extraction module 820 includes: an audio framing unit, an audio windowing unit and a feature extraction unit (not shown in the figure).

[0123] an audio framing unit, configured to perform framing processing on the audio data corresponding to the environmental noise to obtain an i-th audio frame, where i is a positive integer;

[0124] An audio windowing unit, used for performing windowing processing on the i-th audio frame to obtain an i-th audio frame after windowing processing;

[0125] The feature extraction unit is used to perform feature extraction on the i-th audio frame after the windowing process to obtain the i-th audio feature vector of the environmental noise.

[0126] In an exemplary embodiment, the feature extraction unit is used to:

[0127] Performing short-time Fourier transform processing on the i-th windowed audio frame to obtain an i-th first signal;

[0128] Performing low-pass filtering on the i-th first signal based on a preset low-pass cutoff frequency to obtain an i-th second signal;

[0129] Performing fast Fourier transform processing on the i-th second signal to obtain an i-th energy spectrum;

[0130] Processing the i-th energy spectrum through a Mel filter bank to obtain an i-th logarithmic energy;

[0131] Perform discrete cosine transform processing on the i-th logarithmic energy to obtain the i-th audio feature vector.

[0132] In an exemplary embodiment, the feature recognition module 830 is used to:

[0133] The i-th audio feature vector is processed by the noise recognition model to obtain a recognition result of the i-th audio feature vector, and the recognition result of the i-th audio feature vector is used to indicate whether wind noise exists in the i-th audio frame.

[0134] In an exemplary embodiment, the mode determination module 840 is used to:

[0135] In response to the recognition result indicating that wind noise exists in the i-th audio frame, performing wind noise suppression processing on the earphone based on the i-th energy spectrum is determined as the wind noise suppression mode;

[0136] In response to the recognition result indicating that there is no wind noise in the i-th audio frame, not performing wind noise suppression processing on the earphone is determined as the wind noise suppression mode.

[0137] In the exemplary embodiment, the mode determination module 840 includes: an energy determination unit and a mode determination unit (not shown in the figure).

[0138] an energy determination unit, configured to determine, based on the i-th energy spectrum, a low-frequency energy corresponding to the i-th audio frame;

[0139] The mode determination unit is used to determine the wind noise suppression mode by performing wind noise suppression processing on the earphone based on the low-frequency energy corresponding to the i-th audio frame in response to the low-frequency energy corresponding to the i-th audio frame being greater than or equal to the wind noise energy threshold.

[0140] In an exemplary embodiment, the mode determination unit is used to:

[0141] Determine, from a stored correspondence relationship between at least one low-frequency energy range and a microphone gain, a target low-frequency energy range to which the low-frequency energy corresponding to the i-th audio frame belongs;

[0142] Determine the microphone gain corresponding to the target low-frequency energy range in the corresponding relationship as the target microphone gain;

[0143] Performing wind noise suppression processing on the earphone based on the target microphone gain is determined as the wind noise suppression mode.

[0144] In an exemplary embodiment, the mode determination module 840 is further used to:

[0145] In response to the low-frequency energy corresponding to the i-th audio frame being less than the wind noise energy threshold, not performing wind noise suppression processing on the earphone is determined as the wind noise suppression mode.

[0146] It should be noted that the device provided in the above embodiment only uses the division of the above functional modules as an example to implement its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0147] Please refer to Fig. 9 , which shows a structural block diagram of a terminal provided by an embodiment of the present application.

[0148] The terminal in the embodiment of the present application may include one or more of the following components: a processor 910 and a memory 920 .

[0149] The processor 910 may include one or more processing cores. The processor 910 uses various interfaces and lines to connect various parts of the entire terminal, and executes various functions of the terminal and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 920, and calling data stored in the memory 920. Optionally, the processor 910 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 910 can integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 910, but implemented separately through a chip.

[0150] Optionally, the processor 910 implements the methods provided by the above-mentioned various method embodiments when executing program instructions in the memory 920 .

[0151] The memory 920 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 920 includes a non-transitory computer-readable storage medium. The memory 920 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the terminal, etc.

[0152] The structure of the terminal described above is only illustrative. In actual implementation, the terminal may include more or fewer components, such as a display screen, etc. This embodiment does not limit this.

[0153] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than those shown in the figure, or combine certain components, or adopt a different component arrangement.

[0154] In an exemplary embodiment, a computer-readable storage medium is further provided, in which a computer program is stored. The computer program is loaded and executed by a processor of a computer device to implement each step in the above method embodiment.

[0155] In an exemplary embodiment, a computer program product is also provided, the computer program product includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in the above embodiment.

[0156] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining a wind noise suppression mode, characterized in that: The method comprises: Collecting ambient noise through a feedforward microphone of the headset, wherein audio data corresponding to the ambient noise includes multiple audio frames; Performing feature extraction processing on an i-th audio frame among the multiple audio frames to obtain an i-th audio feature vector of the ambient noise, where i is a positive integer; Processing the i-th audio feature vector through a noise recognition model to obtain a recognition result, wherein the recognition result is used to indicate whether the ambient noise contains wind noise; When the recognition result indicates that the ambient noise does not include wind noise, determining that no wind noise suppression processing is performed on the earphone as the wind noise suppression mode of the earphone; In a case where the recognition result is used to indicate that the ambient noise includes wind noise, in response to the low-frequency energy corresponding to the i-th audio frame being greater than or equal to a wind noise energy threshold, determining a target low-frequency energy range to which the low-frequency energy corresponding to the i-th audio frame belongs from a stored correspondence between at least one low-frequency energy range and a microphone gain; determining the microphone gain corresponding to the target low-frequency energy range in the correspondence as a target microphone gain; and determining performing wind noise suppression processing on the headset based on the target microphone gain as the wind noise suppression method; In response to the low-frequency energy corresponding to the i-th audio frame being less than the wind noise energy threshold, not performing wind noise suppression processing on the earphone is determined as the wind noise suppression mode.

2. The method according to claim 1, characterized in that The step of performing feature extraction processing on the i-th audio frame among the multiple audio frames to obtain the i-th audio feature vector of the ambient noise includes: Performing windowing processing on the i-th audio frame to obtain an i-th audio frame after windowing processing; Feature extraction is performed on the i-th windowed audio frame to obtain an i-th audio feature vector of the ambient noise.

3. The method according to claim 2, characterized in that The step of extracting features from the i-th windowed audio frame to obtain the i-th audio feature vector of the ambient noise includes: Performing short-time Fourier transform processing on the i-th windowed audio frame to obtain an i-th first signal; Performing low-pass filtering on the i-th first signal based on a preset low-pass cutoff frequency to obtain an i-th second signal; Performing fast Fourier transform processing on the i-th second signal to obtain an i-th energy spectrum; Processing the i-th energy spectrum through a Mel filter bank to obtain an i-th logarithmic energy; Perform discrete cosine transform processing on the i-th logarithmic energy to obtain the i-th audio feature vector.

4. A device for determining a wind noise suppression method, characterized in that: The device comprises: A noise collection module, used to collect environmental noise through a feedforward microphone of the headset, wherein the audio data corresponding to the environmental noise includes a plurality of audio frames; a feature extraction module, configured to perform feature extraction processing on an i-th audio frame among the multiple audio frames to obtain an i-th audio feature vector of the ambient noise, where i is a positive integer; a feature recognition module, used for processing the i-th audio feature vector through a noise recognition model to obtain a recognition result, wherein the recognition result is used to indicate whether the environmental noise contains wind noise; a mode determination module, configured to determine, when the recognition result indicates that the ambient noise does not include wind noise, that no wind noise suppression processing is performed on the earphone as a wind noise suppression mode of the earphone; The mode determination module is further used for, in a case where the recognition result is used to indicate that the ambient noise contains wind noise, in response to the low-frequency energy corresponding to the i-th audio frame being greater than or equal to a wind noise energy threshold, determining, from a stored correspondence relationship between at least one low-frequency energy range and a microphone gain, a target low-frequency energy range to which the low-frequency energy corresponding to the i-th audio frame belongs; determining the microphone gain corresponding to the target low-frequency energy range in the correspondence relationship as a target microphone gain; and determining the wind noise suppression processing performed on the headset based on the target microphone gain as the wind noise suppression mode; The mode determination module is also used to determine the low-frequency energy corresponding to the i-th audio frame as the wind noise suppression mode, in response to the low-frequency energy corresponding to the i-th audio frame being less than the wind noise energy threshold, not performing wind noise suppression processing on the earphone.

5. A terminal, characterized in that: The terminal includes a processor and a memory, the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the method for determining a wind noise suppression mode according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the method for determining a wind noise suppression mode according to any one of claims 1 to 3.

7. A computer program product, characterized in that The computer program product stores a computer program, and the computer program is loaded and executed by a processor to implement the method for determining a wind noise suppression mode according to any one of claims 1 to 3.

Images