Speech Enhancement Method Based on Nonlinear Residual Noise Estimation

Abstract

The present invention discloses a speech enhancement method based on nonlinear residual noise estimation, including: obtaining a complex spectrum of a noisy speech stream; calculating the noise power spectrum of the complex spectrum based on nonlinear residual noise estimation; calculating the posterior signal-to-noise ratio and the prior signal-to-noise ratio, and updating the gain when speech exists according to the posterior signal-to-noise ratio and the prior signal-to-noise ratio; estimating the gain when speech does not exist and the probability of speech existence; calculating the overall gain according to the gain when speech exists, the gain when speech does not exist and the probability of speech existence, and then calculating the spectral amplitude of the denoised speech signal; synthesizing the time-domain denoised speech signal of the corresponding frame based on the spectral amplitude of the denoised speech signal and the phase of the complex spectrum of the noisy speech of the same frame, and then obtaining the time-domain denoised speech signal of the noisy speech stream. The method of the present invention not only improves the estimated noise level, but also does not cause excessive noise reduction, and finally achieves the improvement of the noise reduction effect.

Description

Technical Field

[0001] The invention belongs to the field of speech noise reduction, and in particular relates to a speech enhancement method based on nonlinear residual noise estimation. Background Art

[0002] Clear and intelligible speech is the key to ensuring the performance of speech processing applications. In real-world environments, widespread noise can affect the quality of speech signals, leading to performance degradation in applications such as communications and speech processing. In order to obtain clear and intelligible speech, speech noise reduction methods are often used to process noisy speech.

[0003] Traditional speech noise reduction methods are still widely used because they are easy to deploy, especially the IMCRA-OMLSA-based speech noise reduction method, in which IMCRA (improved minimum controlled recursive averaging) is used to estimate noise, and OMLSA (optimally modified log-spectral amplitude estimator) is used to estimate the denoised speech gain to complete noise reduction. However, the noise level estimated based on IMCRA is slightly smaller than the actual noise. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a speech enhancement method based on nonlinear residual noise estimation to enhance the noise reduction effect based on improved noise estimation.

[0005] Technical solution: The speech enhancement method based on nonlinear residual noise estimation of the present invention comprises the following steps:

[0006] Get the complex spectrum of the noisy speech stream;

[0007] Based on the nonlinear residual noise estimation, the noise power spectrum of the complex spectrum is calculated; wherein the initial noise power spectrum is the square of the initial complex spectrum amplitude, and the noise power spectrum of other frames includes the noise power spectrum estimated by IMCRA and the nonlinear residual noise power spectrum;

[0008] Calculate the posterior signal-to-noise ratio and the prior signal-to-noise ratio, and update the gain when speech exists based on the OMLSA method according to the posterior signal-to-noise ratio and the prior signal-to-noise ratio;

[0009] Estimate the gain when speech is not present and the probability of speech being present;

[0010] Based on the OMLSA method, the overall gain is calculated according to the gain when speech exists, the gain when speech does not exist and the probability of speech existence, and then the spectral amplitude of the denoised speech signal is calculated;

[0011] Based on the spectral amplitude of the denoised speech signal and the phase of the complex spectrum of the noisy speech in the same frame, the time-domain denoised speech signal of the corresponding frame is synthesized to obtain the time-domain denoised speech signal of the noisy speech stream.

[0012] Further, obtaining the complex spectrum of the noisy speech stream includes: , after frame division and windowing, Fourier transform is performed to obtain the complex spectrum .

[0013] Further, based on the nonlinear residual noise estimation, the noise power spectrum of the complex spectrum is calculated; including:

[0014] Noise Power Spectrum The calculation formula is:

[0015] ,

[0016] in, The first The complex spectrum of frequency points, is the first estimated by IMCRA Frame No. The noise power spectrum of the frequency point, For the Frame No. The nonlinear residual noise power spectrum of frequency points is For the Frame No. The probability of speech existence at a frequency point.

[0017] Furthermore, The expression is:

[0018] ,

[0019] in, and is the coefficient, For the Frame No. The a posteriori signal-to-noise ratio of a frequency point.

[0020] Furthermore, the posterior signal-to-noise ratio calculation formula is:

[0021] ,

[0022] in, For the Frame No. The a posteriori signal-to-noise ratio of the frequency point is For the Frame No. The complex spectrum of frequency points, is the noise power spectrum.

[0023] Furthermore, the a priori signal-to-noise ratio calculation formula is:

[0024] ,

[0025] in, For the Frame No. The prior signal-to-noise ratio of the frequency point, is the smoothing coefficient of the prior signal-to-noise ratio, When the voice exists Frame No. The gain at each frequency point, For the Frame No. The a posteriori signal-to-noise ratio of the frequency point is For the Frame No. The a posteriori signal-to-noise ratio of a frequency point.

[0026] Further, according to the posterior signal-to-noise ratio and the prior signal-to-noise ratio, based on the OMLSA method, the gain when the speech exists is updated; including:

[0027] The gain update expression when speech exists is:

[0028] ,

[0029] in, When the voice exists Frame No. The gain at each frequency point, For the Frame No. The prior signal-to-noise ratio of the frequency point, is the integration variable, is an intermediate variable, and the expression is: , For the Frame No. The a posteriori signal-to-noise ratio of a frequency point.

[0030] Further, the gain when speech does not exist and the probability of speech existence are estimated; including:

[0031] Gain when speech is not present Estimated by IMCRA method or obtained based on empirical values;

[0032] Estimation of speech probability based on IMCRA method , the expression is:

[0033] ,

[0034] in, is the probability of no speech, is an intermediate variable, and the expression is: , For the Frame No. The a posteriori signal-to-noise ratio of the frequency point is For the Frame No. The prior signal-to-noise ratio of a frequency point.

[0035] Furthermore, the method for calculating the spectral amplitude of the denoised speech signal comprises the following steps:

[0036] (1) Calculate the overall gain , the expression is:

[0037] ,

[0038] in, When the voice exists Frame No. Gain at each frequency point of Power, When the voice does not exist Frame No. Gain at each frequency point of Power, is the probability of speech existence;

[0039] (2) Calculate the spectral amplitude of the denoised speech signal , the expression is:

[0040] ,

[0041] in, For the Frame No. The complex spectrum amplitude of the frequency point.

[0042] The speech enhancement system corresponding to the method includes:

[0043] A data processing unit, used for obtaining a complex frequency spectrum of a noisy speech stream;

[0044] A noise power spectrum calculation unit, used to calculate the noise power spectrum of the complex spectrum based on the nonlinear residual noise estimation; wherein the initial noise power spectrum is the square of the initial complex spectrum amplitude, and the noise power spectrum of other frames includes the noise power spectrum estimated by IMCRA and the nonlinear residual noise power spectrum;

[0045] A signal-to-noise ratio calculation unit, used for calculating a posteriori signal-to-noise ratio and a priori signal-to-noise ratio;

[0046] A gain updating unit, used for updating the gain when speech exists based on the OMLSA method according to the posterior signal-to-noise ratio and the prior signal-to-noise ratio;

[0047] An estimation unit, used for estimating the gain when speech is not present and the probability of speech being present;

[0048] A denoised speech signal spectrum amplitude calculation unit is used to calculate the overall gain based on the gain when speech exists, the gain when speech does not exist and the probability of speech existence based on the OMLSA method, and then calculate the denoised speech signal spectrum amplitude;

[0049] The time-domain denoised speech signal synthesis unit is used to synthesize the corresponding frame time-domain denoised speech signal based on the spectral amplitude of the denoised speech signal and the phase of the complex spectrum of the noisy speech in the same frame, so as to obtain the time-domain denoised speech signal of the noisy speech stream.

[0050] Beneficial effects: Compared with the prior art, the significant technical effects of the present invention are: nonlinear residual noise estimation is performed based on a nonlinear function, and the increased residual noise size is adjusted according to the posterior signal-to-noise ratio, and limited to a set range, which not only improves the estimated noise level, but also ensures that the residual noise that is reduced due to the increase of the posterior signal-to-noise ratio will not lead to excessive noise reduction, ultimately achieving an improved noise reduction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0052] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] like Figure 1 As shown, the method of the present invention comprises the following steps:

[0054] Step 1: Get a noisy speech stream , after frame division and windowing, Fourier transform is performed to obtain the complex spectrum .

[0055] Includes: ,in For a clean speech signal, is a noise signal; first Frames are divided according to a certain frame length and frame shift, and then windows are added to prevent spectrum leakage. Finally, short-time Fourier transform is performed to obtain the complex spectrum. Frame No. The complex spectrum of the frequency point is , then the complex spectrum amplitude of this frequency point is . With The symbols represent the Frame No. Frequency point, Frame total The sampling rate of the noisy speech stream can be selected as 16kHz, the frame length can be selected as 256 sampling points, the frame shift can be selected as 160 sampling points, and the window function used for windowing can be selected as the Hamming window.

[0056] Step 2: Calculate the noise power spectrum based on the nonlinear residual noise estimate ; The calculation formula is:

[0057] (1),

[0058] in, The first The complex spectrum of frequency points, is the first estimated by IMCRA Frame No. The noise power spectrum of the frequency point, For the Frame No. The nonlinear residual noise power spectrum of frequency points is For the Frame No. The probability of speech existence at a frequency point is hour, It can be calculated from step 6. Based on the nonlinear function sigmoid: Calculate, then The expression is as follows:

[0059] (2),

[0060] in, is a coefficient, which is generally an empirical value. In this embodiment, it can be selected as 0.1. The maximum value will not exceed . is a coefficient, usually an empirical value, used to control The slope of The bigger, When unchanged, The smaller the value of The optional value is 10. For the Frame No. The a posteriori signal-to-noise ratio of the frequency point is When is 1, It is calculated from step 3. Then When increasing, When the signal-to-noise ratio is large, the value of the estimated nonlinear residual noise decreases accordingly, while when the signal-to-noise ratio is small, Increase.

[0061] Step 3: Calculate the posterior signal-to-noise ratio based on the IMCRA method , the expression is as follows:

[0062] (3),

[0063] Step 4: Calculate the prior signal-to-noise ratio based on the IMCRA method , the expression is as follows:

[0064] (4),

[0065] in, is the smoothing coefficient of the priori signal-to-noise ratio, which can be selected as 0.92; When the voice exists Frame No. The gain at each frequency point, For the Frame No. The a posteriori signal-to-noise ratio of a frequency point.

[0066] Step 5: According to and , based on the OMLSA method, update the first Frame No. Gain at each frequency point , The initial value is 1, and the update expression is:

[0067] (5),

[0068] in, is the integration variable, is an intermediate variable, and the expression is:

[0069] (6),

[0070] Step 6: Estimate the gain in the absence of speech And the probability of speech presence .

[0071] Gain when speech is not present It can be estimated by the IMCRA method or obtained based on empirical values.

[0072] In this embodiment, the gain when there is no speech is taken as an empirical value, which can be selected as 0.052.

[0073] Estimation of speech probability based on IMCRA method , as shown below:

[0074] (7),

[0075] in, is the probability of no speech, and the calculation process is shown in steps 601 to 606.

[0076] Step 601: Obtain a first smoothed noisy speech power spectrum :

[0077] (8),

[0078] in, For the Frame No. The first smoothed noisy speech power spectrum of frequency points, For the Frame No. The first smoothed noisy speech power spectrum of frequency points, For the Frame No. The first spectrum smoothed power spectrum of the frequency point, is the smoothing coefficient of the first smoothed noisy speech power spectrum, which can be selected as 0.9 in this embodiment.

[0079] Formula (8) represents Frame No. The first smoothed noisy speech power spectrum at frequency points By Frame No. The first smoothed noisy speech power spectrum at frequency points Pass Frame No. The first spectrum smoothed power spectrum of the frequency point Decide.

[0080] No. Frame No. The first spectrum smoothed power spectrum of the frequency point Expression based on the power spectrum of noisy speech , as shown below:

[0081] (9),

[0082] in, is the smoothing window function, which can be selected as [0.25, 0.5, 0.25]; is a variable, is an integer, ; For the Frame No. The complex spectrum of frequency points.

[0083] Formula (9) shows that Frame No. The smoothing result of the frequency points is calculated by and adjacent frequency points.

[0084] Step 602: Based on the minimum value tracking method, obtain Frame to Minimum power spectrum of frame-noised speech :

[0085] (10),

[0086] in, For the Frame No. The first smoothed noisy speech power spectrum of frequency points, For the frame, Is a positive integer.

[0087] Step 603: Based on the decision function For rough voice activity detection, the minimum value is regarded as noise, and two signal-to-noise ratios are roughly defined as follows:

[0088] (11),

[0089] in, is the coefficient, For the Frame to Minimum power spectrum of frame-noised speech, is the a posteriori signal-to-noise ratio threshold based on the minimum power spectrum of noisy speech, is the smoothed signal-to-noise ratio threshold; in formula (11), 1 means that speech does not exist, and 0 means that speech exists. , and are empirical values, which can be selected as 1.66, 4.6, and 1.67 respectively. The decision function indicates that when the power spectrum of the noisy speech is equal to the smooth power spectrum, When the minimum smoothed power spectrum ratios in a frame are all less than a specific value, that is, when the roughly estimated signal-to-noise ratio value is small, it is considered that speech does not exist.

[0090] Step 604: Based on Frame No. The decision function of a frequency point , get the second smoothed noisy power spectrum :

[0091] (12),

[0092] in, For the Frame No. The second smoothed noisy power spectrum of the frequency points, For the Frame No. The second smoothed noisy power spectrum of the frequency points, For the Frame No. The second spectrum smoothed power spectrum of the frequency points, is the smoothing coefficient of the first smoothed noisy speech power spectrum, which can be selected as 0.9 in this embodiment.

[0093] Similar to step 601, By Frame No. The second smoothed noisy power spectrum of the frequency points Pass Frame No. The second spectrum smoothed power spectrum of the frequency point Decision. Frame No. The second spectrum smoothed power spectrum of the frequency point The expression is as follows:

[0094] (13),

[0095] in, For the Frame No. The decision function of the frequency point.

[0096] Formula (13) represents Frame No. There is any adjacent frequency point of the frequency point When it is 1, that is, when the voice does not exist, update , otherwise it is set equal to the value of the previous frame.

[0097] Step 605: Based on the minimum value tracking method, obtain Minimum power spectrum of noisy speech from frame to frame l :

[0098] (14),

[0099] Step 606: Calculate the probability of no speech ;

[0100] A posteriori signal-to-noise ratio based on the minimum power spectrum of noisy speech and smoothed signal-to-noise ratio for:

[0101] (15),

[0102] (16),

[0103] The optional value is 1.66, which means the prior probability of no speech for:

[0104] (17),

[0105] Used to adjust the slope, which is an empirical value and can be selected as 3. For the smoothing signal-to-noise ratio threshold, it can be selected as 1.67. is the a posteriori signal-to-noise ratio based on the minimum power spectrum of noisy speech, is the smoothed signal-to-noise ratio based on the minimum power spectrum of noisy speech.

[0106] Step 7: Based on the OMLSA method, , and Calculating overall gain , and then calculate the spectral amplitude of the denoised speech signal .

[0107] Step 701: Calculate the overall gain based on the OMLSA method , the expression is as follows:

[0108] (18),

[0109] in, When the voice exists Frame No. Gain at each frequency point of Power, When the voice does not exist Frame No. Gain at each frequency point of Power, is the probability of speech existence.

[0110] Step 702: Calculate the spectral amplitude of the denoised speech signal , the expression is as follows:

[0111] (19),

[0112] Step 8: Based on the spectral amplitude of the denoised speech signal and noisy speech The phase of the complex spectrum of the frame is synthesized Frame time domain denoising speech signal The inverse Fourier transform can be used to synthesize the time-domain denoised speech signal.

[0113] Step 9: Repeat steps 2 to 8 in order from frame 0 to the last frame.

[0114] The system corresponding to the above method includes:

[0115] A data processing unit, used for obtaining a complex frequency spectrum of a noisy speech stream;

[0116] A noise power spectrum calculation unit, used to calculate the noise power spectrum of the complex spectrum based on the nonlinear residual noise estimation; wherein the initial noise power spectrum is the square of the initial complex spectrum amplitude, and the noise power spectrum of other frames includes the noise power spectrum estimated by IMCRA and the nonlinear residual noise power spectrum;

[0117] A signal-to-noise ratio calculation unit, used for calculating a posteriori signal-to-noise ratio and a priori signal-to-noise ratio;

[0118] A gain updating unit, used for updating the gain when speech exists based on the OMLSA method according to the posterior signal-to-noise ratio and the prior signal-to-noise ratio;

[0119] An estimation unit, used for estimating the gain when speech is not present and the probability of speech being present;

[0120] A denoised speech signal spectrum amplitude calculation unit is used to calculate the overall gain based on the gain when speech exists, the gain when speech does not exist and the probability of speech existence based on the OMLSA method, and then calculate the denoised speech signal spectrum amplitude;

[0121] The time-domain denoised speech signal synthesis unit is used to synthesize the corresponding frame time-domain denoised speech signal based on the spectral amplitude of the denoised speech signal and the phase of the complex spectrum of the noisy speech in the same frame, so as to obtain the time-domain denoised speech signal of the noisy speech stream.

[0122] Experimental simulation:

[0123] The present invention adopts the original IMCRA-OMLSA method and the improved IMCRA-OMLSA based on nonlinear residual noise estimation to reduce the noise of the original speech respectively. The SNR of the noise reduction by the original IMCRA-OMLSA method is 12.9826, and the SNR of the noise reduction by the method of the present invention is 12.9829, which is 0.23‰ higher than that of the original IMCRA-OMLSA method, proving the effectiveness of the method proposed by the present invention.

Claims

1. A speech enhancement method based on nonlinear residual noise estimation, characterized in that: The following steps are involved: Get the complex spectrum of the noisy speech stream; Based on the nonlinear residual noise estimation, the noise power spectrum of the complex spectrum is calculated; wherein the initial noise power spectrum is the square of the initial complex spectrum amplitude, and the noise power spectrum of other frames includes the noise power spectrum estimated by IMCRA and the nonlinear residual noise power spectrum; Calculate the posterior signal-to-noise ratio and the prior signal-to-noise ratio, and update the gain when speech exists based on the OMLSA method according to the posterior signal-to-noise ratio and the prior signal-to-noise ratio; Noise Power Spectrum The calculation formula is: ， in, The first The complex spectrum of frequency points, is the first estimated by IMCRA Frame No. The noise power spectrum of the frequency point, For the Frame No. The nonlinear residual noise power spectrum of frequency points is For the Frame No. The probability of speech existence at each frequency point; The expression is: ， in, and is the coefficient, For the Frame No. The a posteriori signal-to-noise ratio of each frequency point; Estimate the gain when speech is not present and the probability of speech being present; Based on the OMLSA method, the overall gain is calculated according to the gain when speech exists, the gain when speech does not exist and the probability of speech existence, and then the spectral amplitude of the denoised speech signal is calculated; Based on the spectral amplitude of the denoised speech signal and the phase of the complex spectrum of the noisy speech in the same frame, the time-domain denoised speech signal of the corresponding frame is synthesized to obtain the time-domain denoised speech signal of the noisy speech stream.

2. The speech enhancement method based on nonlinear residual noise estimation according to claim 1, characterized in that: Obtain the complex spectrum of the noisy speech stream, including: , after frame division and windowing, Fourier transform is performed to obtain the complex spectrum .

3. The speech enhancement method based on nonlinear residual noise estimation according to claim 1, characterized in that: The a posteriori signal-to-noise ratio calculation formula is: ， in, For the Frame No. The a posteriori signal-to-noise ratio of the frequency point is For the Frame No. The complex spectrum of frequency points, is the noise power spectrum.

4. The speech enhancement method based on nonlinear residual noise estimation according to claim 1, characterized in that: The a priori signal-to-noise ratio calculation formula is: ， in, For the Frame No. The prior signal-to-noise ratio of the frequency point, is the smoothing coefficient of the prior signal-to-noise ratio, When the voice exists Frame No. The gain at each frequency point, For the Frame No. The a posteriori signal-to-noise ratio of the frequency point is For the Frame No. The a posteriori signal-to-noise ratio of a frequency point.

5. The speech enhancement method based on nonlinear residual noise estimation according to claim 1, characterized in that: According to the posterior signal-to-noise ratio and the prior signal-to-noise ratio, the gain when speech exists is updated based on the OMLSA method; include: The gain update expression when speech exists is: ， in, When the voice exists Frame No. The gain at each frequency point, For the Frame No. The prior signal-to-noise ratio of the frequency point, is the integration variable, is an intermediate variable, and the expression is: , For the Frame No. The a posteriori signal-to-noise ratio of a frequency point.

6. The speech enhancement method based on nonlinear residual noise estimation according to claim 1, characterized in that: Estimate the gain when speech is not present and the probability of speech presence; including: Gain when speech is not present Estimated by IMCRA method or obtained based on empirical values; Estimation of speech probability based on IMCRA method , the expression is: ， in, is the probability of no speech, is an intermediate variable, and the expression is: , For the Frame No. The a posteriori signal-to-noise ratio of the frequency point is For the Frame No. The prior signal-to-noise ratio of a frequency point.

7. The speech enhancement method based on nonlinear residual noise estimation according to claim 1, characterized in that: The method for calculating the spectral amplitude of a denoised speech signal comprises the following steps: (1) Calculate the overall gain , the expression is: ， in, When the voice exists Frame No. Gain at each frequency point of Power, When the voice does not exist Frame No. Gain at each frequency point of Power, is the probability of speech existence; (2) Calculate the spectral amplitude of the denoised speech signal , the expression is: ， in, For the Frame No. The complex spectrum amplitude of the frequency point.

8. A speech enhancement system based on nonlinear residual noise estimation, characterized in that: include: A data processing unit, used for obtaining a complex frequency spectrum of a noisy speech stream; A noise power spectrum calculation unit, used to calculate the noise power spectrum of the complex spectrum based on the nonlinear residual noise estimation; wherein the initial noise power spectrum is the square of the initial complex spectrum amplitude, and the noise power spectrum of other frames includes the noise power spectrum estimated by IMCRA and the nonlinear residual noise power spectrum; Noise Power Spectrum The calculation formula is: ， in, The first The complex spectrum of frequency points, is the first estimated by IMCRA Frame No. The noise power spectrum of the frequency point, For the Frame No. The nonlinear residual noise power spectrum of frequency points is For the Frame No. The probability of speech existence at each frequency point; The expression is: ， in, and is the coefficient, For the Frame No. The a posteriori signal-to-noise ratio of each frequency point; A signal-to-noise ratio calculation unit, used for calculating a posteriori signal-to-noise ratio and a priori signal-to-noise ratio; A gain updating unit, used for updating the gain when speech exists based on the OMLSA method according to the posterior signal-to-noise ratio and the prior signal-to-noise ratio; An estimation unit, used for estimating the gain when speech is not present and the probability of speech being present; A denoised speech signal spectrum amplitude calculation unit is used to calculate the overall gain based on the gain when speech exists, the gain when speech does not exist and the probability of speech existence based on the OMLSA method, and then calculate the denoised speech signal spectrum amplitude; The time-domain denoised speech signal synthesis unit is used to synthesize the corresponding frame time-domain denoised speech signal based on the spectral amplitude of the denoised speech signal and the phase of the complex spectrum of the noisy speech in the same frame, so as to obtain the time-domain denoised speech signal of the noisy speech stream.

Images