An adaptive noise reduction method and post-filter based on frequency point gain smoothing

Abstract

The present invention relates to an adaptive noise reduction method and a post-filter based on frequency gain smoothing. The present invention converts a speech signal into a spectrum signal, and then performs gain, and performs smoothing processing on the gain process, and the smoothing processing includes forward smoothing and / or backward smoothing; through the processing mode of forward smoothing and backward smoothing, the frequency gain of speech conversion is smoothed according to the algorithm processing, and the new gain replaces the original gain, so as to obtain smooth and natural frequency data. Accordingly, the present method can improve the naturalness of speech after noise reduction, especially in the scene with low signal-to-noise ratio; the present method significantly reduces the "music noise" caused by excessive noise reduction and spectrum mutation; the present method is a general method, which can be applied to different noise reduction algorithms (models) based on different parameters or variants; the present method is simple to implement and has low computational complexity, and can be combined with other post-filters to achieve better noise reduction and speech preservation effects.

Description

Technical Field

[0001] The present invention relates to a technology for processing audio frequency points, and in particular to a technology for avoiding distortion caused by excessive gain of audio frequency points through gain smoothing processing. Background Art

[0002] Audio signals usually contain noise. In audio processing solutions represented by intercom calls and recording, adaptive noise reduction (ANS) technology is one of the most widely used technologies. There are different types of noise reduction technologies based on different models. A key problem they encounter is the fidelity of speech: that is, many algorithms will damage speech while reducing noise. As an academic research solution, it can obtain a very high signal-to-noise ratio and is considered a feasible solution. However, in actual application, it is difficult to accept the damage to the hearing of speech, and even "music noise" will appear when the signal-to-noise ratio is very low.

[0003] To solve this problem, there are some existing technical solutions, among which the binary mask method is widely used: this method sets only two gains of 0 or 1 for each frequency point, that is, it determines whether it belongs to speech or noise. If it belongs to noise, it is completely removed, and if it belongs to speech, it is fully released. The speech obtained in this way can improve the above situation in some cases, but it is not natural enough in listening.

[0004] To better solve this problem, we can use the adjacent correlation between frequency points, that is, the gain change between similar frequency points should not be very drastic; that is, we can improve the voice quality based on the gain of the existing noise reduction algorithm while avoiding the gain mutation like the binary mask method. Summary of the invention

[0005] The main purpose of the present invention is to provide a method for smoothing the gain of speech frequency points through an algorithm to avoid the problem of sudden change and distortion of data.

[0006] In order to achieve the above object, the present invention provides an adaptive noise reduction method based on frequency point gain smoothing, which converts the speech signal into a spectrum signal and then performs gain, and performs smoothing processing on the gain process, and the smoothing processing includes forward smoothing and / or backward smoothing;

[0007] Among them, forward smoothing:

[0008] Pick ;

[0009] right ;

[0010] in, is the gain attenuation coefficient;

[0011] Replace the original gain with the new gain. ;

[0012] in, It is the virtual gain of the frequency point other than the highest frequency, used as the initial point of calculation;

[0013] For each actual frequency point, is its original gain, which is obtained by the noise reduction algorithm;

[0014] The adjusted gain of the previous higher frequency sound;

[0015] Alternatively, backsmoothing:

[0016] Pick ;

[0017] right ;

[0018] Replace the original gain with the new gain. ;

[0019] It is the virtual gain of the frequency point other than the lowest frequency, used as the initial point of calculation;

[0020] For each actual frequency point, Its original gain, which is obtained by the noise reduction algorithm or from the previous adjustment;

[0021] The adjusted gain for the previous lower frequency sound.

[0022] Preferably, time smoothing is also included;

[0023] Pick ;

[0024] have ;

[0025] Replace the original gain with the new gain. ;

[0026] Here Is the current frequency Previous gain.

[0027] Preferably, this is accomplished by the following steps:

[0028] Preprocessing: real-time speech signal is divided into frames, and the signal is converted into spectrum through FFT transformation, which is steps 1 and 2;

[0029] ANS noise reduction uses a certain noise reduction algorithm to obtain gain, which is step 3;

[0030] Post-filtering: adjust the above gains, i.e. steps 4, 5 and 6;

[0031] Back to the time domain signal: convert the spectrum to the time domain through inverse FFT transform and synthesize it into an audio signal, steps 7 and 8.

[0032] Preferably, this is accomplished by the following steps:

[0033] Step 1: real-time speech signal framing:

[0034] The voice signal is the signal flow after device sampling or previous algorithm processing. ;

[0035] Each sampling point has a certain number of bits. Sampling according to a certain number of bits requires normalization so that ;

[0036] In real-time processing, take each sampling points is a frame, that is, The frame signal is ,in ;

[0037] Step 2, speech signal Transformed into spectrum by FFT ;in,

[0038] is the analysis window;

[0039] is the discrete Fourier transform;

[0040] is the complex spectrum, where , is the frequency point number;

[0041] Step 3: Use a signal processing noise reduction algorithm to obtain the gain at each frequency point ,Generally, , that is, the gain is a real number, which means that is the spectrum after noise reduction; ;

[0042] Step 4, i.e. adopting the forward smoothing step;

[0043] Step 5, i.e. adopting the backward smoothing step;

[0044] Step 6, namely adopting the time smoothing.

[0045] Preferably, the method further comprises the following steps:

[0046] Step 7: Update the spectrum signal: ;

[0047] Step 8: After inverse Fourier transform, it returns to the time domain signal: ;

[0048] The analysis window used for the synthesized signal is determined according to the windowing method.

[0049] Preferably, when and When using the Hanning window, the composite window is the unit window;

[0050] Use Overlap-add method to Synthesize into speech signal frame by frame .

[0051] Preferably, for an audio file with a sampling rate of 16KHz, 32ms is taken as one frame, and the frame shift is 16ms, that is, .

[0052] Preferably, the attenuation coefficients in forward smoothing and backward smoothing are The values ​​are the same or different;

[0053] Attenuation coefficient Adjust according to the signal-to-noise ratio, that is, when the signal-to-noise ratio is high Reduce, when the signal-to-noise ratio is low improve;

[0054] Replace with .

[0055] The present invention also provides an adaptive noise reduction post-filter based on frequency point gain smoothing, and the post-filter is processed by the noise reduction method as described above.

[0056] Preferably, independently set post filters are used for processing during forward smoothing, backward smoothing and time smoothing.

[0057] The beneficial effects of the present invention are as follows: through the forward smoothing and backward smoothing processing methods of the present invention, the frequency point gain of the speech conversion is smoothed according to the algorithm processing, and the new gain replaces the original gain, thereby obtaining smooth and natural frequency point data. Accordingly, the present method can improve the naturalness of the speech after noise reduction, especially in the scene with low signal-to-noise ratio; the present method significantly reduces the "music noise" caused by excessive noise reduction and spectrum mutation; the present method is a general method and can be applied to different noise reduction algorithms (models) based on different parameters or variants; the present method is simple to implement and has low computational complexity, and can be combined with other post-filters to achieve better noise reduction and speech preservation effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1This is a demonstration diagram of the audio data before and after processing of the present invention. The upper side is the time axis from 25ms to 39ms. The figure is divided into four areas to show the processing changes of the audio. DETAILED DESCRIPTION

[0059] The technical scheme in the embodiment of the present invention is described clearly and completely below in conjunction with the accompanying drawings in the embodiment of the present invention. In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0060] The principle of the present invention is explained as follows:

[0061] A speech is a time series , which can be seen as two audios combined, one is voice , one is noise ,Right now, The purpose of noise reduction algorithm or speech separation algorithm is to Bundle Calculate it.

[0062] Because the sound is processed bit by bit, in real time, each time we deal with a small segment of data such as tens of milliseconds, which is step 1, frame division. We usually process the speech in the frequency domain, which is step 2. In this way, corresponding to the frequency domain, the noisy speech , , The calculation process is the content of a noise reduction algorithm, that is, step 3. It is assumed that speech and noise are independent random variables, and the signal processing algorithm does not change the phase. Therefore, when we estimate a speech , which means that we believe For example, usually, we order , are the power spectra of noisy speech and noise respectively, then, is an estimate of the noise gain. In terms of calculation examples, for example, assuming that the energy of the noisy (original) speech is 100 and the noise energy is 10, then a simple estimate is: the speech energy is 90, that is, the gain is g=0.9. Step 7 is to apply this gain to Step 8 is to return from the frequency domain to the time domain, that is, to obtain the voice stream we usually hear.

[0063] An actual noise reduction algorithm will need to consider the speed of noise change, the accuracy of noise estimation, etc. (obeying a certain statistical model). Most of them will tend to have more severe noise reduction, so that the noise will be almost gone and the speech will be greatly damaged. This is not a problem in the paper, because the paper looks at indicators, not the listening experience; but in practical application, it will become a problem: in practical application, we would rather allow some noise than cause great damage to the speech. Therefore, what steps 4 / 5 / 6 below do is this: the principle is to maintain the speech by preventing sudden changes in gain. This process will not reduce the gain, but only increase it. The basic principle of the increase is: if the gain was 0.9 last time, and it suddenly becomes 0.1 this time, it is considered to have changed too much. We set a coefficient, such as =0.8 Then, the next minimum gain of 0.9 is 0.9*0.8=0.72. Steps 4 / 5 / 6 are to do this "next" in three directions: to low frequency, to high frequency and to the next frame. As a result, the signal will be more stable and the listening experience will be better.

[0064] For each actual frequency point, Its original gain, which is obtained by the noise reduction algorithm, that is, the frequency point is obtained by the existing noise reduction algorithm The noise reduction algorithm is calculated iteratively on the frequency point by frame, and does not take into account the gain of adjacent frequency points, so a very aggressive gain will be obtained, that is, unsmooth or unnatural speech will be produced; therefore, it is necessary to Smoothing is performed using the above algorithm.

[0065] like Figure 1 As shown in FIG. 1 , it is a demonstration of the effect of the data algorithm after being processed by the method of the present invention. The first channel above is the effect of the original noise reduction algorithm, and the second channel below is enhanced by the method of this invention. It can be seen that after being processed by the method of this invention, many details in the speech domain are restored, and it can also be noticed that the unsmoothness (empty space) is supplemented to a certain extent, such as 28-29 seconds (the second area), 31-32 seconds (the third area), etc. Example 1

[0066] This embodiment provides an adaptive noise reduction method based on frequency point gain smoothing, which converts a speech signal into a spectrum signal and then performs gain, and performs smoothing processing on the gain process, wherein the smoothing processing includes forward smoothing and / or backward smoothing;

[0067] Among them, the forward smoothing of step 4 is:

[0068] Pick ;

[0069] right ;

[0070] in, is the gain reduction coefficient; in the main scheme, is a fixed gain reduction coefficient, ,like Then the post-filtering does not affect the original signal. The smaller it is, the more it tends to retain the original gain. The larger it is, the smoother the gain;

[0071] Replace the original gain with the new gain. ;

[0072] in, is the virtual gain of the frequency point other than the highest frequency. This frequency point will not be used in practice. It is used here as the starting point of the calculation. Specifically, is the frequency point sequence, when When taking the highest frequency point, due to the existence of So take , which is the frequency point outside the highest frequency.

[0073] For example, in a specific processing case, when processing data with a 16k sampling rate signal, a 16ms frame shift, and a 32ms frame length, The frequency domain signal range is {0,1,…,255}. At this time, the starting point of the calculation from back to front is 256, which is not in this range. Set g(256)=0, then max{0,g(255)}=g(255). According to the induction method, we can gradually get the value of 254, …, the value of 1 from the value of 255, and finally get the value of 0, completing the entire calculation process.

[0074] Similarly, the starting point of the calculation from back to front starts from -1, and is calculated from -1 to 0, 0 to 1, ..., 254 to 255, and the time smoothing is the same.

[0075] The gain of the previous higher frequency sound is adjusted. For the sake of audio quality, we do not want a huge gap between the gains of adjacent frequencies, so we use Adjust for smoothing factor;

[0076] Thus Set to the adjusted frequency gain;

[0077] This gain is used for subsequent adjustments, so setting ;

[0078] In addition, the backward smoothing of step 5:

[0079] Pick ;

[0080] right ;

[0081] Replace the original gain with the new gain. ;

[0082] is the virtual gain of the frequency point other than the lowest frequency. This frequency point will not be used in practice. It is used here as the starting point of calculation. When taking the lowest frequency point 0, due to the existence of So take , that is, the frequency point outside the lowest frequency. Please refer to the description of the aforementioned "handling case".

[0083] For each actual frequency point, Its original gain, which is obtained by the noise reduction algorithm or from the previous adjustment;

[0084] The adjustments in the frequency domain forward, frequency domain backward, and time domain are independent, and the input for each adjustment is a set of gains, that is, function nesting, and the order of nested functions can be adjusted at will.

[0085] For example, the gain of the original noise reduction algorithm is g, the forward filtering algorithm in the frequency domain is Apre, the backward filtering is Apost, and the time domain filtering is Atime; then, the algorithm structure of this article can be very flexible, such as Atime(Apre(Apost)) is the order described here, Apre( Apost ( Atime(g))) is another order, etc. Therefore, "amount adjustment" is achieved through these methods.

[0086] The noise reduction algorithm is calculated iteratively on this frequency point by frame, without considering the gain of adjacent frequency points, thus obtaining a very aggressive gain; similarly, the previous post-filtering does not include adjustments from low frequencies upwards;

[0087] The gain of the previous lower frequency point is adjusted. For the sake of audio quality, we do not want a huge gap between the gains of adjacent frequencies. Therefore, we use Adjust for smoothing factor;

[0088] Thus Set to the adjusted frequency gain;

[0089] This gain can be used for subsequent adjustments, so setting ;

[0090] Preferably, it also includes time smoothing of step 6;

[0091] Pick ;

[0092] have ;

[0093] Replace the original gain with the new gain. ;

[0094] The gain calculation of the noise reduction algorithm is derived from the estimation of the signal-to-noise ratio, and almost does not include the gain comparison adjustment of the previous and next frames. Considering that the signal between the previous and next frames has considerable continuity, the sudden change in signal gain between them is often the cause of signal quality degradation or poor hearing. Therefore, it is necessary to keep the signal gain smooth to a certain extent in adjacent frames.

[0095] Similar to the previous description, here Is the current frequency Previous Specifically, A sequence of frames. That is, starting from -1 (virtual frame), calculate frame 0, then frame 1, then frame 2, ... This is a sequence of natural numbers that represents the order of real-time calculation. Example 2

[0096] Based on Example 1, preferably, the process is completed by the following steps:

[0097] Preprocessing: real-time speech signal is divided into frames, and the signal is converted into a spectrum through FFT (Fourier transform), which is steps 1 and 2;

[0098] ANS noise reduction (Automatic Noise Suppression, background noise suppression), using a noise reduction algorithm to obtain gain, which is step 3;

[0099] Post-filtering: adjust the above gains, i.e. steps 4, 5 and 6;

[0100] Back to the time domain signal: convert the spectrum to the time domain through inverse FFT transform and synthesize it into an audio signal, that is, steps 7 and 8.

[0101] Step 1: real-time speech signal framing:

[0102] The voice signal is the signal flow after device sampling or previous algorithm processing. ;

[0103] Each sampling point has a certain number of bits. Sampling according to a certain number of bits, such as 16-bit sampling, needs to be normalized so that ;

[0104] In real-time processing, take each sampling points is a frame, that is, The frame signal is ,in ; Preferably, for an audio file with a sampling rate of 16KHz, 32ms is taken as one frame, and the frame shift is 16ms, that is .Right now is the number of sampling points in one frame, is the number of sampling points of the frame shift length.

[0105] Step 2, speech signal Transformed into spectrum by FFT ;in,

[0106] is the analysis window;

[0107] is the discrete Fourier transform;

[0108] is the complex spectrum, where , is the frequency point number;

[0109] Step 3: Use the existing signal processing noise reduction algorithm to obtain the gain at each frequency point ,Generally, , that is, the gain is a real number, which means that is the spectrum after noise reduction; ;

[0110] Step 4, using the forward smoothing step in Example 1;

[0111] Step 5, adopting the backward smoothing step in Example 1;

[0112] Step 6: adopt the time smoothing step in Example 1.

[0113] Step 7: Update the spectrum signal: ;

[0114] Step 8: After inverse Fourier transform, it returns to the time domain signal: ;

[0115] According to the windowing method, the analysis window used for the synthesized signal is determined. and When using the Hanning window, the composite window is the unit window;

[0116] Use Overlap-add method to Synthesize into speech signal frame by frame . Example 3

[0117] Preferably, the attenuation coefficients in forward smoothing and backward smoothing are The values ​​are the same or different;

[0118] Attenuation coefficient Adjust according to the signal-to-noise ratio, that is, when the signal-to-noise ratio is high Reduce, when the signal-to-noise ratio is low improve;

[0119] Replace with That is, the typical exponential decay Attack-Decay method is replaced by a simple linear decay method, which will be more effective in certain situations.

[0120] This refers to giving an additional implementation method. Attack-Decay refers to the original implementation method: fast tracking of large values ​​(attack), exponential decay tracking of small values ​​(decay); replace the exponential decay here with linear decay. Example 4

[0121] The present invention also provides an adaptive noise reduction post-filter based on frequency gain smoothing, and the post-filter is processed by the noise reduction method in Embodiments 1-3.

[0122] Preferably, independently set post filters are used for processing during forward smoothing, backward smoothing and time smoothing.

[0123] In order to better illustrate the solution of the present invention, some prior art documents are given below.

[0124] The following overview contains descriptions of common steps (e.g. steps 1-3):

[0125] Mahdi Parchami, Wei-Ping Zhu, Benoit Champagne, and Eric Plourde,Recent Developments in Speech Enhancement in the Short-Time Fourier TransformDomain , July 2016 IEEE Circuits and Systems Magazine 16(3):45-77

[0126] The introduction of classic noise reduction algorithms can be found as follows:

[0127] Y. Ephraim and D. Malah, Speech enhancement using a minimum mean-square error short-time spectral amplitude estimator, IEEE Trans. Acoust.,Speech, Signal Processing, vol. ASSP-32, no. 6, pp. 1109–1121, December 1984.

[0128] A very commonly used noise reduction method is as follows:

[0129] Timo Gerkmann, and Richard C. Hendriks, Unbiased MMSE-Based NoisePower Estimation with Low Complexity and Low Tracking Delay [435 citations]May 2012IEEE Transactions on Audio Speech and Language Processing 20(4):1383-1393,

[0130] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Claims

1. An adaptive noise reduction method based on frequency gain smoothing, which converts the speech signal into a spectrum signal and then performs gain, characterized in that: The gain process is smoothed, and the smoothing includes forward smoothing and / or backward smoothing and / or time smoothing; the adaptive noise reduction method based on frequency point gain smoothing is completed by the following steps: Step 1: real-time speech signal framing: The voice signal is the signal flow after device sampling or previous algorithm processing. ; Each sampling point has a certain number of bits. Sampling according to a certain number of bits requires normalization so that ; In real-time processing, take each sampling points is a frame, that is, The frame signal is ,in ; Step 2, speech signal Transformed into spectrum by FFT ;in, is the analysis window; is the discrete Fourier transform; is the complex spectrum, where , is the frequency point number; Step 3: Use signal processing noise reduction algorithm to obtain the gain at each frequency point , , that is, the gain is a real number, which means that is the spectrum after noise reduction; ; Step 4, i.e. adopting the forward smoothing step; Step 5, i.e. adopting the backward smoothing step; Step 6, i.e. adopting the time smoothing; Among them, forward smoothing: Pick ; right ; in, is the gain attenuation coefficient; Replace the original gain with the new gain. ; in, It is the virtual gain of the frequency point other than the highest frequency, used as the initial point of calculation; For each actual frequency point, is its original gain, which is obtained by the noise reduction algorithm; The adjusted gain of the previous higher frequency sound; Alternatively, backsmoothing: Pick ; right ; Replace the original gain with the new gain. ; It is the virtual gain of the frequency point other than the lowest frequency, used as the initial point of calculation; For each actual frequency point, Its original gain, which is obtained by the noise reduction algorithm or from the previous adjustment; The adjusted gain for the previous lower frequency sound; Additionally, time smoothing: Pick ; have ; Replace the original gain with the new gain. ; Here Is the current frequency Previous gain.

2. The adaptive noise reduction method based on frequency gain smoothing according to claim 1, characterized in that: Complete the following steps: Preprocessing: real-time speech signal is divided into frames, and the signal is converted into spectrum through FFT transformation, which is steps 1 and 2; ANS noise reduction, using the noise reduction algorithm to obtain gain, is step 3; Post-filtering: adjust the above gains, i.e. steps 4, 5 and 6; Back to the time domain signal: convert the spectrum to the time domain through inverse FFT transform and synthesize it into an audio signal, steps 7 and 8.

3. The adaptive noise reduction method based on frequency gain smoothing according to claim 1, characterized in that: It also includes the following steps: Step 7: Update the spectrum signal: ; Step 8: After inverse Fourier transform, it returns to the time domain signal: ; The analysis window used for the synthesized signal is determined according to the windowing method.

4. The adaptive noise reduction method based on frequency gain smoothing according to claim 3, characterized in that: when and When using the Hanning window, the composite window is the unit window; Use Overlap-add method to Synthesize into speech signal frame by frame .

5. The adaptive noise reduction method based on frequency gain smoothing according to claim 1, characterized in that: For an audio file with a sampling rate of 16KHz, 32ms is taken as one frame, and the frame shift is 16ms, that is, .

6. The adaptive noise reduction method based on frequency gain smoothing according to claim 1, characterized in that: Attenuation coefficient in forward and backward smoothing The values ​​are the same or different; Attenuation coefficient Adjust according to the signal-to-noise ratio, that is, when the signal-to-noise ratio is high Reduce, when the signal-to-noise ratio is low improve; Replace with .

7. An adaptive noise reduction post-filter based on frequency gain smoothing, characterized in that: The post-filter is processed by the noise reduction method according to any one of claims 1 to 6.

8. The adaptive noise reduction post-filter based on frequency gain smoothing according to claim 7, characterized in that: Independently set post filters are used for processing during forward smoothing, backward smoothing and time smoothing.

Images