A method for echo cancellation

Abstract

The present invention relates to an echo elimination method, comprising the following steps: adaptively updating a main filter and an auxiliary filter in a dual filter structure, wherein the updating function is as follows: #imgabs0# wherein w(k+1) is a filter updating function, w(k) is a filter original function, u is a step size factor, Q(k) is a step size gain matrix, x(k) is a system input signal, e(k) is an evaluation error, δ is a minimum regularization parameter, and x(k) T The system input signal is transposed; the dual filter structure is used to accelerate the convergence speed of the echo in the sound signal, the echo is linearly processed and the processed sound signal is output; the sound signal after linear processing is nonlinearly processed. The present invention solves the problems of residual echo, noise, fluctuating volume and double-talking mute in existing voice communication by adaptively updating the main filter and auxiliary filter in the dual filter structure, and adding double-talking judgment and double-talking processing method to nonlinear processing.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to an echo elimination method. Background Art

[0002] After years of development, the basic technical route of the existing acoustic echo cancellation (AEC) algorithm has become fixed. For example, the adaptive filtering algorithm technology, the mainstream ones are based on the most basic minimum mean square error algorithm (Least Mean Square, LMS), normalized least mean square algorithm (Normalized Least Mean Square, NLMS), affine projection algorithm (Affine Projection Algorithm, APA) and sub-band adaptive algorithm based on filter group, etc.

[0003] Currently, the industry mainly uses the WebRtc open source framework to develop real-time communication solutions. This is because the algorithm contains related technologies such as delay estimation, linear echo cancellation, and non-linear processing (NLP), so it is widely used in the mobile Internet field.

[0004] During our research on WebRtcAEC, we found that it also faces some shortcomings, such as residual echo, noise, fluctuating volume, and double-talk mute. As people's requirements for voice quality are getting higher and higher, these problems need to be urgently optimized.

[0005] Therefore, it is necessary to provide an echo cancellation method to solve the problems of residual echo, noise, fluctuating volume, double talk and mute in existing voice communications. Summary of the invention

[0006] The object of the present invention is to provide an echo cancellation method to solve the problems of residual echo, noise, fluctuating volume, double-talk mute, etc. in existing voice communications.

[0007] In order to solve the problems existing in the prior art, the present invention provides an echo cancellation method, comprising the following steps:

[0008] Adaptively update the main filter and auxiliary filter in the dual filtering structure, and the update function is as follows:

[0009]

[0010] Where w(k+1) is the filter update function, w(k) is the original filter function, u is the step size factor, Q(k) is the step size gain matrix, x(k) is the system input signal, e(k) is the evaluation error, δ is the minimum regularization parameter, and x(k) T To transpose the system input signal;

[0011] The dual filter structure is used to accelerate the convergence speed of the echo in the sound signal, linearly process the echo and output the processed sound signal;

[0012] Perform nonlinear processing on the linearly processed sound signal.

[0013] Optionally, in the echo cancellation method, the calculation formula of Q(k) is as follows:

[0014] Q(k)=[q0(k)q1(k)…q L-1 (k)];

[0015] Where L is the block length;

[0016] α l (k) = log(1 + p(k)) / log(2), Where K is the number of blocks;

[0017]

[0018] Optionally, in the echo cancellation method, the updating step is as follows:

[0019] The main filter obtains the far-end frequency domain signal and the near-end time domain signal, and the auxiliary filter obtains the far-end frequency domain signal and the near-end time domain signal;

[0020] The main filter and the auxiliary filter estimate the echo signal based on the far-end frequency domain signal respectively;

[0021] The main filter and the auxiliary filter perform IFFT on the echo signal to obtain the echo time domain signal;

[0022] The main filter and the auxiliary filter respectively use the near-end time domain signal to subtract the echo time domain signal to obtain an error signal;

[0023] The main filter and the auxiliary filter calculate the error signal variance respectively;

[0024] calculating the difference in error signals between the main filter and the auxiliary filter;

[0025] calculating the variance of the difference between the error signals of the main filter and the auxiliary filter;

[0026] Calculate the variance of the near-end time domain signal;

[0027] Check whether the main filter diverges according to the above calculation results. If so, the echo estimation result of the auxiliary filter is assigned to the main filter. If not, the auxiliary filter is updated, and the echo estimation result of the auxiliary filter is not passed to the main filter.

[0028] Based on the sparsity of the echo path, the step gain matrix is ​​calculated;

[0029] Based on the step gain matrix, calculate the energy of the remote reference signal;

[0030] According to the above calculation results, the main filter and the auxiliary filter are adaptively updated.

[0031] Optionally, in the echo cancellation method, the dual filtering structure outputs a processed sound signal, and the processed sound signal includes a near-end mixed signal minus an estimated echo signal, wherein the near-end mixed signal includes a signal after the far-end frequency domain signal is transmitted to the near-end, is emitted by a speaker and received by a near-end microphone, and a near-end time domain signal.

[0032] Optionally, in the echo cancellation method, the nonlinear processing is performed as follows:

[0033] Acquire the processed sound signal, and extract the error signal, the near-end time domain signal, and the far-end time domain signal;

[0034] Calculate the power spectral density of the near-end time domain signal and the error signal, and calculate the power spectral density of the near-end time domain signal and the far-end time domain signal;

[0035] Calculating the correlation between the error signal and the near-end time domain signal, and calculating the correlation between the far-end time domain signal and the near-end time domain signal;

[0036] Determine whether the far-end time domain signal and the near-end time domain signal have a large uncorrelation, and the near-end time domain signal and the error signal have a large correlation. If so, it is determined that only the near-end voice exists, and the echo is suppressed by using the suppression gain factor. If not, it is determined to be an echo, and it is determined again whether the processed sound signal is double talk;

[0037] If it is double talk, double talk detection protection is performed, the gain factor suppression degree is reduced, and the echo is suppressed by the suppression gain factor; if it is not double talk, the gain factor suppression is increased for single talk, and the echo is suppressed by the suppression gain factor.

[0038] Optionally, in the echo cancellation method, the nonlinear processing further comprises the following steps:

[0039] Calculating the power of the near-end time domain signal, the power of the far-end time domain signal, and the power of the error signal;

[0040] Calculating a residual echo loss parameter based on the power of the near-end time domain signal and the power of the error signal;

[0041] Calculating near-end and far-end loss parameters based on the power of the near-end time domain signal and the power of the far-end time domain signal;

[0042] Calculating a far-end error loss parameter based on the power of the far-end time domain signal and the power of the error signal;

[0043] The residual echo loss parameter, the near-end loss parameter and the far-end error loss parameter are determined by a preset threshold, and the suppression gain factor coefficient is further adjusted to suppress the echo using the suppression gain factor.

[0044] Optionally, in the echo cancellation method, the preset thresholds include: a residual echo loss parameter threshold, a near-end loss parameter threshold, and a far-end error loss parameter threshold.

[0045] Optionally, in the echo elimination method, the preset threshold is set according to actual conditions.

[0046] Optionally, in the echo cancellation method, the gain factor is calculated by a correlation coefficient between a far-end time domain signal and a near-end time domain signal.

[0047] Optionally, in the echo cancellation method, after the nonlinear processing is performed, nonlinear shearing is further included in the following manner:

[0048] Acquire the error signal after linear processing;

[0049] Calculate the sum of squares of the error signal;

[0050] Whether to eliminate the echo signal is determined based on a preset threshold of the square sum of the error signal.

[0051] In the echo cancellation method provided by the present invention, the echo cancellation method is based on the WebRtcAEC framework. On this basis, a step gain matrix algorithm is first added to the linear part to control the echo convergence speed, and then an auxiliary filter is added to control the main filter. When encountering double talk or the main filter divergence, the auxiliary filter can update the main filter state. Finally, the voice signal correlation calculation is added to improve the suppression degree of NLP for double talk. At the same time, the echo is double-detected to better attenuate the echo part, which effectively solves the shortcomings of the existing technology, greatly improves the voice quality, and meets the real-time communication requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A circuit diagram of an echo cancellation method provided by an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of a flow chart of an echo cancellation method provided by an embodiment of the present invention;

[0054] Figure 3 A flowchart of linear processing of echoes provided by an embodiment of the present invention;

[0055] Figure 4 A flowchart of performing nonlinear processing on echo provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The specific implementation of the present invention will be described in more detail below in conjunction with the schematic diagram. The advantages and features of the present invention will become clearer based on the following description. It should be noted that the drawings are all in a very simplified form and are not in exact proportions, and are only used to facilitate and clearly assist in explaining the purpose of the embodiments of the present invention.

[0057] Hereinafter, if the method described herein includes a series of steps, the order in which the steps are presented herein is not necessarily the only order in which the steps may be performed, and some of the steps described may be omitted and / or some other steps not described herein may be added to the method.

[0058] During the research on WebRtcAEC, it was found that it also faces some shortcomings, such as residual echo, noise, fluctuating volume, and double-talk mute. As people's requirements for voice quality are getting higher and higher, these problems need to be urgently optimized.

[0059] In order to solve the problems existing in the prior art, the present invention provides an echo cancellation method, such as Figure 1 and 2 As shown, the echo cancellation method comprises the following steps:

[0060] Adaptively update the main filter and auxiliary filter in the dual filtering structure, and the update function is as follows:

[0061]

[0062] Where w(k+1) is the filter update function, w(k) is the original filter function, u is the step size factor, Q(k) is the step size gain matrix, x(k) is the system input signal, e(k) is the evaluation error, δ is the minimum regularization parameter, and x(k) T To transpose the system input signal;

[0063] The dual filter structure is used to accelerate the convergence speed of the echo in the sound signal, linearly process the echo and output the processed sound signal;

[0064] Perform nonlinear processing on the linearly processed sound signal.

[0065] Specifically, Figure 2 As shown, Figure 2The flow chart of the echo cancellation method provided by the embodiment of the present invention is as follows: first, the hardware microphone collects the near-end voice signal and the reference signal transmitted from the far-end in real time, performs voice framing processing, generally set to 10ms per frame, performs fast Fourier transform of voice data into frequency domain signal, calculates the power spectrum of the far-end and near-end, enters the linear part and nonlinear part filtering processing, and adopts a dual filter structure instead of the WebrtcAEC single filter structure when processing the linear part, which is mainly used to solve the problem of near-end signal stuttering when double talk exists. One of them is the main filter, which is updated at all times to cope with environmental changes, and the other is the auxiliary filter, which gives its state to the main filter when double talk exists or the main filter diverges to solve the stuttering phenomenon. At the same time, an optimal step gain matrix is ​​introduced to control the step factor of the NLMS algorithm, which speeds up its convergence speed. When processing the nonlinear part, since the original NLP algorithm has only two states, either the echo exists or the echo does not exist, it has a more serious suppression of near-end voice. The present invention takes into account the double talk state, reasonably controls the suppression degree of NLP for double talk, and increases the attenuated echo part at the same time, and obtains a better voice quality. Then nonlinear shearing is added to remove obvious echoes. Finally, comfort noise is added and the processed frequency domain signal is converted into a time domain signal output through inverse fast Fourier transform.

[0066] Furthermore, the calculation formula of the step gain matrix Q(k) is as follows:

[0067] Q(k)=[q0(k)q1(k)…q L-1 (k)];

[0068] Where L is the block length;

[0069] α l (k) = log(1 + p(k)) / log(2), Where K is the number of blocks;

[0070]

[0071] like Figure 3As shown, the present invention adopts a dual-filter structure, in which the main filter always tracks the dynamic acoustic path, and the auxiliary filter is enabled when the main filter diverges. This structure can well protect the convergence speed of the algorithm. The echo pulse response has a sparse characteristic, and the absolute value of its coefficient is only greater than zero in a short period of time, and the rest are infinitely close to zero. In fact, only a few hundred of the thousands of filter coefficients play a practical role and determine the accuracy of the echo path estimation. According to the sparsity of the echo path, the present invention introduces the idea of ​​proportional adaptation, and the core of the filter adopts the block frequency domain sparse control MPNLMS algorithm. The algorithm balances the updates between the filter size coefficients through the optimal step size control matrix instead of the fixed step size factor, so that the present invention can adapt to more complex scenarios (such as highly sparse and low-sparse situations).

[0072] For further information, please refer to Figure 3 , the update steps are as follows:

[0073] The main filter obtains the far-end frequency domain signal and the near-end time domain signal, and the auxiliary filter obtains the far-end frequency domain signal and the near-end time domain signal;

[0074] The main filter and the auxiliary filter estimate the echo signal based on the far-end frequency domain signal respectively;

[0075] The main filter and the auxiliary filter perform IFFT (Inverse Fast Fourier Transform, frequency domain signal to time domain signal) on the echo signal to obtain the echo time domain signal;

[0076] The main filter and the auxiliary filter respectively use the near-end time domain signal to subtract the echo time domain signal to obtain an error signal;

[0077] The main filter and the auxiliary filter calculate the error signal variance respectively;

[0078] calculating the difference in error signals between the main filter and the auxiliary filter;

[0079] calculating the variance of the difference between the error signals of the main filter and the auxiliary filter;

[0080] Calculate the variance of the near-end time domain signal;

[0081] Check whether the main filter diverges according to the above calculation results. If so, the echo estimation result of the auxiliary filter (i.e., the echo time domain signal) is assigned to the main filter. If not, the auxiliary filter is updated, and the echo estimation result of the auxiliary filter is not passed to the main filter.

[0082] Based on the sparsity of the echo path, the step gain matrix is ​​calculated;

[0083] Based on the step gain matrix, calculate the energy of the remote reference signal;

[0084] According to the above calculation results, the main filter and the auxiliary filter are adaptively updated.

[0085] Preferably, the dual filter structure can be updated in real time, and the dual filter structure is used to accelerate the convergence speed of the echo in the sound signal, perform linear processing on the echo and output the processed sound signal. The processed sound signal includes the near-end mixed signal minus the estimated echo signal, wherein the near-end mixed signal includes the signal transmitted from the far-end frequency domain signal to the near-end, emitted by the speaker and received by the near-end microphone, and the near-end time domain signal.

[0086] like Figure 4 As shown, the nonlinear processing method is as follows:

[0087] Acquire the processed sound signal, and extract the error signal, the near-end time domain signal, and the far-end time domain signal;

[0088] Calculate the power spectral density of the near-end time domain signal and the error signal, and calculate the power spectral density of the near-end time domain signal and the far-end time domain signal;

[0089] Calculating the correlation between the error signal and the near-end time domain signal, and calculating the correlation between the far-end time domain signal and the near-end time domain signal;

[0090] Determine whether the far-end time domain signal and the near-end time domain signal have a large uncorrelation, and the near-end time domain signal and the error signal have a large correlation. If so, it is determined that only the near-end voice exists, and the echo is suppressed by using the suppression gain factor. If not, it is determined to be an echo, and it is determined again whether the processed sound signal is double talk;

[0091] If it is double talk, double talk detection protection is performed, the gain factor suppression degree is reduced, and the echo is suppressed by the suppression gain factor; if it is not double talk, the gain factor suppression is increased for single talk, and the echo is suppressed by the suppression gain factor.

[0092] Preferably, the nonlinear processing further comprises the following steps:

[0093] Calculating the power of the near-end time domain signal, the power of the far-end time domain signal, and the power of the error signal;

[0094] Calculating a residual echo loss parameter based on the power of the near-end time domain signal and the power of the error signal;

[0095] Calculating near-end and far-end loss parameters based on the power of the near-end time domain signal and the power of the far-end time domain signal;

[0096] Calculating a far-end error loss parameter based on the power of the far-end time domain signal and the power of the error signal;

[0097] The residual echo loss parameter, the near-end loss parameter and the far-end error loss parameter are determined by a preset threshold, and the suppression gain factor coefficient is further adjusted to suppress the echo using the suppression gain factor.

[0098] The preset thresholds include residual echo loss parameter thresholds, near-end and far-end loss parameter thresholds, and far-end error loss parameter thresholds. The preset thresholds are set according to actual conditions. The gain factor is calculated by the correlation coefficient between the far-end time domain signal and the near-end time domain signal.

[0099] The nonlinear post-processing part is mainly designed based on the criteria of the far-end time domain signal and the near-end time domain signal, the correlation between the near-end time domain signal and the error signal, the residual echo loss parameters, etc., to filter out the nonlinear echo. The original WebRtc open source algorithm nonlinear processing does not make special treatment for double talk, so the double talk loss is very serious. In order to solve this problem, the present invention introduces double talk detection, which mainly judges the correlation between the far-end time domain signal and the near-end time domain signal, the near-end time domain signal and the error signal, and suppresses the single talk to a greater extent. So as to better protect the double talk voice.

[0100] Preferably, after the nonlinear processing, nonlinear shearing is also included, in the following manner:

[0101] Acquire the error signal after linear processing;

[0102] Calculate the sum of squares of the error signal;

[0103] Whether to eliminate the echo signal is determined based on a preset threshold of the square sum of the error signal.

[0104] In summary, in the echo cancellation method provided by the present invention, the echo cancellation method is based on the WebRtc AEC framework, and on this basis, a step gain matrix algorithm is first added to the linear part to control the echo convergence speed, and then an auxiliary filter is added to control the main filter. When encountering double talk or the main filter diverges, the auxiliary filter can update the main filter state, and finally, the voice signal correlation calculation is added to improve the suppression degree of NLP for double talk, and the echo is double-detected at the same time, so as to better attenuate the echo part, effectively solve the shortcomings of the prior art, greatly improve the voice quality, and meet the real-time communication requirements.

[0105] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.

Claims

1. An echo cancellation method, characterized in that: The following steps are involved: The update steps are as follows: The main filter obtains the far-end frequency domain signal and the near-end time domain signal, and the auxiliary filter obtains the far-end frequency domain signal and the near-end time domain signal; The main filter and the auxiliary filter estimate the echo signal based on the far-end frequency domain signal respectively; The main filter and the auxiliary filter perform IFFT on the echo signal to obtain the echo time domain signal; The main filter and the auxiliary filter respectively use the near-end time domain signal to subtract the echo time domain signal to obtain an error signal; The main filter and the auxiliary filter calculate the error signal variance respectively; calculating the difference in error signals between the main filter and the auxiliary filter; calculating the variance of the difference between the error signals of the main filter and the auxiliary filter; Calculate the variance of the near-end time domain signal; Check whether the main filter diverges according to the variance of the error signal, the difference of the error signals, the variance of the difference of the error signals and the variance of the near-end time domain signal. If so, the echo estimation result of the auxiliary filter is assigned to the main filter. If not, the auxiliary filter is updated, and the echo estimation result of the auxiliary filter is not passed to the main filter. Based on the sparsity of the echo path, the step gain matrix is ​​calculated; Based on the step gain matrix, calculate the energy of the remote reference signal; Adaptively updating the main filter and the auxiliary filter according to the error signal variance, the difference between the error signals, the variance of the difference between the error signals, the variance of the near-end time domain signal, and whether the main filter diverges; Adaptively update the main filter and auxiliary filter in the dual filtering structure, and the update function is as follows: Where w(k+1) is the filter update function, w(k) is the original filter function, u is the step size factor, Q(k) is the step size gain matrix, x(k) is the system input signal, e(k) is the evaluation error, δ is the minimum regularization parameter, and x(k) T To transpose the system input signal; The dual filter structure is used to accelerate the convergence speed of the echo in the sound signal, linearly process the echo and output the processed sound signal; Perform nonlinear processing on the linearly processed sound signal.

2. The echo cancellation method according to claim 1, characterized in that: The calculation formula of Q(k) is as follows: Q(k)=[q0(k)q1(k)…q L-1 (k)]; Where L is the block length; α l (k) = log(1 + p(k)) / log(2), Where K is the number of blocks; 3. The echo cancellation method according to claim 2, characterized in that: The dual filtering structure outputs a processed sound signal, which includes a near-end mixed signal minus an estimated echo signal, wherein the near-end mixed signal includes a signal transmitted from a far-end frequency domain signal to a near-end speaker and received by a near-end microphone, as well as a near-end time domain signal.

4. The echo cancellation method according to claim 2, characterized in that: The nonlinear processing is as follows: Acquire the processed sound signal, and extract the error signal, the near-end time domain signal, and the far-end time domain signal; Calculate the power spectral density of the near-end time domain signal and the error signal, and calculate the power spectral density of the near-end time domain signal and the far-end time domain signal; Calculating the correlation between the error signal and the near-end time domain signal, and calculating the correlation between the far-end time domain signal and the near-end time domain signal; Determine whether the far-end time domain signal and the near-end time domain signal have a large uncorrelation, and the near-end time domain signal and the error signal have a large correlation. If so, it is determined that only the near-end voice exists, and the echo is suppressed by using the suppression gain factor. If not, it is determined to be an echo, and it is determined again whether the processed sound signal is double talk; If it is double talk, double talk detection protection is performed, the gain factor suppression degree is reduced, and the echo is suppressed by the suppression gain factor; if it is not double talk, the gain factor suppression is increased for single talk, and the echo is suppressed by the suppression gain factor.

5. The echo cancellation method according to claim 4, characterized in that: The nonlinear processing also includes the following steps: Calculating the power of the near-end time domain signal, the power of the far-end time domain signal, and the power of the error signal; Calculating a residual echo loss parameter based on the power of the near-end time domain signal and the power of the error signal; Calculating near-end and far-end loss parameters based on the power of the near-end time domain signal and the power of the far-end time domain signal; Calculating a far-end error loss parameter based on the power of the far-end time domain signal and the power of the error signal; The residual echo loss parameter, the near-end loss parameter and the far-end error loss parameter are determined by a preset threshold, and the suppression gain factor coefficient is further adjusted to suppress the echo using the suppression gain factor.

6. The echo cancellation method according to claim 5, characterized in that: The preset thresholds include: a residual echo loss parameter threshold, a near-end loss parameter threshold, and a far-end error loss parameter threshold.

7. The echo cancellation method according to claim 5, characterized in that: The preset threshold is set according to the actual situation.

8. The echo cancellation method according to claim 4, characterized in that: The gain factor is calculated by the correlation coefficient between the far-end time domain signal and the near-end time domain signal.

9. The echo cancellation method according to claim 2, characterized in that: After nonlinear processing, nonlinear shearing is also included, as follows: Obtaining linear processing of the echo and outputting the processed sound signal; Calculate the sum of squares of the error signal; Whether to eliminate the echo signal is determined based on a preset threshold of the square sum of the error signal.

Images