Echo processing method and apparatus, computer readable storage medium

Abstract

An echo processing method and device, and a computer readable storage medium, the echo processing method comprising: determining a residual signal according to a main microphone input signal and an echo reference signal; estimating linear echo leakage in the residual signal; and removing the linear echo leakage from the residual signal. With the above scheme, linear echo leakage can be effectively eliminated without damaging the speech signal.

Description

Technical Field

[0001] This invention relates to the field of sound processing technology, and in particular to an echo processing method and apparatus, and a computer-readable storage medium. Background Technology

[0002] In voice communication systems, the far-end signal is transmitted through the network to the near-end, played by the near-end speaker, and then picked up again by the microphone and transmitted to the far-end, forming an acoustic echo. Acoustic echo cancellation technology typically uses adaptive filters for linear filtering to remove components in the echo that are linearly correlated with the far-end signal. Even after adaptive filtering, some echo may remain, which is usually addressed using nonlinear processing and residual echo suppression modules.

[0003] Ideally, the adaptive filter gets closer and closer to the real echo path as the number of iterations increases, and the filtering error tends to be minimized. However, in practical applications, due to the limitations of the filter order, near-end interference, spectral leakage, and other factors affecting convergence, the adaptive filter cannot completely eliminate linear echoes and linear echo leakage will exist. These linear leakages are usually treated together with nonlinear echo remnants in residual echo suppression, but this will cause some speech loss. Summary of the Invention

[0004] The present invention addresses the technical problem of damage to speech signals during residual echo suppression.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide an echo processing method, comprising: determining a residual signal based on a main microphone input signal and an echo reference signal; estimating linear echo leakage in the residual signal; and removing the linear echo leakage from the residual signal.

[0006] Optionally, determining the residual signal based on the main microphone input signal and the echo reference signal includes: obtaining an estimated echo signal based on the main microphone input signal and the echo reference signal; removing the estimated echo signal from the main microphone input signal to determine the residual signal.

[0007] Optionally, estimating the linear echo leakage in the residual signal includes: obtaining an echo rejection ratio based on the main microphone input signal and the residual signal; determining a search interval for the leakage factor based on the relationship between the echo rejection ratio and a preset threshold; determining a target leakage factor from the search interval based on an optimization objective; and estimating the linear echo leakage in the residual signal using the target leakage factor.

[0008] Optionally, determining the linear echo leakage in the residual signal using the leakage factor includes: multiplying the target leakage factor by the estimated echo signal as the linear echo leakage; the estimated echo signal is obtained from the main microphone input signal and the echo reference signal.

[0009] Optionally, determining the linear echo leakage in the residual signal using the leakage factor includes: multiplying the target leakage factor by the echo reference signal as the linear echo leakage.

[0010] Optionally, determining the target leakage factor from the search interval according to the optimization objective includes: traversing the leakage factors in the search interval and determining the target leakage factor as: the leakage factor with the minimum residual echo energy in the time domain of the signal after removing the linear echo leakage.

[0011] Optionally, determining the target leakage factor from the search interval according to the optimization objective includes: traversing the leakage factors in the search interval and determining the target leakage factor as: the leakage factor with the minimum residual echo power spectrum in the frequency domain of the signal after removing the linear echo leakage.

[0012] This invention also provides an echo processing apparatus, comprising: a determining unit for determining a residual signal based on a main microphone input signal and an echo reference signal; an estimating unit for estimating linear echo leakage in the residual signal; and a processing unit for removing the linear echo leakage from the residual signal.

[0013] This invention also provides a computer-readable storage medium, which is a non-volatile or non-transient storage medium, storing a computer program thereon. When the computer program is run by a processor, it executes the steps of any of the above-described echo processing methods.

[0014] This invention also provides another echo processing apparatus, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of any of the echo processing methods described above when running the computer program.

[0015] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0016] After determining the residual signal, the linear echo leakage in the residual signal is estimated and then removed from the residual signal. Since the linear echo leakage is calculated based on the linearly estimated echo signal, no speech loss is caused while reducing echo residue. Attached Figure Description

[0017] Figure 1 This is a flowchart of an echo processing method according to an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of the structure of an echo processing device according to an embodiment of the present invention. Detailed Implementation

[0019] In voice communication systems, the signal from the far end is transmitted to the near end via the network, played by the near-end speaker, and then picked up again by the microphone and transmitted to the far end, creating an acoustic echo. Acoustic echo severely affects the call experience and can cause howling, damaging user equipment. Therefore, acoustic echo cancellation is a crucial processing step in real-time audio and video communication, requiring complete echo removal while achieving good full-duplex performance.

[0020] In acoustic echo cancellation technology, adaptive filters are a crucial step. They simulate the echo propagation path by using a filter and adaptively track and adjust the filter coefficients to continuously approximate the actual echo path. Ideally, the filtering error tends to zero with increasing iterations. However, in real-world scenarios, due to limitations in filter order, near-end interference, and steady-state errors, adaptive filters cannot eliminate all linear echoes. Their cancellation performance typically achieves an echo suppression ratio of around 20dB, but some linear echo leakage still occurs. Furthermore, nonlinear echo components cannot be eliminated by adaptive filtering. Therefore, echo cancellation systems usually include residual echo suppression and nonlinear processing modules to completely suppress residual echoes.

[0021] Residual echoes are typically processed in two ways: one is a coherence-based method, which estimates a corresponding suppression factor for each frequency point and applies the suppression factor to the near-end signal to suppress residual echoes. The other method is to estimate and filter residual echoes using noise suppression techniques.

[0022] However, existing methods for suppressing residual echoes do not distinguish between linear and nonlinear echo leakage; instead, they suppress both. To minimize echoes, residual echo suppression is often intensified, leading to damage to the speech signal.

[0023] In this embodiment of the invention, after determining the residual signal, the linear echo leakage in the residual signal is estimated, and the linear echo leakage is removed from the residual signal. Since the linear echo leakage is calculated based on the linearly estimated echo signal, no speech loss is caused while reducing echo residue.

[0024] To make the above-mentioned objectives, features and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] This invention provides an echo processing method, referring to... Figure 1 The following will provide a detailed explanation through specific steps.

[0026] Step 101: Determine the residual signal based on the main microphone input signal and the echo reference signal.

[0027] In this embodiment of the invention, an estimated echo signal can first be estimated based on the main microphone input signal and the echo reference signal. Then, the estimated echo signal is removed from the main microphone input signal to obtain the residual signal.

[0028] In practice, the main microphone input signal and the echo reference signal can be input to a preset adaptive filter (AF) to obtain the estimated echo signal.

[0029] Specifically, adaptive filters can be generated using the Normalized Least Mean Square Error (NLMS) algorithm, the Recursive Least Squares (RLS) algorithm, or the Kalman filtering algorithm.

[0030] It is understood that the specific process of obtaining the estimated echo signal using an adaptive filter can refer to existing technology, and will not be elaborated upon in the embodiments of this invention.

[0031] Step 102: Estimate the linear echo leakage in the residual signal.

[0032] In this embodiment of the invention, the target leakage factor can be obtained first. Based on the target leakage factor, the linear echo leakage in the residual signal is determined.

[0033] Step 103: Remove linear echo leakage from the residual signal.

[0034] In this embodiment of the invention, after obtaining the linear echo leakage, the linear echo leakage can be removed from the residual signal, thereby reducing echo residue without damaging the speech signal.

[0035] In this embodiment of the invention, the signal after removing linear echo leakage can be characterized in the time domain by the following equation (1):

[0036] e = e AF -u′·y; (1)

[0037] In equation (1) above, e is the signal after removing linear echo leakage in the time domain. AF y is the residual signal in the time domain, u′ is the target leakage factor, and y is the echo reference signal or estimated echo signal in the time domain.

[0038] In this embodiment of the invention, the signal after removing linear echo leakage can be characterized in the frequency domain by the following equation (2):

[0039] E = E AF -u′·Y; (2)

[0040] In equation (2) above, E is the signal after removing linear echo leakage in the frequency domain. AF Y is the residual signal in the frequency domain, and Y is the echo reference signal or estimated echo signal in the frequency domain.

[0041] In practice, the target leakage factor is related to the echo rejection ratio (ERR) of the current frame. The EERR of the current frame is the ratio of the sum of the energy of the main microphone input signal to the sum of the energy of the residual signal. The larger the EERR of the current frame, the smaller the target leakage factor can be; conversely, the smaller the EERR of the current frame, the larger the target leakage factor can be.

[0042] In this embodiment of the invention, when calculating the echo rejection ratio of the current frame, the energy sum of the main microphone input signal and the energy sum of the residual signal can be smoothed. By smoothing the energy sum of the main microphone input signal and the energy sum of the residual signal, the impact of voice signal energy fluctuations can be reduced.

[0043] In practice, the sum of the energy of the main microphone input signal and the sum of the energy of the residual signal can be calculated separately in the time domain; alternatively, they can be calculated separately in the frequency domain. It is understood that when calculating the echo rejection ratio, if the sum of the energy of the main microphone input signal is calculated in the time domain, then the sum of the energy of the residual signal calculated in the time domain is also used; similarly, if the sum of the energy of the main microphone input signal is calculated in the frequency domain, then the sum of the energy of the residual signal calculated in the frequency domain is also used.

[0044] In practical implementation, considering that the linear echo component is more abundant in the low-frequency region than in the high-frequency region, the echo rejection ratio can be calculated only in the low-frequency linear region. Furthermore, calculating only the echo rejection ratio in the low-frequency linear region reduces the computational load and increases the calculation speed.

[0045] Referring to the following formula (3), a calculation of the echo suppression ratio in one embodiment of the present invention is given:

[0046]

[0047] In equation (3) above, ERLE is the echo suppression ratio, α is the smoothing coefficient, and the value of α is 0 to 1; d(n,l-1) is the main microphone input signal at the nth time-domain sampling point in the previous frame (the (l-1)th frame); d(n,l) is the main microphone input signal at the nth time-domain sampling point in the current frame (the lth frame). AF (n, l-1) represents the residual signal of the nth time-domain sampling point in the previous frame (the (l-1)th frame); e AF (n,l) represents the residual signal of the nth time-domain sampling point in the current frame (the lth frame).

[0048] d 2 (n, l-1) represents the energy of the main microphone input signal at the nth time-domain sampling point in the previous frame (the (l-1)th frame), d 2 (n,l) represents the energy of the main microphone input signal at the nth time-domain sampling point in the current frame (frame l). The sum of the main microphone input signals for the previous frame can be obtained by calculating the energy of all time-domain sampling points in the previous frame. Similarly, the sum of the main microphone input signals for the current frame can be obtained by calculating the energy of all time-domain sampling points in the current frame. The sum of the main microphone input signals for the current frame is then smoothed using a smoothing coefficient α. The smoothed result is: α·∑d 2 (n,l-1)+1-α)·∑d 2 (n,l).

[0049] In the above formula (3), This represents the energy of the residual signal at the nth time-domain sampling point in the previous frame (frame l-1). Let be the energy of the residual signal at the nth time-domain sampling point in the current frame (frame l). The energy of the residual signal in the current frame is smoothed using a smoothing coefficient α, and the smoothed result is:

[0050] It should be noted that Equation (3) above is only one specific implementation for calculating the echo rejection ratio. In Equation (3) above, the echo rejection ratio is calculated in the time domain.

[0051] In practical implementation, there may be other ways to calculate the echo rejection ratio. For example, referring to the following equation (4), a method for calculating the echo rejection ratio in the frequency domain is given:

[0052]

[0053] In equation (4) above, ERLE is the echo rejection ratio, α is the smoothing coefficient, and the value of α is 0 to 1; D(k,l-1) is the main microphone input signal after time-frequency conversion at the k-th frequency point in the previous frame (the (l-1)-th frame); D(k,l) is the main microphone input signal after time-frequency conversion at the k-th frequency point in the current frame (the l-th frame). AF (k, l-1) represents the residual signal at the k-th frequency point in the previous frame (the (l-1)-th frame) after time-frequency conversion; E AF (k,l) is the residual signal at the k-th frequency point in the current frame (the l-th frame) after time-frequency conversion.

[0054] Accordingly, D 2 (n,l-1) represents the energy corresponding to D(n,l-1), where D 2 (n,l) represents the energy corresponding to D(n,l). For E AF The energy corresponding to (n, l-1) For E AF The energy corresponding to (n,l).

[0055] It is understandable that there are other ways to calculate the echo suppression ratio, not limited to the examples above.

[0056] In this embodiment of the invention, after calculating the echo rejection ratio, the echo rejection ratio can be compared with a preset threshold Thd. Based on the relationship between the echo rejection ratio and the threshold Thd, the search range for the leakage factor is determined.

[0057] When the echo rejection ratio (ERR) is higher than the threshold Thd, the echo leakage is considered small, thus the coefficient of the leakage factor search interval can be reduced. Conversely, when the EERR is lower than the threshold Thd, the echo leakage is considered large, thus the coefficient of the leakage factor search interval can be increased. In other words, a larger EERR indicates smaller echo leakage, which in turn reduces the coefficient of the leakage factor search interval; a larger EERR indicates higher echo leakage, which in turn increases the coefficient of the leakage factor search interval.

[0058] In practical implementation, when the echo suppression ratio is higher than the threshold Thd, the corresponding search interval is: 0.5×{u1,u2,…,u m When the echo rejection ratio is lower than the threshold Thd, the corresponding search interval is: {u1, u2, ..., u m}

[0059] The example above only provides one threshold, Thd. In other embodiments of the invention, thresholds with different values ​​may also be provided. The corresponding search interval is determined based on the relationship between the echo suppression ratio and the different thresholds.

[0060] For example, set thresholds Thd1 and Thd2, where Thd1 > Thd2. When the echo suppression ratio is greater than Thd1, the corresponding search interval is 0.5 × {u1, u2, ..., u m When the echo suppression ratio is less than Thd1 and greater than Thd2, the corresponding search interval is 0.75×{u1,u2,…,u}. m When the echo suppression ratio is less than Thd2, the corresponding search interval is {u1, u2, ..., u}. m}

[0061] As can be seen, in this embodiment of the invention, as the echo suppression ratio increases, the coefficient of the corresponding search space can be reduced accordingly, and the final value of the leakage factor should also be smaller.

[0062] In this embodiment of the invention, since the echo rejection ratio fluctuates greatly under different signal-to-noise ratio environments, by setting the search range of the leakage factor and the coefficient of the search range, the optimal target leakage factor can be found within a large dynamic range to adapt to the fluctuation of the threshold under different environments.

[0063] In this embodiment of the invention, the target leakage factor can be determined from the search interval obtained above, based on the optimization objective.

[0064] In practical implementation, the optimization objective can be to minimize the residual echo energy of the signal with linear echo leakage in the time domain. Based on this optimization objective, the search interval can be traversed, the residual echo energy corresponding to each leakage factor in the search interval can be calculated, and the leakage factor with the minimum echo energy sum can be taken as the target leakage factor.

[0065] In one embodiment of the present invention, the optimization objective is to minimize the residual echo energy and sume(l) in the following equation (5):

[0066]

[0067] In equation (5) above, y(n,l) is the estimated echo signal / echo reference signal of the nth time-domain sampling point in the current frame (the lth frame), the current frame (the lth frame) includes M time-domain sampling points, and u is the leakage factor.

[0068] In practical implementation, the optimization objective can also be: to minimize the sum of the residual echo power spectrum of the signal with linear echo leakage in the frequency domain. Based on this optimization objective, the search interval can be traversed, the sum of the residual echo power spectrum corresponding to each leakage factor in the search interval can be calculated, and the leakage factor that minimizes the sum of the echo power spectrum can be taken as the target leakage factor.

[0069] In one embodiment of the present invention, the optimization objective is to minimize the residual echo power spectrum and sume|E(l)| in the following equation (6):

[0070]

[0071] In equation (6) above, k is the starting frequency, bin is the frequency domain length, and Y(k,l) is the frequency domain signal corresponding to y(n,l).

[0072] It is understandable that the residual echo power spectrum of all frequency points (i.e., the entire frequency band) can also be calculated in equation (6) above. In specific implementation, after completing step 103, the residual signal after removing linear echo leakage can be subjected to nonlinear processing to further eliminate nonlinear echoes in the residual signal.

[0073] In summary, in this embodiment of the invention, by calculating and removing linear echo leakage, it is possible to reduce echo residue without damaging the speech signal.

[0074] Reference Figure 2 The present invention also provides an echo processing device 20, comprising: a determining unit 201, an estimating unit 202, and a processing unit 203, wherein:

[0075] The determining unit 201 is used to determine the residual signal based on the main microphone input signal and the echo reference signal;

[0076] Estimation unit 202 is used to estimate the linear echo leakage in the residual signal;

[0077] Processing unit 203 is used to remove the linear echo leakage from the residual signal.

[0078] In specific implementation, the specific execution process of the above-mentioned determining unit 201, estimating unit 202 and processing unit 203 can be referred to steps 101 to 103, which will not be elaborated here.

[0079] In specific implementation, the modules / units included in the various devices and products described in the above embodiments can be software modules / units, hardware modules / units, or a combination of both.

[0080] For example, for various devices and products applied to or integrated into a chip, each module / unit can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, each module / unit can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0081] This invention also provides a computer-readable storage medium, which is a non-volatile or non-transient storage medium, storing a computer program thereon. When the computer program is run by a processor, it executes the steps of the echo processing method provided in steps 101 to 103.

[0082] This invention also provides an echo processing apparatus, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the steps of the echo processing method provided in steps 101 to 103.

[0083] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include ROM, RAM, disk, or optical disk, etc.

[0084] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. An echo processing method, characterized in that, include: The residual signal is determined based on the main microphone input signal and the echo reference signal; Estimating linear echo leakage in the residual signal includes: obtaining an echo rejection ratio based on the main microphone input signal and the residual signal; determining a search interval for a leakage factor based on the relationship between the echo rejection ratio and a preset threshold; determining a target leakage factor from the search interval based on an optimization objective; and estimating linear echo leakage in the residual signal using the target leakage factor. Remove the linear echo leakage from the residual signal.

2. The echo processing method as described in claim 1, characterized in that, The step of determining the residual signal based on the main microphone input signal and the echo reference signal includes: The estimated echo signal is obtained based on the main microphone input signal and the echo reference signal; The estimated echo signal is removed from the main microphone input signal to determine the residual signal.

3. The echo processing method as described in claim 1, characterized in that, The step of using the leakage factor to determine the linear echo leakage in the residual signal includes: The product of the target leakage factor and the estimated echo signal is taken as the linear echo leakage; the estimated echo signal is obtained from the main microphone input signal and the echo reference signal.

4. The echo processing method as described in claim 1, characterized in that, The step of using the leakage factor to determine the linear echo leakage in the residual signal includes: The product of the target leakage factor and the echo reference signal is taken as the linear echo leakage.

5. The echo processing method as described in claim 1, characterized in that, The step of determining the target leakage factor from the search interval according to the optimization objective includes: By traversing the leakage factors in the search interval, the target leakage factor is determined as: the residual echo energy in the time domain of the signal after removing the linear echo leakage and the minimum leakage factor.

6. The echo processing method as described in claim 1, characterized in that, The step of determining the target leakage factor from the search interval according to the optimization objective includes: By traversing the leakage factors in the search interval, the target leakage factor is determined as: the residual echo power spectrum of the signal after removing the linear echo leakage in the frequency domain and the leakage factor with the minimum value.

7. An echo processing device, characterized in that, include: The determination unit is used to determine the residual signal based on the main microphone input signal and the echo reference signal; An estimation unit is used to estimate linear echo leakage in the residual signal, including: obtaining an echo rejection ratio based on the main microphone input signal and the residual signal; determining a search interval for a leakage factor based on the relationship between the echo rejection ratio and a preset threshold; determining a target leakage factor from the search interval based on an optimization objective; and estimating linear echo leakage in the residual signal using the target leakage factor. A processing unit is used to remove the linear echo leakage from the residual signal.

8. A computer-readable storage medium, wherein the computer-readable storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer program is stored thereon, characterized in that, When the computer program is run by the processor, it performs the steps of the echo processing method according to any one of claims 1 to 6.

9. An echo processing apparatus, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the echo processing method according to any one of claims 1 to 6.

Images