Echo processing method and apparatus, computer readable storage medium

Abstract

An echo processing method and device, and a computer readable storage medium, the echo processing method comprising: determining a residual signal according to a main microphone input signal and an echo reference signal; estimating a linear echo leakage in the residual signal; adjusting and optimizing the linear echo leakage; removing the optimized linear echo leakage from the residual signal. With the above scheme, the linear echo leakage can be effectively eliminated without damaging the voice signal.

Description

Technical Field

[0001] This invention relates to the field of sound processing technology, and in particular to an echo processing method and apparatus, and a computer-readable storage medium. Background Technology

[0002] In voice communication systems, the far-end signal is transmitted through the network to the near-end, played by the near-end speaker, and then picked up again by the microphone and transmitted to the far-end, forming an acoustic echo. Acoustic echo cancellation technology typically uses adaptive filters for linear filtering to remove components in the echo that are linearly correlated with the far-end signal. Even after adaptive filtering, some echo may remain, which is usually addressed using nonlinear processing and residual echo suppression modules.

[0003] Ideally, the adaptive filter gets closer and closer to the real echo path as the number of iterations increases, and the filtering error tends to be minimized. However, in practical applications, due to the limitations of the filter order, near-end interference, spectral leakage, and other factors affecting convergence, the adaptive filter cannot completely eliminate linear echoes and linear echo leakage will exist. These linear leakages are usually treated together with nonlinear echo remnants in residual echo suppression, but this will cause some speech loss. Summary of the Invention

[0004] The present invention addresses the technical problem of damage to speech signals during residual echo suppression.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide an echo processing method, comprising: determining a residual signal based on a main microphone input signal and an echo reference signal; estimating linear echo leakage in the residual signal; adjusting and optimizing the linear echo leakage; and removing the optimized linear echo leakage from the residual signal.

[0006] Optionally, determining the residual signal based on the main microphone input signal and the reference signal includes: obtaining an estimated echo signal based on the main microphone input signal and the echo reference signal; and removing the estimated echo signal from the main microphone input signal to determine the residual signal.

[0007] Optionally, estimating the linear echo leakage in the residual signal includes: determining a target leakage factor; performing a calculation on the target leakage factor and the estimated echo signal, and using the calculation result as the linear echo leakage.

[0008] Optionally, determining the target leakage factor includes estimating the target leakage factor based on the coherence between the residual signal and the estimated echo signal.

[0009] Optionally, determining the target leakage factor includes: estimating the target leakage factor based on a first cross-correlation value and a first autocorrelation value; the first cross-correlation value is obtained by cross-correlation operation between the residual signal in the frequency domain and the estimated echo signal in the frequency domain; the first autocorrelation value is obtained by autocorrelation operation between the estimated echo signal in the frequency domain.

[0010] Optionally, estimating the linear echo leakage in the residual signal includes: determining a target leakage factor; performing a calculation on the target leakage factor and the echo reference signal, and using the result as the linear echo leakage.

[0011] Optionally, determining the target leakage factor includes: determining the target leakage factor based on the coherence between the residual signal and the echo reference signal.

[0012] Optionally, determining the target leakage factor includes: estimating the linear echo leakage in the residual signal based on the second cross-correlation value and the second autocorrelation value; the second cross-correlation value is obtained by cross-correlation operation between the residual signal in the frequency domain and the echo reference signal in the frequency domain; the second autocorrelation value is obtained by autocorrelation operation between the echo reference signal in the frequency domain.

[0013] Optionally, the adjustment and optimization of the linear echo leakage includes: determining a target adjustment factor based on the optimization objective; and adjusting and optimizing the linear echo leakage using the target adjustment factor.

[0014] Optionally, determining the target adjustment factor based on the optimization objective includes: traversing the candidate adjustment factor group and determining the target adjustment factor as: the candidate adjustment factor with the smallest power spectrum corresponding to the target signal; the target signal is the signal from the residual signal after removing the optimized linear echo leakage.

[0015] This invention also provides an echo processing apparatus, comprising: a determining unit for determining a residual signal based on a main microphone input signal and an echo reference signal; an estimating unit for estimating linear echo leakage in the residual signal; an adjusting unit for adjusting and optimizing the linear echo leakage; and a processing unit for removing the optimized linear echo leakage from the residual signal.

[0016] This invention also provides a computer-readable storage medium, which is a non-volatile or non-transient storage medium, storing a computer program thereon. When the computer program is run by a processor, it executes the steps of any of the above-described echo processing methods.

[0017] This invention also provides another echo processing apparatus, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of any of the echo processing methods described above when running the computer program.

[0018] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0019] After determining the residual signal, the linear echo leakage in the residual signal is estimated, and the linear echo leakage is adjusted and optimized before being removed from the residual signal. Since the linear echo leakage is calculated based on the linearly estimated echo signal, no speech loss is caused while reducing echo residue. Attached Figure Description

[0020] Figure 1 This is a flowchart of an echo processing method according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of an echo processing device according to an embodiment of the present invention. Detailed Implementation

[0022] In voice communication systems, the signal from the far end is transmitted to the near end via the network, played by the near-end speaker, and then picked up again by the microphone and transmitted to the far end, creating an acoustic echo. Acoustic echo severely affects the call experience and can cause howling, damaging user equipment. Therefore, acoustic echo cancellation is a crucial processing step in real-time audio and video communication, requiring complete echo removal while achieving good full-duplex performance.

[0023] In acoustic echo cancellation technology, adaptive filters are a crucial step. They simulate the echo propagation path by using a filter and adaptively track and adjust the filter coefficients to continuously approximate the actual echo path. Ideally, the filtering error tends to zero with increasing iterations. However, in real-world scenarios, due to limitations in filter order, near-end interference, and steady-state errors, adaptive filters cannot eliminate all linear echoes. Their cancellation performance typically achieves an echo suppression ratio of around 20dB, but some linear echo leakage still occurs. Furthermore, nonlinear echo components cannot be eliminated by adaptive filtering. Therefore, echo cancellation systems usually include residual echo suppression and nonlinear processing modules to completely suppress residual echoes.

[0024] Residual echoes are typically processed in two ways: one is a coherence-based method, which estimates a corresponding suppression factor for each frequency point and applies the suppression factor to the near-end signal to suppress residual echoes. The other method is to estimate and filter residual echoes using noise suppression techniques.

[0025] However, existing methods for suppressing residual echoes do not distinguish between linear and nonlinear echo leakage; instead, they suppress both. To minimize echoes, residual echo suppression is often intensified, leading to damage to the speech signal.

[0026] In this embodiment of the invention, after determining the residual signal, the linear echo leakage in the residual signal is estimated, and the linear echo leakage is adjusted and optimized, and the adjusted and optimized linear echo leakage is removed from the residual signal. Since the linear echo leakage is calculated based on the linearly estimated echo signal, no speech loss is caused while reducing echo residue.

[0027] To make the above-mentioned objectives, features and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0028] This invention provides an echo processing method, referring to... Figure 1 The following will provide a detailed explanation through specific steps.

[0029] Step 101: Determine the residual signal based on the main microphone input signal and the echo reference signal.

[0030] In this embodiment of the invention, an estimated echo signal can first be estimated based on the main microphone input signal and the echo reference signal. Then, the estimated echo signal is removed from the main microphone input signal to obtain the residual signal.

[0031] In practice, the main microphone input signal and the echo reference signal can be input to a preset adaptive filter (AF) to obtain the estimated echo signal.

[0032] Specifically, adaptive filters can be generated using the Normalized Least Mean Square Error (NLMS) algorithm, the Recursive Least Squares (RLS) algorithm, or the Kalman filtering algorithm.

[0033] It is understood that the specific process of obtaining the estimated echo signal using an adaptive filter can refer to existing technology, and will not be elaborated upon in the embodiments of this invention.

[0034] Step 102: Estimate the linear echo leakage in the residual signal.

[0035] In this embodiment of the invention, linear echo leakage can be estimated based on the estimated echo signal obtained from the adaptive filter. Alternatively, linear echo leakage can be estimated based on the echo reference signal. Since the estimated echo signal and echo reference signal do not include speech, the estimated linear echo leakage is independent of speech. This avoids speech loss during the subsequent removal of linear echo leakage.

[0036] In practice, when estimating linear echo leakage, the target leakage factor can be obtained first; then, based on the target leakage factor, the linear echo leakage in the residual signal can be determined.

[0037] In this embodiment of the invention, the target leakage factor can be calculated based on the residual signal and the estimated echo signal. Alternatively, the target leakage factor can also be calculated based on the residual signal and the echo reference signal.

[0038] In one embodiment of the present invention, the target leakage factor is calculated using the following formula (1):

[0039]

[0040] In equation (1) above, l is the frame index of the current frame, k is the frequency point (the value range is the start frequency point and the end frequency point), and R EY (l,k) represents the cross-correlation value between the residual signal and the estimated echo signal at the k-th frequency point after the time-frequency conversion of the current frame; R YY (l,k) represents the autocorrelation value of the estimated echo signal at the k-th frequency point after the time-frequency conversion of the current frame.

[0041] Specifically, R can be calculated using the following formula (2). EY (l,k):

[0042] R EY (l,k)=(1-β(l))R EY (l-1,k)+β(l)P Y (k)P E (k); (2)

[0043] In equation (2) above, β(l) is the smoothing coefficient corresponding to the current frame, with a value of 0 to 1; R EY (l-1,k) represents the cross-correlation value of the residual signal and the estimated echo signal at the k-th frequency point after the time-frequency conversion of the previous frame (the (l-1)-th frame); P Y (k) represents the power spectrum of the estimated echo signal at the k-th frequency point after the time-frequency conversion of the current frame; P E (k) represents the power spectrum of the residual signal at the k-th frequency point after the time-frequency conversion of the current frame.

[0044] R can be calculated using the following formula (3).YY (l,k):

[0045] R YY (l,k)=(1-β(l))R YY (l-1,k)+β(l)(P Y (k)) 2 (3)

[0046] In equation (3) above, β(l) is the smoothing coefficient corresponding to the current frame, with a value of 0 to 1; R YY (l-1,k) represents the autocorrelation value of the estimated echo signal at the k-th frequency point after time-frequency conversion of the previous frame (the (l-1)-th frame); P Y (k) represents the power spectrum of the estimated echo signal at the k-th frequency point after the time-frequency conversion of the current frame.

[0047] It should be noted that equations (2) and (3) above are based on the estimated echo signal. It is understood that an echo reference signal can also be used instead of the estimated echo signal, i.e.: R EY (l-1,k) represents the cross-correlation value of the residual signal and the echo reference signal at the k-th frequency point after the time-frequency conversion of the previous frame (the (l-1)-th frame), P Y (k) represents the power spectrum of the echo reference signal at the k-th frequency point after the time-frequency conversion of the current frame, R YY (l-1,k) is the autocorrelation value of the echo reference signal at the k-th frequency point after time-frequency conversion of the previous frame (the (l-1)-th frame).

[0048] In practical implementation, the smoothing coefficient β for each frame can be a fixed value or adjusted in real time. In this embodiment of the invention, the smoothing coefficient β(l) corresponding to the current frame can be adjusted using the following formula (4):

[0049]

[0050] In equation (4) above, β0 is a constant; if E AF If (l,k) is a time-domain signal, then E AF (l,k) represents the residual signal at the k-th temporal sampling point in the current frame, |E AF (l,k)| 2 For E AF The energy corresponding to (l,k); if E AF If (l,k) is a frequency domain signal, then E AF (l,k) represents the residual signal at the k-th frequency point in the current frame, |E AF (l,k)| 2 For E AF The power spectrum corresponding to (l,k) is N, where N is the total number of frequency points.

[0051] Accordingly, if Y(l,k) is a time-domain signal, then Y(l,k) represents the estimated echo signal or echo reference signal of the k-th time-domain sampling point in the current frame; |Y(l,k)| 2 Let Y(l,k) be the energy corresponding to Y(l,k); if Y(l,k) is a frequency domain signal, then Y(l,k) represents the estimated echo signal or echo reference signal at the k-th frequency point in the current frame; |Y(l,k)| 2 Let Y(l,k) be the power spectrum.

[0052] In specific implementation, if Y(l,k) represents the estimated echo signal of the kth time-domain sampling point in the current frame, then the first cross-correlation value is calculated by equation (2) above, and the first autocorrelation value is calculated by equation (3) above. If Y(l,k) represents the echo reference signal of the kth time-domain sampling point in the current frame, then the second cross-correlation value is calculated by equation (2) above, and the second autocorrelation value is calculated by equation (3) above.

[0053] In practice, the target leakage factor can also be obtained by calculating the coherence between the residual signal and the estimated echo signal (echo reference signal).

[0054] Specifically, see the following formula (5):

[0055]

[0056] In equation (5) above, η(l,k) is the target leakage factor corresponding to the k-th frequency point of the current frame, and S E (l,k) represents the smoothed power spectrum of the residual signal at the k-th frequency point in the current frame, S Y (l,k) is the smoothed power spectrum of the estimated echo signal (echo reference signal) at the k-th frequency point in the current frame, where:

[0057] S E (l,k) can be calculated using the following formula (6):

[0058] S E (l,k)=(1-α)·S E (l-1,k)+α·E AF (l,k)·E AF (l,k) * (6)

[0059] S Y (l,k) can be calculated using the following formula (7):

[0060] S Y (l,k)=(1-α)·S Y (l-1,k)+α·Y(l,k)·Y(l,k)* (7)

[0061] S EY (l,k) can be calculated using the following formula (8):

[0062] S EY (l,k)=(1-α)·S EY (l-1,k)+α·E AF (l,k)·Y(l,k) * (8)

[0063] In equations (6) to (8) above, α is the smoothing coefficient, with a value of 0 to 1; S E (l-1,k) represents the smoothed power spectrum of the residual signal at the k-th frequency point in the previous frame; S Y (l-1,k) represents the smoothed power spectrum of the estimated echo signal (echo reference signal) at the k-th frequency point in the previous frame; E AF (l,k) * For E AF Complex conjugate of (l,k) For S EY The complex conjugate of (l,k), Y(l,k) * It is the complex conjugate of Y(l,k).

[0064] In practice, the target leakage factor can be calculated for each frequency point. The target leakage factors of all frequency points are averaged, and the average value is used as the target leakage factor of the current frame.

[0065] In this embodiment of the invention, after obtaining the target leakage factor of the current frame, the linear echo leakage of the current frame can be estimated as: η(l)*Y(l). Y(l) is the estimated echo signal or echo reference signal corresponding to the current frame.

[0066] Step 103: Adjust and optimize the linear echo leakage.

[0067] Step 104: Remove the optimized linear echo leakage from the residual signal.

[0068] In this embodiment of the invention, since echo remnants have a high correlation with the estimated echo signal, while near-end interference has a low correlation, echo remnants and near-end interference can be roughly distinguished by estimating linear echo leakage. However, the estimated target leakage factor may deviate in complex scenarios, affecting the accuracy of the estimated linear echo leakage. To improve the robustness of linear echo leakage elimination, the estimated linear echo leakage can be optimized.

[0069] In practical implementation, a candidate adjustment factor group can be set, and multiple candidate adjustment factors can be set in the candidate adjustment factor group. For example, the candidate adjustment factor group can be set as {0, 1, μ1, μ2, ..., μ...} m Under the set optimization objective, each candidate adjustment factor is traversed to determine the target adjustment factor. When the adjustment factor is 0, it means that no linear echo leakage removal is performed on the residual signal; when the adjustment factor is 1, it means that no linear echo leakage is adjusted, but the linear echo leakage is directly removed from the residual signal.

[0070] In this embodiment of the invention, the optimization objective can be: to make Minimize, that is, minimize the power spectrum of the signal after removing linear echo leakage, where N is the total number of frequency points. The above optimization objective minimizes the residual echo in the signal after removing linear echo leakage.

[0071] After obtaining the target adjustment factor, the estimated linear echo leakage is optimized.

[0072] In practical implementation, the linear echo leakage in the current frame after optimization by the target adjustment factor is: μ*η(l)*Y(l). Removing the optimized linear echo leakage from the residual signal yields the signal: E=E AF -μ*η(l)*Y(l), where μ is the target adjustment factor.

[0073] In summary, in this embodiment of the invention, after determining the residual signal, the linear echo leakage in the residual signal is estimated, and the linear echo leakage is adjusted and optimized, removing the adjusted and optimized linear echo leakage from the residual signal. Since the linear echo leakage is calculated based on the linearly estimated echo signal, speech loss is not caused while reducing echo residue.

[0074] Reference Figure 2 An echo processing apparatus 20 according to an embodiment of the present invention is provided, comprising: a determining unit 201, an estimating unit 202, an adjusting unit 203, and a processing unit 204, wherein:

[0075] The determining unit 201 is used to determine the residual signal based on the main microphone input signal and the echo reference signal;

[0076] Estimation unit 202 is used to estimate the linear echo leakage in the residual signal;

[0077] The adjustment unit 203 is used to adjust and optimize the linear echo leakage;

[0078] Processing unit 204 is used to remove the optimized linear echo leakage from the residual signal.

[0079] In specific implementation, the specific execution process of the above-mentioned determining unit 201, estimating unit 202, adjusting unit 203 and processing unit 204 can be referred to steps 101 to 104, which will not be elaborated here.

[0080] In specific implementation, the modules / units included in the various devices and products described in the above embodiments can be software modules / units, hardware modules / units, or a combination of both.

[0081] For example, for various devices and products applied to or integrated into a chip, each module / unit can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, each module / unit can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0082] This invention also provides a computer-readable storage medium, which is a non-volatile or non-transient storage medium, storing a computer program thereon. When the computer program is run by a processor, it executes the steps of the echo processing method provided in steps 101 to 104.

[0083] This invention also provides an echo processing apparatus, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the steps of the echo processing method provided in steps 101 to 104.

[0084] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include ROM, RAM, disk, or optical disk, etc.

[0085] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. An echo processing method, characterized in that, include: The residual signal is determined based on the main microphone input signal and the echo reference signal; Estimate the linear echo leakage in the residual signal; The adjustment and optimization of the linear echo leakage includes: determining a target adjustment factor according to the optimization objective; and adjusting and optimizing the linear echo leakage using the target adjustment factor. The determination of the target adjustment factor according to the optimization objective includes: traversing a group of candidate adjustment factors to determine the target adjustment factor as: the candidate adjustment factor with the smallest power spectrum corresponding to the target signal; the target signal is the signal from which the optimized linear echo leakage has been removed from the residual signal. Remove the optimized linear echo leakage from the residual signal.

2. The echo processing method as described in claim 1, characterized in that, The step of determining the residual signal based on the main microphone input signal and the reference signal includes: The estimated echo signal is obtained based on the main microphone input signal and the echo reference signal; The estimated echo signal is removed from the main microphone input signal to determine the residual signal.

3. The echo processing method as described in claim 2, characterized in that, The estimation of linear echo leakage in the residual signal includes: Determine the target leakage factor; The target leakage factor is calculated with the estimated echo signal, and the result is taken as the linear echo leakage.

4. The echo processing method as described in claim 3, characterized in that, The determination of the target leakage factor includes: The target leakage factor is estimated based on the coherence between the residual signal and the estimated echo signal.

5. The echo processing method as described in claim 3, characterized in that, The determination of the target leakage factor includes: The target leakage factor is estimated based on the first cross-correlation value and the first autocorrelation value; the first cross-correlation value is obtained by cross-correlation operation between the residual signal in the frequency domain and the estimated echo signal in the frequency domain; the first autocorrelation value is obtained by autocorrelation operation between the estimated echo signal in the frequency domain.

6. The echo processing method as described in claim 1, characterized in that, The estimation of linear echo leakage in the residual signal includes: Determine the target leakage factor; The target leakage factor is calculated with the echo reference signal, and the result is taken as the linear echo leakage.

7. The echo processing method as described in claim 6, characterized in that, The determination of the target leakage factor includes: The target leakage factor is determined based on the coherence between the residual signal and the echo reference signal.

8. The echo processing method as described in claim 6, characterized in that, The determination of the target leakage factor includes: Based on the second cross-correlation value and the second autocorrelation value, the linear echo leakage in the residual signal is estimated; the second cross-correlation value is obtained by cross-correlation operation between the residual signal in the frequency domain and the echo reference signal in the frequency domain; the second autocorrelation value is obtained by autocorrelation operation between the echo reference signal in the frequency domain.

9. An echo processing device, characterized in that, include: The determination unit is used to determine the residual signal based on the main microphone input signal and the echo reference signal; An estimation unit is used to estimate the linear echo leakage in the residual signal; An adjustment unit is used to adjust and optimize the linear echo leakage, including: determining a target adjustment factor according to an optimization objective; and adjusting and optimizing the linear echo leakage using the target adjustment factor. The step of determining the target adjustment factor according to the optimization objective includes: traversing a group of candidate adjustment factors to determine the target adjustment factor as: the candidate adjustment factor with the smallest power spectrum corresponding to the target signal; the target signal is the signal from which the optimized linear echo leakage has been removed from the residual signal. A processing unit is used to remove the optimized linear echo leakage from the residual signal.

10. A computer-readable storage medium, said computer-readable storage medium being a non-volatile storage medium or a non-transient storage medium, having stored thereon a computer program, characterized in that, The computer program is executed by the processor to perform the steps of the echo processing method according to any one of claims 1 to 8.

11. An echo processing apparatus, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the echo processing method according to any one of claims 1 to 8.

Images