An echo cancellation method in delay jitter scenario

Abstract

The present invention provides an echo cancellation method in a delay jitter scenario in the field of echo processing technology, comprising: step S10, collecting a far-end signal and a near-end signal; step S20, calibrating and aligning the far-end signal and the near-end signal through a delay calibration module and then inputting them into a linear filtering module; step S30, the linear filter of the linear filtering module outputs a linear filter coefficient and an error signal to a far-end signal synthesis unit based on the far-end signal and the near-end signal; step S40, the far-end signal synthesis unit calculates a far-end reference signal and a delay state based on the linear filter coefficient, and outputs the far-end reference signal, error signal and delay state to a nonlinear suppression module; step S50, the nonlinear suppression module outputs an echo-free signal based on the far-end reference signal, error signal and delay state. The advantages of the present invention are: it realizes compensation for the delay jitter of the echo, greatly improves the echo cancellation effect, and thus greatly improves the call quality.

Description

Technical Field

[0001] The present invention relates to the technical field of echo processing, and in particular to an echo cancellation method in a delay jitter scenario. Background Art

[0002] Voice quality is a key factor in determining the quality of a video conference. Voice is transmitted from a local conference endpoint to a remote conference endpoint, played through the speaker, and then picked up by a microphone and sent back to the local endpoint for playback. If the round-trip delay exceeds 20 milliseconds, a noticeable echo will be heard locally, seriously disrupting the conference. Therefore, to maintain voice quality, echo cancellation is necessary.

[0003] To eliminate echoes, there is traditionally a WebRTC algorithm, which includes a delay calibration module, a linear filtering module, and a nonlinear suppression module. The delay calibration module aligns the delays of the far-end signal and the near-end signal, then uses the linear filtering module to obtain the error signal and the far-end reference signal. The output signal is then obtained through the nonlinear suppression module, so that the echo signal generated by the far-end signal can be removed from the near-end signal, thereby eliminating the echo.

[0004] However, when the conference terminal is connected to an external device such as a TV, speaker, or microphone, the delay difference between the far-end reference signal and the near-end signal will jitter and jump. At this time, the linear filter module will diverge and degrade, and cannot provide the correct far-end reference signal, resulting in the output effect of the nonlinear suppression module being degraded, and then echo leakage.

[0005] To address delay jitter, the traditional approach is to add a delay calibration module, such as the delay estimator disclosed in Chinese invention patent CN102318328B. This module searches for the corresponding delay difference based on the binary spectrum of the far-end and near-end signals and uses the delay difference for calibration and alignment. However, when conference call scenarios are subject to interference factors such as reverberation, noise, and speaker distortion, the delay calibration module often fails to provide timely and accurate feedback on delay jitter and transitions. During this period, the linear filtering module undergoes a process of divergence and reconvergence, failing to provide an accurate far-end reference signal and suppress linear echo components. Consequently, after processing by the nonlinear suppression module, a significant amount of echo signal remains, which is noticeable to the human ear and affects the call experience. This phenomenon persists until the delay calibration module provides correct feedback.

[0006] Therefore, how to provide an echo cancellation method in a delay jitter scenario to compensate for the delay jitter of the echo, improve the echo cancellation effect, and thus improve the call quality has become a technical problem that needs to be solved urgently. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide an echo cancellation method in a delay jitter scenario, so as to compensate for the delay jitter of the echo, improve the echo cancellation effect, and thus improve the call quality.

[0008] The present invention is implemented as follows: a method for echo cancellation in a delay jitter scenario, comprising the following steps:

[0009] Step S10: collecting the far-end signal of the remote conference terminal and the near-end signal of the local conference terminal in real time;

[0010] Step S20: calibrating and aligning the far-end signal and the near-end signal through a delay calibration module and then inputting the signals into a linear filtering module;

[0011] Step S30: The linear filter of the linear filtering module outputs a linear filter coefficient and an error signal to the far-end signal synthesis unit based on the far-end signal and the near-end signal;

[0012] Step S40: The far-end signal synthesis unit of the linear filtering module calculates a far-end reference signal and a delay state based on the linear filter coefficient, and outputs the far-end reference signal, error signal and delay state to the nonlinear suppression module;

[0013] Step S50: The nonlinear suppression module outputs an anechoic signal based on the input far-end reference signal, error signal, and delay state.

[0014] Furthermore, the step S20 is specifically as follows:

[0015] The far-end binarized spectrum of the far-end signal is calculated by the delay calibration module, the near-end binarized spectrum of the near-end signal is calculated, the delay difference between the far-end binarized spectrum and the near-end binarized spectrum is calculated, and the far-end signal and the near-end signal are calibrated and aligned based on the delay difference and then input into the linear filtering module.

[0016] Furthermore, the step S30 is specifically as follows:

[0017] The linear filter of the linear filtering module performs linear filtering on the far-end signal to obtain an estimated echo signal, subtracts the estimated echo signal from the near-end signal to obtain an error signal, uses the error signal as a feedback signal to update the linear filter, and outputs the linear filter coefficients of the linear filter and the error signal to the far-end signal synthesis unit.

[0018] Furthermore, in step S30, the linear filter is an adaptive filter based on short-time Fourier transform.

[0019] Furthermore, the step S40 specifically includes:

[0020] Step S41: The far-end signal synthesis unit of the linear filter module divides the sampling points of the linear filter into blocks based on the segmented block frequency domain method, calculates the quadratic norm of the linear filter coefficient in each block, sets the index value of each quadratic norm based on the block, and sets an interval threshold;

[0021] Step S42: Arrange the quadratic norms in descending order, select the top two quadratic norms and record them as E1 and E2, record the index values ​​corresponding to E1 and E2 as m1 and m2, and determine whether the absolute value of the time difference between m1 and m2 is greater than or equal to the interval threshold. If so, proceed to step S43; if not, it indicates that there is no delay jitter, and proceed to step S45;

[0022] Step S43: determine whether the time-varying trends of E1 and E2 are the same. If so, it indicates that there is delay jitter, and the process proceeds to step S44; if not, it indicates that there is no delay jitter, and the process proceeds to step S45;

[0023] Step S44: Perform weighted averaging on the remote buffer frames corresponding to m1 and m2 to obtain a remote reference signal, set the delay state to true, and proceed to step S46;

[0024] Step S45: Use the remote signal corresponding to the largest index value as the remote reference signal, set the delay state to false, and proceed to step S46;

[0025] Step S46: output the far-end reference signal, error signal, and delay state to a nonlinear suppression module.

[0026] Furthermore, in step S41, the interval threshold is a positive integer greater than or equal to 2.

[0027] Furthermore, the step S50 specifically includes:

[0028] Step S51: The nonlinear suppression module sets a default suppression strength and an enhancement coefficient, receives the remote reference signal, the error signal, and the delay status, and determines whether the delay status is true or false. If true, the process proceeds to step S52; if false, the process proceeds to step S53.

[0029] Step S52: The nonlinear suppression module multiplies the default suppression strength by the enhancement coefficient to obtain an enhanced suppression strength, suppresses the echo of the far-end signal by using the enhanced suppression strength, the far-end reference signal, and the error signal, and outputs an echo-free signal.

[0030] Step S53: The nonlinear suppression module suppresses the echo of the far-end signal by using the default suppression strength, the far-end reference signal and the error signal, and outputs an echo-free signal.

[0031] The advantages of the present invention are:

[0032] A linear filtering module including a linear filter and a far-end signal synthesis unit is provided. The linear filter generates an error signal, and the far-end signal synthesis unit generates a far-end reference signal and a delay status. The error signal, far-end reference signal, and delay status are then sent to a nonlinear suppression module. The nonlinear suppression module determines whether jitter occurs in the delay of the far-end signal based on the delay status. If jitter occurs, the far-end signal echo is enhanced and suppressed based on a default suppression strength and enhancement coefficient to eliminate residual echo. This effectively accelerates echo convergence, shortens the duration of missed echoes, and improves the call experience when jitter or jumps occur in the delay and delay calibration feedback has not been timely fed back. If jitter does not occur, the far-end signal echo is suppressed based on the default suppression strength to ensure the transparency of the near-end signal. Ultimately, the delay jitter of the echo is compensated, greatly improving the echo cancellation effect and thus greatly improving the call quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0034] Figure 1 This is a flow chart of an echo cancellation method in a delay jitter scenario of the present invention.

[0035] Figure 2 This is a schematic diagram of the architecture of an echo cancellation method in a delay jitter scenario of the present invention.

[0036] Figure 3 It is a flow chart of the linear filtering module of the present invention.

[0037] Figure 4 It is a flow chart of the remote signal synthesis unit of the present invention.

[0038] Figure 5 It is a flow chart of the nonlinear suppression module of the present invention. DETAILED DESCRIPTION

[0039] The technical solution in the embodiments of the present application has the following overall concept: a far-end signal synthesis unit is added to the linear filtering module to generate a far-end reference signal and a delay state, and the error signal, the far-end reference signal, and the delay state are sent to the nonlinear suppression module. The nonlinear suppression module determines whether the delay of the far-end signal generates jitter based on the delay state to determine the nonlinear suppression strength of the echo. When jitter occurs, the suppression is enhanced to eliminate the residual echo. When jitter does not occur, the suppression is performed based on the default suppression strength to ensure the transparency of the near-end signal. Ultimately, the delay jitter of the echo is compensated to improve call quality.

[0040] Please refer to Figures 1 to 5As shown, a preferred embodiment of the echo cancellation method in a delay jitter scenario of the present invention includes the following steps:

[0041] Step S10: collecting the far-end signal of the remote conference terminal and the near-end signal of the local conference terminal in real time;

[0042] Step S20: calibrating and aligning the far-end signal and the near-end signal through a delay calibration module and then inputting the signals into a linear filtering module;

[0043] Step S30: The linear filter of the linear filtering module outputs a linear filter coefficient and an error signal to the far-end signal synthesis unit based on the far-end signal and the near-end signal;

[0044] Step S40: The far-end signal synthesis unit of the linear filtering module calculates a far-end reference signal and a delay state based on the linear filter coefficient, and outputs the far-end reference signal, error signal and delay state to the nonlinear suppression module;

[0045] Step S50: The nonlinear suppression module outputs an anechoic signal based on the input far-end reference signal, error signal, and delay state.

[0046] The step S20 is specifically as follows:

[0047] The delay calibration module calculates a far-end binarized spectrum of the far-end signal, calculates a near-end binarized spectrum of the near-end signal, calculates a delay difference between the far-end binarized spectrum and the near-end binarized spectrum, calibrates and aligns the far-end and near-end signals based on the delay difference, and then inputs the signals into the linear filtering module. After the far-end and near-end signals are calibrated and aligned, the delay difference between the far-end and near-end signals falls within a range that can be processed by the linear filtering module.

[0048] The step S30 is specifically as follows:

[0049] The linear filter of the linear filtering module performs linear filtering on the far-end signal to obtain an estimated echo signal, subtracts the estimated echo signal from the near-end signal to obtain an error signal, uses the error signal as a feedback signal to update the linear filter, and outputs the linear filter coefficients of the linear filter and the error signal to the far-end signal synthesis unit.

[0050] In step S30, the linear filter is an adaptive filter based on short-time Fourier transform (DTFT).

[0051] The step S40 specifically includes:

[0052] Step S41: The far-end signal synthesis unit of the linear filter module divides the sampling points of the linear filter into blocks based on the segmented block frequency domain method (FLMS), calculates the quadratic norm of the linear filter coefficients in each block, sets the index value of each quadratic norm based on the block, and sets an interval threshold;

[0053] Assuming that the total sampling length of the linear filter is L = M·N, which is divided into M blocks containing N samples, the time domain impulse response of the linear filter is w(n), n = 0, 1, ..., L-1, which can be represented in the frequency domain as a collection of segmented filters:

[0054]

[0055] Among them, W m (k)=[w mN ...w (m+1)N-1 ] T , m=0.1,...,M-1; F represents the DTFT matrix, and F=2N×2N; I N is the N×N identity matrix; O N is an N×N zero matrix; the far-end signal is represented in the frequency domain as a set of segmented far-end signals X m (k).

[0056] The calculation formula of the quadratic norm is:

[0057]

[0058] Step S42: Arrange the quadratic norms in descending order, select the top two quadratic norms and record them as E1 and E2, record the index values ​​corresponding to E1 and E2 as m1 and m2, and determine whether the absolute value of the time difference between m1 and m2 is greater than or equal to the interval threshold. If so, proceed to step S43; if not, it indicates that there is no delay jitter, and proceed to step S45;

[0059] Step S43: determine whether the time-varying trends of E1 and E2 are the same. If so, it indicates that there is delay jitter, and the process proceeds to step S44; if not, it indicates that there is no delay jitter, and the process proceeds to step S45;

[0060] Step S44: Perform weighted averaging on the remote buffer frames corresponding to m1 and m2 to obtain a remote reference signal, set the delay state to true, and proceed to step S46;

[0061] The formula for weighted average is:

[0062] X m (k) = α1·X m1 (k)+α2·X m2 (k);

[0063] Wherein, α1 and α2 are weighted coefficients, and α1+α2=1 to ensure that the value does not overflow;

[0064] Step S45: Use the remote signal corresponding to the largest index value as the remote reference signal, set the delay state to false, and proceed to step S46;

[0065] Step S46: output the far-end reference signal, error signal, and delay state to a nonlinear suppression module.

[0066] In step S41, the interval threshold is a positive integer greater than or equal to 2.

[0067] The step S50 specifically includes:

[0068] Step S51: The nonlinear suppression module sets a default suppression strength and an enhancement coefficient, receives the remote reference signal, the error signal, and the delay status, and determines whether the delay status is true or false. If true, the process proceeds to step S52; if false, the process proceeds to step S53.

[0069] Step S52: The nonlinear suppression module multiplies the default suppression strength by the enhancement coefficient to obtain an enhanced suppression strength, suppresses the echo of the far-end signal by using the enhanced suppression strength, the far-end reference signal, and the error signal, and outputs an echo-free signal.

[0070] Step S53: The nonlinear suppression module suppresses the echo of the far-end signal by using the default suppression strength, the far-end reference signal, and the error signal, and outputs an echo-free signal to retain more near-end signals and ensure the transparency of the near-end signals, that is, to ensure the transparency of duplex voice.

[0071] In summary, the advantages of the present invention are:

[0072] A linear filtering module including a linear filter and a far-end signal synthesis unit is provided. The linear filter generates an error signal, and the far-end signal synthesis unit generates a far-end reference signal and a delay status. The error signal, far-end reference signal, and delay status are then sent to a nonlinear suppression module. The nonlinear suppression module determines whether jitter occurs in the delay of the far-end signal based on the delay status. If jitter occurs, the far-end signal echo is enhanced and suppressed based on a default suppression strength and enhancement coefficient to eliminate residual echo. This effectively accelerates echo convergence, shortens the duration of missed echoes, and improves the call experience when jitter or jumps occur in the delay and delay calibration feedback has not been timely fed back. If jitter does not occur, the far-end signal echo is suppressed based on the default suppression strength to ensure the transparency of the near-end signal. Ultimately, the delay jitter of the echo is compensated, greatly improving the echo cancellation effect and thus greatly improving the call quality.

[0073] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for echo cancellation in a delay jitter scenario, characterized by: The steps include: Step S10: collecting the far-end signal of the remote conference terminal and the near-end signal of the local conference terminal in real time; Step S20: calibrating and aligning the far-end signal and the near-end signal through a delay calibration module and then inputting the signals into a linear filtering module; Step S30: The linear filter of the linear filtering module outputs a linear filter coefficient and an error signal to the far-end signal synthesis unit based on the far-end signal and the near-end signal; Step S40: The far-end signal synthesis unit of the linear filtering module calculates a far-end reference signal and a delay state based on the linear filter coefficient, and outputs the far-end reference signal, error signal and delay state to the nonlinear suppression module; Step S50: The nonlinear suppression module outputs an anechoic signal based on the input far-end reference signal, error signal, and delay state; The step S40 specifically includes: Step S41: The far-end signal synthesis unit of the linear filter module divides the sampling points of the linear filter into blocks based on the segmented block frequency domain method, calculates the quadratic norm of the linear filter coefficient in each block, sets the index value of each quadratic norm based on the block, and sets an interval threshold; Step S42: Arrange the quadratic norms in descending order, select the top two quadratic norms and record them as E1 and E2, record the index values ​​corresponding to E1 and E2 as m1 and m2, and determine whether the absolute value of the time difference between m1 and m2 is greater than or equal to the interval threshold. If so, proceed to step S43; if not, it indicates that there is no delay jitter, and proceed to step S45; Step S43: determine whether the time-varying trends of E1 and E2 are the same. If so, it indicates that there is delay jitter, and the process proceeds to step S44; if not, it indicates that there is no delay jitter, and the process proceeds to step S45; Step S44: Perform weighted averaging on the remote buffer frames corresponding to m1 and m2 to obtain a remote reference signal, set the delay state to true, and proceed to step S46; Step S45: Use the remote signal corresponding to the largest index value as the remote reference signal, set the delay state to false, and proceed to step S46; Step S46: output the far-end reference signal, error signal, and delay state to a nonlinear suppression module.

2. The echo cancellation method in a delay jitter scenario according to claim 1, characterized in that: The step S20 is specifically as follows: The far-end binarized spectrum of the far-end signal is calculated by the delay calibration module, the near-end binarized spectrum of the near-end signal is calculated, the delay difference between the far-end binarized spectrum and the near-end binarized spectrum is calculated, and the far-end signal and the near-end signal are calibrated and aligned based on the delay difference and then input into the linear filtering module.

3. The echo cancellation method in a delay jitter scenario according to claim 1, characterized in that: The step S30 is specifically as follows: The linear filter of the linear filtering module performs linear filtering on the far-end signal to obtain an estimated echo signal, subtracts the estimated echo signal from the near-end signal to obtain an error signal, uses the error signal as a feedback signal to update the linear filter, and outputs the linear filter coefficients of the linear filter and the error signal to the far-end signal synthesis unit.

4. The echo cancellation method in a delay jitter scenario according to claim 1, wherein: In step S30, the linear filter is an adaptive filter based on short-time Fourier transform.

5. The echo cancellation method in a delay jitter scenario according to claim 1, characterized in that: In step S41, the interval threshold is a positive integer greater than or equal to 2.

6. The echo cancellation method in a delay jitter scenario according to claim 1, wherein: The step S50 specifically includes: Step S51: The nonlinear suppression module sets a default suppression strength and an enhancement coefficient, receives the remote reference signal, the error signal, and the delay status, and determines whether the delay status is true or false. If true, the process proceeds to step S52; if false, the process proceeds to step S53. Step S52: The nonlinear suppression module multiplies the default suppression strength by the enhancement coefficient to obtain an enhanced suppression strength, suppresses the echo of the far-end signal by using the enhanced suppression strength, the far-end reference signal, and the error signal, and outputs an echo-free signal. Step S53: The nonlinear suppression module suppresses the echo of the far-end signal by using the default suppression strength, the far-end reference signal and the error signal, and outputs an echo-free signal.

Images