Acoustic echo cancellation method for double-talk call

Abstract

The invention provides an acoustic echo cancellation method for two-way communication. By adopting the echo cancellation method provided by the invention, the neural network model is utilized to carry out end-to-end learning and modeling on the input signal and the residual echo signal, so that more accurate and effective echo cancellation is realized. After a near-end signal and a far-end signal are subjected to time delay alignment and linear echo cancellation, part of noise and echo still may remain in the near-end signal. Through the neural network, the residual components can be better eliminated, and the residual problem which affects the hearing feeling is avoided. The method combines the advantages of a traditional method and the capability of a neural network technology, can better adapt to a complex call environment, reduces the calculation cost of a system, and provides higher-quality call experience.

Description

Technical Field

[0001] The present invention relates to the field of electronic information technology, and more specifically, to an echo elimination method. Background Art

[0002] In a two-way call, echo refers to the phenomenon that the remote voice signal sent from the far end to the near end is re-collected by the near end microphone and transmitted back to the far end after being played by the near end speaker. This echo signal is caused by the reflection and attenuation of the audio signal when it is transmitted between the near end devices. During the call, the speaker will play the voice signal from the far end, and the near end microphone will capture the reflection of this signal while receiving this signal, forming an echo signal. This echo signal is mixed with the sound signal generated locally, causing the far end to hear the echo of their own speech, thus affecting the call quality and clarity.

[0003] Traditional echo cancellation methods usually use digital signal processing technology. Among them, a common method is to monitor the echo signal through an adaptive filter and try to remove it from the input signal to achieve the effect of echo cancellation.

[0004] An adaptive filter is a digital signal processing technology used to reduce certain types of interference, such as echo, from an input signal. It works by analyzing the correlation between the input signal and the interference signal and dynamically adjusting the filter parameters to minimize or eliminate this correlation, thereby reducing or eliminating the impact of the interference signal. In a two-way call, the role of the adaptive filter is to monitor the echo signal and try to remove it from the input signal to reduce or eliminate the impact of the echo on the call quality. It usually continuously adjusts its own filter parameters during the real-time processing of the call signal to adapt to changes in the call environment, such as the strength and delay of the echo signal. Through this dynamic adjustment, the adaptive filter can improve the call quality to a certain extent, making the call process clearer and more audible. However, the adaptive filter also has some disadvantages. First, its performance is highly dependent on the characteristics of the environment and the choice of parameters. On resource-constrained edge devices, it not only faces challenges in real time, but also cannot quickly adapt to environmental changes, resulting in reduced effectiveness. Moreover, the adaptive filter needs to process a large amount of data in real time and continuously update the filter parameters. Advanced algorithms may require large computing resources, which reduces the efficiency of running on resource-constrained devices. In addition, the adaptive filter will cause misoperation on non-echo signals, resulting in reduced call quality. Summary of the invention

[0005] The present invention proposes a method for acoustic echo cancellation of double-talk calls by combining an adaptive filter and a neural network model. At present, the neural network model has made remarkable achievements in the field of speech processing. Its powerful nonlinear modeling ability and adaptability to large-scale data make it an ideal choice for improving the echo cancellation effect. The present invention uses a neural network model to perform end-to-end learning and modeling of input signals and residual echo signals, thereby achieving more accurate and effective echo cancellation. By combining the advantages of traditional methods and the capabilities of neural network technology, it can better adapt to complex call environments, reduce the computing cost of the system, and provide a higher quality call experience.

[0006] In order to achieve the above object of the invention, the present invention provides an echo cancellation method, which comprises the following steps:

[0007] (1) acquiring a first frame of an audio signal of an audio sequence; the audio signal comprising a far-end signal and a near-end signal;

[0008] (2) outputting the far-end signal and the near-end signal to a delay alignment module;

[0009] (3) The delay alignment module calculates the delay between the far-end signal and the near-end signal and advances the far-end signal by the delay time.

[0010] (4) passing the advanced far-end signal and the near-end signal through a linear echo cancellation module;

[0011] (5) Linear echo cancellation suppresses the near-end signal based on the far-end signal;

[0012] (6) outputting the processed near-end signal to a residual echo cancellation module;

[0013] (7) Residual echo cancellation processes the noise and residual echo in the near-end signal;

[0014] (8) Transmit the processed near-end signal to the remote device

[0015] (9) Determine whether the audio sequence is the last frame. If not, obtain the next frame signal and return to step (2); if the audio sequence is the last frame, end the echo cancellation.

[0016] The present invention proposes a method for acoustic echo cancellation of double-talk calls by combining an adaptive filter and a neural network model. Compared with traditional methods, this method significantly improves the call quality while maintaining real-time communication. Especially in the face of complex echo situations, the generalization ability of the neural network model effectively improves the clarity and stability of the call. The present invention also focuses on algorithm optimization and performance improvement. By accelerating the convergence speed and reducing the amount of calculation, the method can run in real time on resource-constrained edge devices and still achieve good echo cancellation effects.

[0017] The trained neural network includes an input layer, a convolutional layer, an activation layer, a pooling layer, a fully connected layer and an output layer.

[0018] The trained neural network training samples are far-end signal frequency domain signals, and the target samples are near-end signal frequency domain signals. Mixed audio signals are easier to distinguish near-end signals and echo signals in the frequency domain, which is conducive to extracting feature information of echo signals during training.

[0019] When suppressing residual echo, suppression is performed based on correlation between signals, wherein the correlation between signals includes at least one of the following signal correlations: correlation between near-end signal and far-end signal, correlation between near-end signal and error signal, and correlation between far-end signal and error signal.

[0020] The adaptive filter is one of the following filters: a FIR filter or an IIR filter.

[0021] The adaptive filter adopts one of the following adaptive algorithms: LMS algorithm or NLMS algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The present invention provides a flow chart of an echo cancellation method.

[0023] Figure 2 Flow chart of the convolutional neural network training model used in the present invention DETAILED DESCRIPTION

[0024] The following, in conjunction with the drawings, further describes the technical means adopted by the present invention to achieve the predetermined invention objectives.

[0025] like Figure 1 As shown in Figure 1 The present invention provides a flow chart of an echo cancellation method.

[0026] Step 110: Acquire a first frame of an audio signal of an audio sequence, where the audio sequence includes a near-end signal and a far-end signal, wherein the near-end signal includes at least an echo signal and a noise signal, and the far-end signal is a far-end speech signal.

[0027] Step 120, estimate the time delay between the far - end voice and the near - end voice, which includes three steps:

[0028] 1. Use the fast Fourier transform to transform the far - end signal and the near - end signal into the frequency domain;

[0029] 2. Select several frequency point values in the frequency - domain data block, take the mean of these frequency point values as the threshold, and then compare all the frequency - domain data with the threshold value in turn for binarization, so as to convert each frequency - domain data block into a 32 - bit unsigned integer type;

[0030] 3. Perform an exclusive - OR operation on the far - end binarized integer and the historical data in the cache, take the minimum value of the result. The near - end data block it represents has the smallest spectral difference from the current far - end data block, and then calculate the echo time delay according to its position in the cache.

[0031] Step 130, perform linear echo cancellation on the input signal, which includes seven steps:

[0032] 1. The input signal is first divided into several time blocks, and each time block contains a certain number of signal samples. The purpose of block division is to process the signal in the frequency domain and reduce the processing complexity;

[0033] 2. Apply the Hanning window function to each time block:

[0034]

[0035] 3. Perform a discrete Fourier transform (DFT) on the signal of each windowed time block to convert the time - domain signal into a frequency - domain signal. Thus, the signal is represented as the amplitude and phase of a series of frequency components;

[0036] Given a discrete - time sequence X[n] of length N, its discrete Fourier transform X[K] is defined as:

[0037]

[0038] where X[K] is the complex value at the frequency of which represents the amplitude and phase information of the signal at that frequency, k represents the frequency, and its value range is 0 ≤ k < N, there are N frequency points in total, n represents the time index of the sequence, and its value range is 0 ≤ n < N, and j is the imaginary unit;

[0039] 4. For each time block in the frequency domain, calculate the coefficients of the adaptive filter in the frequency domain according to the error between the current signal and the desired signal using the LMS algorithm. These filter coefficients are used to perform frequency - domain filtering on this time block:

[0040] w(n+1)=w(n)+μ·e(n)·x(n) (2-3)

[0041] Where w(n) is the coefficient vector of the filter at time n, μ is the step size parameter (learning rate), e(n) is the error signal at the current time, and x(n) is the input signal at the current time;

[0042] 5. The updated filter coefficients are applied to the signal in the frequency domain to perform a filtering operation. That is, a multiplication operation is performed on the frequency domain signal to adjust the amplitude and phase of the signal to achieve a filtering effect;

[0043] 6. Perform an inverse discrete Fourier transform (IDFT) on the processed frequency domain signal to convert the signal from the frequency domain back to the time domain;

[0044] Given a discrete time series X[k] of length N, where k = 0, 1, 2, ..., N-1, its inverse discrete Fourier transform X[n] is defined as:

[0045]

[0046] Where X[n] is the value of the frequency domain signal sequence X[k] at time n, and X[k] is the value of the frequency domain signal sequence at frequency k. is the rotation factor, j is the imaginary unit, k represents the frequency index, and N is the length of the signal sequence;

[0047] 7. Since there is usually overlap when the signal is divided into blocks, the processed time domain signal needs to be overlap-added to merge the processing results of adjacent time blocks.

[0048] Step 140, performing residual echo elimination on the input signal, includes nine steps:

[0049] 1. First, perform a low-pass filtering operation on the input signal to remove high-frequency noise and irrelevant signal components. Then perform a downsampling operation to reduce the sampling rate of the signal to reduce the computational complexity;

[0050] 2. Perform autocorrelation on the low-pass filtered and downsampled signal to analyze the periodic components in the sound signal;

[0051]

[0052] Where y[n] is the output signal of the filter, x[n] is the input signal, h[k] is the coefficient of the filter, which is used to weight the input signal, and N is the length of the filter;

[0053] 3. Using the information of the autocorrelation function, the linear prediction analysis is performed using the square error minimization method to obtain the linear prediction coefficients (LPC coefficients). LPC analysis is used to establish a model that attempts to describe the vocal tract characteristics of the signal with fewer parameters;

[0054] Given a signal x[n] of length N, its autocorrelation function r x [m] is defined as:

[0055]

[0056] Linear prediction analysis aims to find a model x′[n] to approximate x[n] so that the sum of the squares of the prediction errors e[n] = x[n]-x′[n] between the model output and the original signal is minimized. This problem can be solved by minimizing the sum of the squares of the prediction errors:

[0057]

[0058] By taking the derivative of E and setting the derivative to zero, we can get the linear prediction coefficient;

[0059] 4. Apply the obtained linear prediction coefficients to the low-pass filtered signal to obtain the LPC residual signal. The LPC residual signal contains the high-frequency and periodic components in the original signal that are not captured by the linear prediction model. These components represent the vocal cord excitation signal;

[0060] 5. Perform autocorrelation operation on the LPC residual signal to determine the period value of the vocal cord excitation signal, that is, the period length of the periodic pattern that recurs in the sound signal;

[0061] 6. Perform a bandpass filtering operation on the input signal to retain the signal components within a specific frequency range. Then, calculate the cepstral coefficients of the signal, which are the representation of the signal's spectral information in the cepstral domain. By applying bandpass filtering within a specific frequency range, extract the BFCC (Bandpass Filtered Cepstral Coefficients) features of the input signal;

[0062] 7. After the BFCC feature is input into the neural network model, it is processed by the neural network model to obtain the output gain value. This gain value represents the correlation between the clean speech and the residual echo learned by the neural network model, and is used to adjust the input signal to reduce the impact of the echo;

[0063] 8. Strengthen the resonance peak in the BFCC feature and increase the amplitude of the resonance peak;

[0064] 9. Multiply the output signal obtained in step 8 by the gain value obtained in step 7 to obtain an output signal; step 150, subject the output of step 140 to inverse BFCC processing to obtain a final output signal.

Claims

1. An acoustic echo cancellation method for double-talk communication, characterized in that: It includes two stages: training and elimination. The training stage includes two steps, specifically: Step 1, initialization: use all normal near-end and far-end samples to train the reconstruction model. After the training is completed, record the STFT consistency loss value of each sample; Step 2, sort the several STFT consistency loss values ​​obtained in step 1 from small to large, select the first several models, and test these models on some abnormal near-end and far-end samples. After the test is completed, select the model with the smallest STFT consistency loss value; The elimination stage includes: first, align the delay of the input near-end and far-end signals, then pass through the linear elimination module, and finally use the model selected in the training stage to perform residual echo elimination, and finally obtain the output signal after echo elimination.

Images