Adaptive path length echo cancellation method, device and storage medium

Abstract

This application discloses an echo cancellation method, device, and storage medium with adaptive path length. The echo cancellation method includes: performing echo estimation using a block-based frequency domain filter to obtain a priori estimated echo signal; subtracting the priori estimated echo signal from the audio signal acquired by a microphone to obtain a priori residual signal; inputting the priori residual signal and the acquired signal into a neural network model to obtain the block activation probability; counting the number of effectively activated filter blocks based on the block activation probability to obtain the adaptive path length; calculating an update step size based on the adaptive path length; calculating an update gradient based on the update step size, the acquired signal, and the prior residual signal; updating the block-based frequency domain filter based on the update gradient; and performing echo cancellation using the updated block-based frequency domain filter. This method achieves adaptation of the update gradient and echo path length, improves the update accuracy of the filter, and does not increase the computational resource requirements.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to an echo cancellation method, apparatus and storage medium with adaptive path length. Background Technology

[0002] Adaptive filtering technology is widely used in echo cancellation tasks. In echo cancellation, the microphone receives sound from the device's speakers. If this sound is not canceled, it will be picked up by the microphone, creating an echo and affecting audio quality. Furthermore, if the device's acoustic hardware or structural design is flawed, if low-quality speakers are used to reduce costs, or if the device is played at high volumes, the sound emitted by the speakers will experience significant distortion, greatly increasing the difficulty of the echo cancellation task.

[0003] Traditional echo cancellation solutions have low computational complexity but limited effectiveness and are difficult to handle echoes caused by devices or equipment with high distortion. While existing deep learning solutions can improve echo cancellation performance, they often require high computational resources and are difficult to implement and deploy in real-world end-to-end device scenarios. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides at least one echo cancellation method, apparatus, and storage medium with adaptive path length.

[0005] The first aspect of this application provides an echo cancellation method with adaptive path length. The method includes: performing echo estimation on the back-sampled signal from a loudspeaker using a block frequency domain filter to obtain a priori estimated echo signal; subtracting the priori estimated echo signal from the audio signal collected by a microphone to obtain a priori residual signal; inputting the prior residual signal and the back-sampled signal into a pre-trained neural network model to obtain the block activation probability output by the neural network model; wherein, the block activation probability represents the probability of effective activation of each filter block in the block frequency domain filter; counting the number of effectively activated filter blocks based on the block activation probability to obtain an adaptive path length; calculating an update step size based on the adaptive path length; calculating an update gradient based on the update step size, the back-sampled signal, and the prior residual signal; updating the block frequency domain filter based on the update gradient; performing echo estimation on the back-sampled signal using the updated block frequency domain filter to obtain a posterior estimated echo signal; and subtracting the posterior estimated echo signal from the audio signal to obtain the echo-cancelled audio signal.

[0006] In one embodiment, the adaptive path length is obtained by calculating the number of effectively activated filter blocks based on the block activation probability, including: summing the block activation probabilities corresponding to each filter block; and using the summation result as the number of effectively activated filter blocks to obtain the adaptive path length.

[0007] In one embodiment, the neural network model also outputs a frequency domain step size, which characterizes the update intensity of the block frequency domain filter; the update step size is calculated based on the adaptive path length, and the update gradient is calculated based on the update step size, the backsampled signal, and the prior residual signal, including: calculating a normalized step size based on the adaptive path length and the frequency domain step size; and multiplying the normalized step size, the conjugate of the backsampled signal in the frequency domain, and the prior residual signal to obtain the update gradient.

[0008] In one embodiment, the normalized step size is calculated based on the adaptive path length and the frequency domain step size, including: calculating the power spectrum of the frequency-domain backsampled signal to obtain the frequency-domain backsampled power spectrum; multiplying the frequency-domain backsampled power spectrum and the adaptive path length, and weighting and summing the product result with the frequency-domain self-orthogonalization factor of the previous frame to obtain the frequency-domain self-orthogonalization factor of the current frame; and calculating the ratio between the frequency-domain step size and the frequency-domain self-orthogonalization factor of the current frame to obtain the normalized step size.

[0009] In one embodiment, updating the block frequency domain filter based on the update gradient includes: for any filter block in the block frequency domain filter, multiplying the block activation probability corresponding to the filter block and the update gradient to obtain the gradient update amount; and summing the filter parameters of the filter block and the gradient update amount to obtain the updated filter block.

[0010] In one embodiment, updating the block frequency domain filter based on the update gradient includes: selecting filter blocks from the block frequency domain filter whose block activation probability is greater than a preset probability threshold to obtain a filter block to be updated; updating the filter parameters of the filter block to be updated based on the update gradient corresponding to the filter block to be updated to obtain the updated filter block.

[0011] In one embodiment, the prior residual signal and the back sampled signal are input into a pre-trained neural network model to obtain the block activation probability output by the neural network model, including: concatenating the absolute values ​​of the back sampled signal and the prior residual signal in the frequency domain to obtain input features; and inputting the input features into the pre-trained neural network model to obtain the frequency domain step size and block activation probability output by the neural network model.

[0012] In one embodiment, the neural network model includes an input linear layer, a gated recurrent unit, a step size prediction linear layer, and a block effectiveness probability prediction linear layer.

[0013] A second aspect of this application provides an echo cancellation device with adaptive path length. The device includes: a priori frequency domain filtering module, used to perform echo estimation on the back-sampled signal from a loudspeaker using a block-based frequency domain filter to obtain a priori estimated echo signal, and subtract the priori estimated echo signal from the audio signal collected by a microphone to obtain a priori residual signal; and an activation probability prediction module, used to input the priori residual signal and the back-sampled signal into a pre-trained neural network model to obtain the block activation probability output by the neural network model; wherein, the block activation probability represents the probability of effective activation of each filter block in the block-based frequency domain filter. The system comprises the following modules: a path length calculation module, used to calculate the number of effectively activated filter blocks based on the block activation probability, and obtain the adaptive path length; an update gradient calculation module, used to calculate the update step size based on the adaptive path length, and to calculate the update gradient based on the update step size, the backsampled signal, and the prior residual signal; and a posterior frequency domain filtering module, used to update the block frequency domain filters based on the update gradient, to perform echo estimation on the backsampled signal using the updated block frequency domain filters, to obtain the posterior estimated echo signal, and to subtract the posterior estimated echo signal from the audio signal to obtain the echo-cancelled audio signal.

[0014] A third aspect of this application provides an electronic device including a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the aforementioned adaptive path length echo cancellation method.

[0015] A fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the aforementioned echo cancellation method with adaptive path length.

[0016] The above scheme obtains the block activation probability by inputting the prior residual signal and the back-sampled signal into a pre-trained neural network model. Based on the block activation probability, the number of effectively activated filter blocks is counted to obtain the adaptive path length. The update step size is calculated based on the adaptive path length. The update gradient is calculated based on the update step size, the back-sampled signal, and the prior residual signal, thus achieving the adaptation of the update gradient and the echo path length. Then, the block frequency domain filter is updated based on the update gradient, enabling the block frequency domain filter to simultaneously estimate the optimal echo path for both short-path and long-path echoes, improving the update accuracy of the filter, optimizing the performance of subsequent linear echo cancellation, and the overall algorithm does not increase the computational resource requirements, thus realizing a low-complexity linear echo cancellation scheme.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a flowchart illustrating an adaptive path length echo cancellation method as shown in an exemplary embodiment of this application; Figure 2 This is a schematic diagram illustrating echo cancellation in an exemplary embodiment of this application; Figure 3 This is a schematic diagram illustrating filter updating in an exemplary embodiment of this application; Figure 4 This is a schematic diagram illustrating the filter update dimension in an exemplary embodiment of this application; Figure 5 This is a schematic diagram illustrating the effect of echo cancellation on short-path echoes in an exemplary embodiment of this application; Figure 6 This is a schematic diagram illustrating the effect of echo cancellation on long-path echoes, as shown in an exemplary embodiment of this application. Figure 7 This is a block diagram illustrating an echo cancellation device with adaptive path length, as shown in an exemplary embodiment of this application. Figure 8 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application; Figure 9 This is a schematic diagram illustrating the structure of a computer-readable storage medium, as shown in an exemplary embodiment of this application. Detailed Implementation

[0020] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0021] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0022] In this document, the term "and / or" is merely a description of the association information of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0023] Echoes are mainly caused by the sound played by the speaker being picked up again by the microphone, thus mixing into the original audio acquisition signal. This causes the devices on both sides of the audio communication to play the same sound repeatedly, affecting the clarity and naturalness of the audio playback.

[0024] Adaptive filtering techniques are commonly used for echo cancellation. They mainly simulate the echo channel by estimating a set of filter coefficients W and convolving these filter coefficients with the reference signal to obtain the estimated echo. The audio signal collected by the microphone is then subtracted from the estimated echo to obtain the echo-cancelled signal.

[0025] Normalized Least Mean Squares (NLMS) is a common adaptive filter update criterion; Frequency Domain Adaptive Filter (FDAF) is an NLMS algorithm performed in the frequency domain; and sometimes the room impact response is long. In order to track echo data with long echo channels, Partitioned Block Frequency Domain Adaptive Filter (PBFDAF) is generally used. It increases the length of the filter, divides the frequency domain filter into blocks, and each block of the filter is multiplied by the frame of the corresponding reference channel to update the block frequency domain filter, thus realizing the PBFDAF algorithm.

[0026] In addition, Kalman filtering (KF) updates the filter by introducing state transition variables and Kalman gain coefficients, and by updating the Kalman gain coefficients in real time with the goal of minimizing the variance of the estimation error.

[0027] In related technologies, neural networks are generally used to predict the frequency domain step size to assist in filter updates, thereby optimizing the performance of the PBFDAF algorithm. Current experimental results show that neural networks can indeed improve the performance of PBFDAF, enabling it to reach or even surpass the performance of Kalman filtering. However, these neural network frequency domain step size estimation schemes encounter the following problems in practice: Since the echo path length varies in reality, if the filter used is too long, it will lead to insufficient convergence of the filter for early echoes when processing short path echoes, resulting in poor processing results. However, if the filter used is too short, it will be unable to cover long echo paths, also leading to insufficient echo cancellation.

[0028] Therefore, in order to enable the block frequency domain filter PBFDAF to simultaneously accommodate both long and short echo paths, this application provides at least one echo cancellation method, device, and storage medium with adaptive path length, which improves the echo cancellation effect without increasing computational complexity.

[0029] The echo cancellation method with adaptive path length of this application can be applied to any audio communication scenario, such as voice calls, online meetings, intelligent voice control, live streaming, etc. It is executed by audio communication devices and / or servers that are connected to the audio communication devices. This embodiment does not limit the implementation environment and execution subject to which the method is applicable.

[0030] The echo cancellation method with adaptive path length provided in the embodiments of this application is described below.

[0031] Please see Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of the echo cancellation method with adaptive path length, which can be applied to an audio communication terminal or other electronic devices connected to the audio communication terminal, such as a server. Figure 1 As shown, the echo cancellation method with adaptive path length includes at least steps S110 to S150, which are described in detail below: Step S110: Use a block frequency domain filter to perform echo estimation on the speaker's echo signal to obtain a priori estimated echo signal. Subtract the priori estimated echo signal from the audio signal collected by the microphone to obtain the priori residual signal.

[0032] Audio communication terminals include speakers and microphones.

[0033] A microphone is a transducer that converts sound signals into electrical signals. Microphones collect ambient sounds to obtain audio signals, which include near-field sound signals and echo signals generated by sound played by speakers.

[0034] Specifically, the audio signal is a mixed signal actually captured by the microphone. In addition to the sound signal that needs to be retained, the audio signal also contains other noise that needs to be eliminated, especially the echo played from the speaker and captured by the microphone. Taking a phone call as an example, the audio signal usually contains near-end human voice, echo, and ambient noise.

[0035] The echo signal refers to the signal collected from the playback loop. The echo signal is the reference signal used to eliminate echo.

[0036] For example, a sampled signal can be obtained from the digital audio signal played by the speaker. For instance, if an audio signal is to be played through the speaker, it can be directly copied before being sent to the speaker driver circuit and digital-to-analog converter for playback, thus obtaining the sampled signal.

[0037] For example, after the speaker plays sound, an additional microphone can be placed near the speaker to capture the actual sound emitted by the speaker and obtain a feedback signal.

[0038] For example, the replicated audio signal that the speaker needs to play can be combined with the acquired sound signal actually emitted by the speaker to obtain the re-sampled signal.

[0039] The method of acquiring the acquired signal can be selected according to the specific application scenario. This application does not limit the specific method of acquiring the acquired signal.

[0040] The block frequency domain filter is a filter implemented based on the block frequency domain adaptive filtering algorithm. It divides the back-sampled signal in the time domain into blocks, converts it to the frequency domain, and performs filtering operations in the frequency domain to predict the echo generated at the microphone after the back-sampled signal passes through the echo path. The echo prediction result output before the filter is updated is used as the prior estimated echo signal.

[0041] Then, the prior estimated echo signal is subtracted from the audio signal to obtain the prior residual signal.

[0042] Step S120: Input the prior residual signal and the retrieval signal into the pre-trained neural network model to obtain the block activation probability output by the neural network model.

[0043] Here, the block activation probability represents the probability that each filter block in the block frequency domain filter is effectively activated.

[0044] A pre-trained neural network model is used to analyze the probability of effective activation of each filter block in the block frequency domain filter relative to the input prior residual signal and the sampled signal, that is, the probability that each filter block is in an effective working state.

[0045] For example, if the block activation probability ranges from 0 to 1, if the block activation probability of the filter block is 1, it means that the block is in an active working state; if the block activation probability of the filter block is 0, it means that the block is not activated.

[0046] Step S130: Based on the block activation probability, count the number of effectively activated filter blocks to obtain the adaptive path length.

[0047] The adaptive path length is used to characterize how many filter blocks in a block frequency domain filter are in an active state.

[0048] For example, the block activation probabilities of all filter blocks can be summed directly, and the number of effectively activated filter blocks can be obtained from the summation result, which is the adaptive path length.

[0049] For example, filter blocks with a block activation probability greater than a preset probability threshold can be used as valid activated filter blocks, and the number of filter blocks with a block activation probability greater than the preset probability threshold can be counted to obtain the adaptive path length.

[0050] The specific method for calculating the number of effectively activated filter blocks based on the block activation probability can be flexibly selected according to the application scenario, and this application does not limit it.

[0051] Step S140: Calculate the update step size based on the adaptive path length, and calculate the update gradient based on the update step size, the backsampled signal, and the prior residual signal.

[0052] The update step size controls the update speed of the filter, while the update gradient indicates how the current filter parameters should be adjusted to minimize the error in the fastest way.

[0053] Specifically, the update step size is calculated based on the adaptive path length. The adaptive path length and the update step size are inversely correlated. The instantaneous correlation between the sampled signal and the prior residual signal is calculated to obtain the estimated gradient. The estimated gradient and the update step size are multiplied to obtain the update gradient, thus achieving the matching of the update intensity with the actual requirements of the echo path.

[0054] Taking the Normalized Least Mean Squares (NLMS) algorithm as an example, existing NLMS algorithms generally use a fixed filter length L to calculate the frequency domain self-orthogonalization factor. Then, the ratio of the frequency domain step size and the frequency domain self-orthogonalization factor is calculated to obtain the normalized step size. The normalized step size is then used as the update step size and the estimated gradient to calculate the updated gradient. Since the filter length L is fixed, the filter cannot take into account both long and short echo paths.

[0055] This application can replace the fixed filter length L in the normalized least mean square algorithm with an adaptive path length, so that the update step size can be dynamically adjusted according to the current adaptive path length. When the path is short, the step size is increased to accelerate convergence, and when the path is long, the step size is normalized to ensure stability.

[0056] Of course, in addition to the normalized least mean square algorithm, other algorithms can also be used to calculate the update gradient. For example, in the frequency-domain least mean square (FD-LMS) algorithm, the fixed step size is replaced with an update step size calculated based on the adaptive path length to calculate the update gradient; or in the frequency-domain affine projection algorithm (FD-APA), the update step size calculated in this application is increased to control the update speed. The specific calculation method of the update gradient can be flexibly selected according to the actual application scenario, and this application does not limit it.

[0057] Step S150: Update the block frequency domain filter based on the update gradient, use the updated block frequency domain filter to perform echo estimation on the re-sampled signal to obtain the posterior estimated echo signal, and subtract the posterior estimated echo signal from the audio signal to obtain the echo-cancelled audio signal.

[0058] It can store or transmit the echo-cancelled audio signal, such as transmitting it to another audio communication terminal.

[0059] After obtaining the updated gradient, the block frequency domain filter is updated to obtain the updated block frequency domain filter. The echo estimation of the resampled signal is re-performed using the updated block frequency domain filter to obtain a more accurate posterior estimated echo signal. The posterior estimated echo signal is subtracted from the audio signal to obtain the posterior residual signal, which is the audio signal after echo cancellation.

[0060] For example, please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating echo cancellation in an exemplary embodiment of this application, as shown below. Figure 2 As shown, the audio signal and the sampled signal are input to the block frequency domain filter for prior frequency domain filtering to obtain the prior residual signal. The prior residual signal and the sampled signal are input to the pre-trained neural network model to obtain the block activation probability output by the neural network model. The number of effectively activated filter blocks is determined according to the block activation probability of each filter block to obtain the adaptive path length. The update gradient is calculated based on the adaptive path length and updated according to the update gradient to obtain the updated block frequency domain filter. The updated block frequency domain filter is then used for posterior frequency domain filtering to obtain the posterior residual signal.

[0061] The following describes some embodiments of this application in detail.

[0062] For better explanation, the symbols in the formulas involved in the subsequent embodiments are explained below: The number of filter blocks in the block frequency domain filter is L, and the variables required for the calculation of the k-th frame are as follows: Audio signals in the time domain: ; Time-domain sampled signal, frequency-domain sampled signal: ; Frequency domain multi-frame retrieval signal: ; Prior estimation of echo signal in the time domain and prior estimation of echo signal in the frequency domain: ; Posterior estimation of echo signal in the time domain and posterior estimation of echo signal in the frequency domain: ; Prior residual signal in the time domain and prior residual signal in the frequency domain: ; Posterior residual signal in the time domain and posterior residual signal in the frequency domain: ; Block frequency domain filter: ; Gradient update for block frequency domain filter: ; Frequency domain step size: ; Block activation probability: ; Frequency domain self-orthogonalization factor: .

[0063] Taking L=B, 2*N point Discrete Fourier Transform (DFT) as an example, the Discrete Fourier Transform of real number input and its inverse transform are expressed as follows: , ; All are N-dimensional real vectors; All are complex vectors of N+1 dimensions; Both are complex arrays of shape [B, N+1], where They represent The array of column t, i.e. All are complex vectors of N+1 dimensions; It is an N+1 dimensional real vector. Let be a B-dimensional real vector. It is a real number.

[0064] in addition, The input is a 2*N dimensional real vector, and the output is an N+1 dimensional complex vector; The input is an N+1 dimensional complex vector, and the output is a 2*N dimensional real vector.

[0065] If the number of filter blocks in the block frequency domain filter is L, the sampled signal of the current frame is transformed to the frequency domain and spliced ​​with the sampled signal of the previous L-1 frames in the frequency domain to obtain the input of the block frequency domain filter.

[0066] Taking the 2*N Discrete Fourier Transform as an example, for the retrieval signal of the current k-th frame in the time domain... Use the following formula 1 to Transform to the frequency domain: (Formula 1) in, It is the re-sampled signal of the current k-th frame in the frequency domain. cat is the splicing algorithm, which specifically splices the re-sampled signal of the current frame and the previous frame, that is, splices two N-dimensional real vectors into a 2*N-dimensional real vector.

[0067] Then, the sampled signal in the frequency domain of the current k-th frame will be concatenated with the sampled signal in the frequency domain of the previous L-1 frames to obtain the input of the block frequency domain filter, as shown in Formula 2 below: (Formula 2) Frequency domain multi-frame retrieval signal It is obtained by continuously splicing the frequency domain backsampled signals of the current frame and the past L-1 frames. As input to the filter.

[0068] Of course, in addition to the discrete Fourier transform exemplified, algorithms such as the short-time Fourier transform (STFT) or wavelet transform (WT) can also be used for frequency domain transformation, and this application does not limit this to any particular algorithm.

[0069] Then, the audio signal and the sampled signal are input to the block frequency domain filter for a priori frequency domain filtering to obtain the a priori residual signal.

[0070] For example, the echo estimation of the re-sampled signal is first performed using the following formula 3 to obtain the prior estimated echo signal in the frequency domain. : (Formula 3) Multi-frame resampled signal in the frequency domain The t-th column array in With block frequency domain filter The t-th column array Perform dot product, then sum all the product results to obtain the echo estimation result of the current frame, which is the prior estimated echo signal in the frequency domain.

[0071] Then, the prior estimated echo signal in the time domain is obtained using the following formula 4: (Formula 4) in, , Let N represent the zero matrix and the identity matrix of dimension N, respectively. When using a 2*N point discrete Fourier transform, with each frame length taking N points... go through The transformation yields a real vector of dimension 2*N, which is then compared with the matrix. Multiply the data and retain only the last N points to obtain the prior estimate of the echo signal in the time domain. .

[0072] The time-domain conversion method in Formula 4 above is merely an illustrative example. Other methods can also be used for time-domain conversion, and this application does not limit this method.

[0073] Audio signals in the time domain The prior estimated echo signal is subtracted from the prior estimated echo signal to obtain the prior residual signal in the time domain. For detailed calculations, please refer to Formula 5 below: (Formula 5) The prior residual signal in the time domain is then transformed to the frequency domain, as detailed in Formula 6 below: (Formula 6) The prior residual signal in the time domain and the zero vector at point N The concatenation process yields a real vector of dimension 2*N. A Fourier transform is then performed on the concatenated result to obtain the prior residual signal in the frequency domain. .

[0074] The frequency domain conversion method in Formula 6 above is merely an illustrative example. Other methods can also be used for frequency domain conversion, and this application does not limit them.

[0075] Then, the prior residual signal and the sampled signal are input into a pre-trained neural network model to predict the probability of effective activation of each filter block in the block frequency domain filter, thus obtaining the block activation probability.

[0076] In some implementations, step S120 involves inputting the prior residual signal and the retrieval signal into a pre-trained neural network model to obtain the block activation probability output by the neural network model, including steps S121 to S122 below.

[0077] Step S121: Concatenate the absolute values ​​of the frequency domain sampled signal and the frequency domain prior residual signal to obtain the input features.

[0078] Specifically, the frequency domain sampling signal is spliced ​​using the following formula 7. and the prior residual signal in the frequency domain , to obtain input features : (Formula 7) The above-described input features are merely illustrative examples. Other methods can also be used to construct input features, such as further splicing prior estimated echo signals and / or audio signals to obtain input features. This application does not limit this approach.

[0079] Step S122: Input the input features into the pre-trained neural network model to obtain the block activation probability output by the neural network model.

[0080] The trained neural network model can predict the probability of effective activation of each filter block in the block frequency domain filter based on the input features.

[0081] Among them, supervised training, semi-supervised training, and unsupervised training can be used to train the neural network model so that it has the function of predicting the effective activation probability of filter blocks.

[0082] Furthermore, the structure of the neural network model can be flexibly adjusted according to the actual application scenario, and this application does not impose any limitations on it.

[0083] For example, a neural network model includes an input linear layer, a gated recurrent unit, a step size prediction linear layer, and a block activation probability prediction linear layer. In addition to predicting the effective activation probability of filter blocks, the neural network model can also predict the frequency domain step size, which represents the update intensity of the block frequency domain filter.

[0084] Specifically, input features The input is fed into the neural network model to obtain the frequency domain step size of the neural network model output. Probability of block activation For details, please refer to Formula 8 below: (Formula 8) in, To hide state variables, The updated hidden state variables are passed to the next time step for computation. Net is a neural network model.

[0085] After obtaining the block activation probability, the adaptive path length is calculated based on the block activation probability. The adaptive path length reflects the number of active filter blocks, i.e., the effective filter length.

[0086] In some implementations, step S130 involves calculating the number of effectively activated filter blocks based on the block activation probability to obtain the adaptive path length, including steps S131 to S132.

[0087] Step S131: Sum the block activation probabilities corresponding to each filter block.

[0088] Step S132: Use the summation result as the number of effectively activated filter blocks to obtain the adaptive path length.

[0089] The total number of active blocks is estimated by summing the block activation probabilities of each filter block in the block frequency domain filter.

[0090] For example, suppose the number of filter blocks in a block frequency domain filter is L=B. If all filter blocks in the block frequency domain filter are fully activated, and the block activation probability of each filter block is 1, then the adaptive path length calculated by summing is B, indicating that the entire block frequency domain filter is being used. If only the first 10 blocks are activated, and the block activation probability of the first 10 filter blocks is 1, and the last B-10 filter blocks are not activated, and the block activation probability of the last B-10 filter blocks is 0, then the adaptive path length calculated by summing is 10, indicating that the current acoustic environment only requires 10 filter blocks for echo estimation. In this case, the effective length of the equivalent filter is 10.

[0091] It should be noted that the above example uses a block activation probability of 0 or 1. In actual application scenarios, the block activation probability can also be other values, such as 0.5, to represent the probability of filter block activation.

[0092] In addition to the above embodiment which obtains the adaptive path length by summing the block activation probabilities corresponding to each filter block, other methods can also be used to calculate the adaptive path length, such as counting the number of filter blocks whose block activation probabilities are greater than a preset probability threshold, and using that number of filter blocks as the adaptive path length.

[0093] Then, the update step size of the filter is controlled according to the adaptive path length.

[0094] The adaptive path length can be directly substituted into the step size calculation function set based on experience to obtain the updated step size. Alternatively, the updated step size can be calculated by combining other parameters, such as the backsampled signal and the frequency domain step size.

[0095] For example, the neural network model also outputs a frequency domain step size; in step S140, the updated step size is calculated based on the adaptive path length, and the updated gradient is calculated based on the updated step size, the backsampled signal, and the prior residual signal, including: Step S141: Calculate the normalized step size based on the adaptive path length and frequency domain step size.

[0096] Specifically, the power of the backsampled signal is estimated based on the adaptive path length and the backsampled signal, and then normalized with the frequency domain step size to prevent the filter from updating too quickly when the signal power is high, leading to unstable updates, or the filter from updating too slowly when the power is low, leading to slow convergence.

[0097] Taking the normalized least mean square algorithm as an example, the normalized step size is calculated based on the adaptive path length and the frequency domain step size. This includes: calculating the power spectrum of the backsampled signal in the frequency domain to obtain the frequency domain backsampled power spectrum; multiplying the frequency domain backsampled power spectrum and the adaptive path length; weighting and summing the product result with the frequency domain self-orthogonalization factor of the previous frame to obtain the frequency domain self-orthogonalization factor of the current frame; and calculating the ratio between the frequency domain step size and the frequency domain self-orthogonalization factor of the current frame to obtain the normalized step size.

[0098] For example, the frequency domain self-orthogonalization factor can be calculated using the following formula 9: (Formula 9) in, It is the frequency domain self-orthogonalization factor of the current frame; This is a hyperparameter (usually set to 0.9) used to control the degree of retention of historical information in order to smooth the estimation of the power of the back-collected signal; Used to calculate the power spectrum of the frequency domain-sampled signal, thus obtaining the frequency domain-sampled power spectrum; This is used to sum the block activation probabilities corresponding to each filter block to obtain the adaptive path length, which replaces the fixed filter length parameter L in the original formula.

[0099] Then, the frequency domain backsampled power spectrum and the adaptive path length are multiplied together, and the product result is weighted and summed with the frequency domain self-orthogonalization factor of the previous frame to obtain the frequency domain self-orthogonalization factor of the current frame. .

[0100] Additionally, by copying, the N+1-dimensional complex vector is expanded into a complex array of shape [B, N+1], as detailed in Formula 10 below: (Formula 10) Formula 10 aligns the dimensions of the prior residual signal, frequency step size, and frequency self-orthogonalization factor in the frequency domain with the dimensions of the block frequency domain filter, facilitating subsequent calculations.

[0101] Then calculate the ratio of the frequency domain step size to the frequency domain self-orthogonalization factor to obtain the updated step size.

[0102] Of course, other methods can also be used to calculate the normalization step size. For example, in addition to using the frequency domain step size output by the neural network model, a fixed frequency domain step size can be used, or other methods can be used to estimate the frequency domain step size, or the frequency domain self-orthogonalization factor of the previous time step can be ignored when calculating the frequency domain self-orthogonalization factor of the current frame. The specific calculation method of the normalization step size can be flexibly adjusted according to the actual application scenario, and this application does not limit it.

[0103] Step S142: Calculate the product of the normalized step size, the conjugate of the frequency domain sampled signal, and the prior residual signal to obtain the update gradient.

[0104] Based on the above embodiments, the normalization step size is: The updated gradient is calculated using the following formula 11. : (Formula 11) in, This indicates the frequency domain multi-frame retrieval signal. Taking the conjugate, the updated gradient calculated above... It is also a complex array of [B, N+1].

[0105] Of course, in addition to the gradient calculation methods exemplified above, other algorithms can also be used to calculate the gradient update, and this application does not limit this.

[0106] Then, the block frequency domain filter is updated based on the updated gradient.

[0107] The updated gradient and the block frequency domain filter can be summed directly to obtain the updated block frequency domain filter; the update intensity of the filter block and / or the filter block that needs to be updated can be determined according to the block activation probability of each filter block, so as to control the update step size of the block frequency domain filter from the block dimension.

[0108] For example, updating a block-based frequency domain filter based on an update gradient includes: for any filter block in the block-based frequency domain filter, multiplying the block activation probability corresponding to the filter block with the update gradient to obtain the gradient update amount; and summing the filter parameters of the filter block with the gradient update amount to obtain the updated filter block.

[0109] Specifically, the block frequency domain filter is updated using the following formula 12: (Formula 12) pass This represents the update intensity of the corresponding filter block in the block frequency domain filter, that is, the update gradient is weighted using the block activation probability.

[0110] For example, if only 10 filter blocks are needed for echo estimation in the current acoustic environment, then the last B-10 filter blocks of the block frequency domain filter do not need to be updated. That is, the values ​​of the last B-10 filter blocks in the ideal block effectiveness probability should all be 0.

[0111] In practical applications, if the echo path is short, only a portion of the filter blocks need to be updated more strongly, while the coefficients of the remaining filter blocks are updated less or almost not at all, in order to save computational resources. If the echo path is long, more filter blocks are updated to cover the longer echo path.

[0112] The above embodiments support the estimation of long echo paths and can reduce or disable the updating of redundant blocks when the path is short, thus saving computation.

[0113] For example, updating a block-based frequency domain filter based on an update gradient includes: selecting filter blocks from the block-based frequency domain filter whose block activation probability is greater than a preset probability threshold to obtain a filter block to be updated; and updating the filter parameters of the filter block to be updated based on the update gradient corresponding to the filter block to be updated to obtain the updated filter block.

[0114] That is, only filter blocks whose block activation probability is greater than a preset probability threshold are updated. When updating the filter block to be updated, the update gradient and the block frequency domain filter can be summed directly, or the update intensity can be determined according to the block activation probability, as in the previous embodiment. This application does not limit this.

[0115] By updating only the blocks that need updating, the long filter can be flexibly adapted, allowing most of the inactive blocks of the filter to skip complex update operations when the echo path is short, thereby significantly reducing the amount of computation.

[0116] The filter update process is illustrated with a specific embodiment; please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram illustrating filter updates in an exemplary embodiment of this application, as shown below. Figure 3 As shown, the neural network model outputs the frequency domain step size and block activation probability. Based on the frequency domain step size, block activation probability, and other variables (such as the sampled signal and the prior residual signal), gradient calculation is performed to obtain the update gradient. The block update activation is performed by combining the update gradient and the block activation probability. Specifically, the filter block to be updated and / or the update intensity of the filter block are determined based on the block activation probability, and the update amount of the filter block is determined based on the update gradient to obtain the updated block frequency domain filter.

[0117] Additionally, please see Figure 4 , Figure 4 This is a schematic diagram illustrating the filter update dimension in an exemplary embodiment of this application, as shown below. Figure 4As shown, the block update activation mechanism multiplies the corresponding block gradient in the update gradient from both the block dimension and the frequency domain dimension, thereby further optimizing the accuracy of the filter and enabling it to adapt the echo path length.

[0118] Then, the echo estimation of the re-sampled signal is performed using the updated block frequency domain filter to obtain the posterior estimated echo signal. The posterior estimated echo signal is then subtracted from the audio signal to obtain the posterior residual signal.

[0119] For example, similar to the a priori frequency domain filtering process, the following formulas 13 to 15 are used for posterior frequency domain filtering: (Formula 13) (Formula 14) (Formula 15) Take the posterior residual signal in the frequency domain The last N points are taken as the final output of the linear filtering in the k-th frame, i.e., the echo cancellation result.

[0120] For example, please see Figure 5 , Figure 5 This is a schematic diagram illustrating the effect of echo cancellation on short-path echoes, as shown in an exemplary embodiment of this application. Figure 5 As shown, the four channels represent: the original echo data, the result of short filter processing, the result of long filter processing, and the result of block frequency domain filter (belonging to the long filter) using the adaptive path length echo cancellation method of this application. It is evident that when processing short path echoes with a fixed-length filter, the short filter performs significantly better than the long filter. However, after incorporating the adaptive path length filter update mechanism of this application, the long filter essentially achieves the same effect as the short filter.

[0121] For example, please see Figure 6 , Figure 6 This is a schematic diagram illustrating the effect of echo cancellation on long-path echoes, as shown in an exemplary embodiment of this application. Figure 6 As shown, the four channels represent: the original echo data, the result of short filter processing, the result of long filter processing, and the result of block frequency domain filter (belonging to the long filter) using the adaptive path length echo cancellation method of this application. It is evident that when a fixed-length filter processes short-path echoes, the long filter performs significantly better than the short filter because the short filter cannot cover the entire echo path. Furthermore, with the addition of the adaptive path length filter update mechanism of this application, the long filter, due to its more accurate estimation of the early echo path, outperforms the traditional long filter.

[0122] This application obtains the block activation probability by inputting the prior residual signal and the back-sampled signal into a pre-trained neural network model. Based on the block activation probability, the number of effectively activated filter blocks is counted to obtain the adaptive path length. The update step size is calculated based on the adaptive path length. The update gradient is calculated based on the update step size, the back-sampled signal, and the prior residual signal, thus achieving the adaptation of the update gradient and the echo path length. Then, the block frequency domain filter is updated based on the update gradient, enabling the block frequency domain filter to simultaneously estimate the optimal echo path for both short-path and long-path echoes, improving the update accuracy of the filter, optimizing the performance of subsequent linear echo cancellation, and the overall algorithm does not add much more computational resources, thus realizing a low-complexity linear echo cancellation scheme.

[0123] Figure 7 This is a block diagram illustrating an adaptive path length echo cancellation device, as shown in an exemplary embodiment of this application. Figure 7 As shown, the exemplary adaptive path length echo cancellation device 700 includes: The prior frequency domain filtering module 710 is used to perform echo estimation on the speaker's echo signal using a block frequency domain filter to obtain a prior estimated echo signal. The prior estimated echo signal is then subtracted from the audio signal acquired by the microphone to obtain the prior residual signal. The effective probability prediction module 720 is used to input the prior residual signal and the sampled signal into a pre-trained neural network model to obtain the block effective probability output by the neural network model; wherein, the block effective probability represents the probability of effective activation of each filter block in the block frequency domain filter; The path length calculation module 730 is used to calculate the number of effectively activated filter blocks based on the block activation probability to obtain the adaptive path length; The gradient calculation module 740 is updated to calculate the update step size based on the adaptive path length, and to calculate the update gradient based on the update step size, the backsampled signal and the prior residual signal. The posterior frequency domain filtering module 750 is used to update the block frequency domain filter based on the update gradient, use the updated block frequency domain filter to perform echo estimation on the re-sampled signal to obtain the posterior estimated echo signal, and subtract the posterior estimated echo signal from the audio signal to obtain the echo-cancelled audio signal.

[0124] It should be noted that the adaptive path length echo cancellation device and the adaptive path length echo cancellation method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the adaptive path length echo cancellation device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0125] Please see Figure 8 , Figure 8 This is a schematic diagram illustrating the structure of an electronic device in an exemplary embodiment of this application. The electronic device 800 includes a memory 810 and a processor 820. The processor 820 executes program instructions stored in the memory 810 to implement the steps in any of the above-described embodiments of the adaptive path length echo cancellation method. In a specific implementation scenario, the electronic device 800 may include, but is not limited to, a microcomputer or a server. Furthermore, the electronic device 800 may also include mobile devices such as laptops and tablets, without limitation.

[0126] Specifically, processor 820 controls itself and memory 810 to implement the steps in any of the above-described adaptive path length echo cancellation method embodiments. Processor 820 may also be referred to as a Central Processing Unit (CPU). Processor 820 may be an integrated circuit chip with signal processing capabilities. Processor 820 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 820 may be implemented using integrated circuit chips.

[0127] Please see Figure 9 , Figure 9 This is a schematic diagram illustrating the structure of a computer-readable storage medium in an exemplary embodiment of this application. The computer-readable storage medium 900 stores program instructions 910 executable by a processor, the program instructions 910 being used to implement the steps in any of the above-described embodiments of the adaptive path length echo cancellation method.

[0128] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0129] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0131] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An echo cancellation method with adaptive path length, characterized in that, The method includes: The echo estimation of the loudspeaker's echo signal is performed using a block frequency domain filter to obtain a priori estimated echo signal. The priori estimated echo signal is then subtracted from the audio signal collected by the microphone to obtain a priori residual signal. The prior residual signal and the sampled signal are input into a pre-trained neural network model to obtain the block activation probability output by the neural network model; wherein, the block activation probability represents the probability of effective activation of each filter block in the block frequency domain filter; The adaptive path length is obtained by calculating the number of effectively activated filter blocks based on the block activation probability. The update step size is calculated based on the adaptive path length, and the update gradient is calculated based on the update step size, the backsampled signal, and the prior residual signal. The block frequency domain filter is updated based on the update gradient. The echo estimation of the sampled signal is performed using the updated block frequency domain filter to obtain the posterior estimated echo signal. The posterior estimated echo signal is subtracted from the audio signal to obtain the echo-cancelled audio signal.

2. The method according to claim 1, characterized in that, The process of calculating the number of effectively activated filter blocks based on the block activation probability to obtain the adaptive path length includes: The block activation probabilities corresponding to each filter block are summed and calculated. The summation result is used as the number of effectively activated filter blocks to obtain the adaptive path length.

3. The method according to claim 1, characterized in that, The neural network model also outputs a frequency domain step size, which characterizes the update intensity of the block-based frequency domain filter; the update step size is calculated based on the adaptive path length, and the update gradient is calculated based on the update step size, the sampled signal, and the prior residual signal, including: Based on the adaptive path length and the frequency domain step size, the normalized step size is calculated. The update gradient is obtained by multiplying the normalized step size, the conjugate of the frequency domain sampled signal, and the prior residual signal.

4. The method according to claim 3, characterized in that, The calculation of the normalized step size based on the adaptive path length and the frequency domain step size includes: Calculate the power spectrum of the sampled signal in the frequency domain to obtain the frequency domain sampled power spectrum; The frequency domain backsampled power spectrum and the adaptive path length are multiplied together, and the product calculation result and the frequency domain self-orthogonalization factor of the previous frame are weighted and summed to obtain the frequency domain self-orthogonalization factor of the current frame. The normalized step size is obtained by calculating the ratio between the frequency domain step size and the frequency domain self-orthogonalization factor of the current frame.

5. The method according to claim 1, characterized in that, The step of updating the block frequency domain filter based on the update gradient includes: For any filter block in the block frequency domain filter, the block activation probability and update gradient corresponding to the filter block are multiplied to calculate the gradient update amount. The filter parameters and gradient update amount of the filter block are summed to obtain the updated filter block.

6. The method according to claim 1, characterized in that, The step of updating the block frequency domain filter based on the update gradient includes: Select filter blocks from the block frequency domain filters whose block activation probability is greater than a preset probability threshold to obtain the filter blocks to be updated. Based on the update gradient corresponding to the filter block to be updated, the filter parameters of the filter block to be updated are updated to obtain the updated filter block.

7. The method according to claim 1, characterized in that, The step of inputting the prior residual signal and the sampled signal into a pre-trained neural network model to obtain the block activation probability output by the neural network model includes: The absolute values ​​of the frequency domain sampled signal and the frequency domain prior residual signal are concatenated to obtain the input features; The input features are fed into a pre-trained neural network model to obtain the frequency domain step size and block activation probability output by the neural network model.

8. The method according to claim 7, characterized in that, The neural network model includes an input linear layer, a gated recurrent unit, a step size prediction linear layer, and a block effectiveness probability prediction linear layer.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the steps of the method as described in any one of claims 1-8.

Images