Audio echo processing method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides an audio echo processing method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining a microphone voice signal corresponding to a current audio frame and a loudspeaker voice signal after delay alignment, inputting the loudspeaker voice signal into a foreground filter and a background filter which are set, obtaining a first linear echo and a second linear echo, and determining a target residual signal according to the first linear echo, the second linear echo and the microphone voice signal; updating a first filter coefficient of a background filter based on the microphone voice signal, the loudspeaker voice signal, the target residual signal and the second linear echo; and performing nonlinear echo cancellation processing on the target residual signal according to the loudspeaker voice signal to obtain a near-end voice signal. According to the scheme, the overall calculation convergence speed of the audio echo processing process is high, divergence is not prone to occurring, the acoustic echo of the audio is effectively eliminated, and the quality of the output audio is guaranteed.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio technology, and in particular to an audio echo processing method, apparatus, device, storage medium, and program product. Background Art

[0002] With the rapid development of network technology, network bandwidth and transmission speed have been greatly improved, providing a favorable network environment for real-time voice interaction. In mobile application scenarios, such as live broadcasting and game voice interaction, users are often in full-duplex communication. In the full-duplex communication process, when the user's terminal device is in the external speaker state, the sound played by the speaker of the terminal device will be collected by its microphone to form an acoustic echo, which is equivalent to the voice emitted by the user being mixed with the sound played by the speaker for transmission, affecting the communication quality.

[0003] Related technologies use adaptive filtering algorithms such as minimum mean square filtering or Kalman filtering to eliminate acoustic echoes of audio signals. However, such adaptive filtering algorithms have many artificially set empirical parameters, which make it difficult to guarantee the performance of the algorithm on large amounts of audio data. In addition, the algorithm is prone to divergence and has a slow convergence speed, making it impossible to guarantee the quality of the output audio. Summary of the invention

[0004] The embodiments of the present application provide an audio echo processing method, apparatus, device, storage medium and program product, which solve the problems that the adaptive filtering algorithm in the related art has many artificially set empirical parameters, it is difficult to guarantee the performance of the algorithm on a large amount of audio data, and it is easy to diverge, the convergence speed is slow, and the quality of the output audio cannot be guaranteed. The overall calculation of the audio echo processing process is fast in convergence speed and not easy to diverge, and the acoustic echo of the audio is effectively eliminated to ensure the quality of the output audio.

[0005] In a first aspect, an embodiment of the present application provides an audio echo processing method, the method comprising:

[0006] Acquire a microphone speech signal and a speaker speech signal after delay alignment corresponding to a current audio frame, input the speaker speech signal into a set foreground filter and a set background filter respectively, obtain a first linear echo and a second linear echo, and determine a target residual signal according to the first linear echo, the second linear echo and the microphone speech signal;

[0007] Based on the microphone speech signal, the loudspeaker speech signal, the target residual signal and the second linear echo, updating the first filter coefficient of the background filter, so that the background filter performs filtering processing on the loudspeaker speech signal corresponding to the next audio frame based on the updated first filter coefficient;

[0008] The target residual signal is subjected to nonlinear echo cancellation processing according to the loudspeaker speech signal to obtain a near-end speech signal.

[0009] In a second aspect, an embodiment of the present application further provides an audio echo processing device, the device comprising:

[0010] A residual signal determination module is configured to obtain a microphone voice signal corresponding to a current audio frame and a speaker voice signal after delay alignment, input the speaker voice signal into a set foreground filter and a set background filter respectively, obtain a first linear echo and a second linear echo, and determine a target residual signal according to the first linear echo, the second linear echo and the microphone voice signal;

[0011] a filter coefficient updating module, configured to update a first filter coefficient of the background filter based on the microphone voice signal, the loudspeaker voice signal, the target residual signal and the second linear echo, so that the background filter performs filtering processing on the loudspeaker voice signal corresponding to the next audio frame based on the updated first filter coefficient;

[0012] The near-end speech signal determination module is configured to perform nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain a near-end speech signal.

[0013] In a third aspect, an embodiment of the present application further provides an audio echo processing device, the device comprising:

[0014] one or more processors;

[0015] a storage device configured to store one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the audio echo processing method described in the embodiment of the present application.

[0017] In a fourth aspect, an embodiment of the present application further provides a non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions are configured to execute the audio echo processing method described in the embodiment of the present application when executed by a computer processor.

[0018] In a fifth aspect, an embodiment of the present application further provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the audio echo processing method described in the embodiment of the present application.

[0019] In an embodiment of the present application, by obtaining a microphone voice signal corresponding to a current audio frame and a speaker voice signal after delay alignment, the speaker voice signal is respectively input into a set foreground filter and a background filter to obtain a first linear echo and a second linear echo, and a target residual signal is determined according to the first linear echo, the second linear echo and the microphone voice signal; based on the microphone voice signal, the speaker voice signal, the target residual signal and the second linear echo, the first filter coefficient of the background filter is updated, and the background filter is used to filter the speaker voice signal corresponding to the next audio frame based on the updated first filter coefficient; and nonlinear echo cancellation processing is performed on the target residual signal according to the speaker voice signal to obtain a near-end voice signal. In the above scheme, by inputting the speaker voice signal into the set foreground filter and background filter, the linear component of the acoustic echo can be effectively estimated to obtain the first linear echo and the second linear echo. By determining the target residual signal according to the first linear echo, the second linear echo and the microphone voice signal, the linear component of the acoustic echo in the microphone voice signal can be effectively eliminated to obtain the target residual signal. By updating the first filter coefficient of the background filter frame by frame, the echo path change can be dynamically tracked to achieve a better effect of eliminating the acoustic echo. By performing nonlinear echo elimination processing on the target residual signal according to the speaker voice signal to obtain the near-end voice signal, the nonlinear component in the target residual signal can be further eliminated to achieve a better echo removal effect. The overall calculation of the audio echo processing process has a fast convergence speed and is not easy to diverge, which can ensure the quality of the output audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of an audio echo processing method provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of an audio echo processing method including a process of updating a first filter coefficient of a background filter provided by an embodiment of the present application;

[0022] Figure 3 A flowchart of an audio echo processing method including a process of determining target filter coefficients of odd-numbered audio frames provided by an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the structure of a neural network model for calculating the first step length information and the first threshold information provided in an embodiment of the present application;

[0024] Figure 5 A flowchart of an audio echo processing method including a process of determining a target filter coefficient of an even-numbered audio frame provided by an embodiment of the present application;

[0025] Figure 6A flowchart of an audio echo processing method including a process of determining a target residual signal provided by an embodiment of the present application;

[0026] Figure 7 A flowchart of an audio echo processing method including a process of adjusting a second filter coefficient of a foreground filter provided by an embodiment of the present application;

[0027] Figure 8 A flowchart of an audio echo processing method including a process of adjusting a first filter coefficient of a background filter provided by an embodiment of the present application;

[0028] Fig. 9 A schematic diagram of a process of performing audio processing by applying an audio echo processing method provided in an embodiment of the present application;

[0029] Fig.10 A structural block diagram of an audio echo processing device provided in an embodiment of the present application;

[0030] Fig.11 A schematic diagram of the structure of an audio echo processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the embodiments of the present application, rather than to limit the embodiments of the present application. It should also be noted that, for ease of description, only parts related to the embodiments of the present application are shown in the accompanying drawings, rather than all structures.

[0032] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0033] The audio echo processing method provided in the embodiment of the present application can be used to eliminate the acoustic echo of the microphone voice signal and improve the communication quality. Relevant application scenarios may include: live broadcasting, video calls, game voice, etc. The several application scenarios listed above are only exemplary and explanatory. In actual applications, the audio echo processing method can also be used in audio processing in other scenarios, and the embodiment of the present application does not limit this. The audio echo processing method, device, equipment, storage medium and program product provided in the embodiment of the present application are intended to solve the problem that the adaptive filtering algorithm of the related technology has many artificially set empirical parameters, it is difficult to guarantee the performance of the algorithm on a large amount of audio data, and it is easy to diverge, converge slowly, and cannot guarantee the quality of the output audio.

[0034] In the audio echo processing method provided in the embodiment of the present application, the execution subject of each step can be a computer device, which refers to any electronic device with data calculation, processing and storage capabilities, such as mobile phones, PCs (Personal Computers), tablet computers and other terminal devices, which are not limited in the embodiment of the present application.

[0035] Figure 1 A flowchart of an audio echo processing method provided in an embodiment of the present application. The audio echo processing method is applied to a voice interaction scenario in which the user's terminal device is in an external speaker state, such as Figure 1 As shown, the following steps are included:

[0036] Step S101, obtain the microphone voice signal corresponding to the current audio frame and the speaker voice signal after delay alignment, input the speaker voice signal into the set foreground filter and background filter respectively, obtain the first linear echo and the second linear echo, and determine the target residual signal according to the first linear echo, the second linear echo and the microphone voice signal.

[0037] Among them, in the audio echo processing process, the audio signal to be processed can be divided into audio frames of preset time length for frame-by-frame processing. For example, the preset time length is 10ms, 20ms, etc. The specific preset length can be adaptively set by the developer according to the audio characteristics and accuracy requirements of the actual application scenario, and this application does not limit it. In this embodiment, each audio frame can correspond to the acquisition of the microphone voice signal and the speaker voice signal after delay alignment. Among them, the microphone voice signal is a frequency domain signal, which can be obtained by discrete Fourier transforming the microphone recording time domain signal, and the microphone recording time domain signal is specifically mixed with the user's near-end voice time domain signal and the speaker's far-end voice time domain signal. In addition, the speaker voice signal is a frequency domain signal, which can be obtained by discrete Fourier transforming the speaker's time domain signal to be played, and the time domain signal to be played is the opposite-end voice signal that the current terminal device receives and needs to play. Due to reasons such as playback buffering and the physical distance between the speaker and the microphone, there will be a delay from the speaker playback to the microphone recording, and the time domain signal to be played is obtained in advance. Therefore, it is necessary to delay the speaker voice signal obtained by frequency domain conversion processing based on the time domain signal to be played, so that the speaker voice signal and the microphone voice signal are aligned on the time axis, ensuring the true reference of the speaker voice signal, and better realizing the acoustic echo elimination of the microphone voice signal. In one embodiment, the speaker voice signal is delayed and estimated, and the autocorrelation algorithm can be used to match the peak points of the speaker voice signal and the microphone voice signal to determine the delay time point. In one embodiment, a fixed delay value can be calculated according to the playback buffer time and the physical distance between the speaker and the microphone signal, and the speaker voice signal is delayed based on the fixed delay value. After obtaining the microphone voice signal corresponding to the current audio frame and the speaker voice signal after delay alignment, the speaker voice signal can be input into the set foreground filter and background filter respectively. Among them, the foreground filter and the background filter are specifically used to estimate the linear echo, and the foreground filter can be used as a coefficient backup of the background filter to reduce the double-talk divergence of the filter.

[0038] In one embodiment, the calculation formulas for the foreground filter and the background filter are as follows:

[0039]

[0040]

[0041] Among them, Y est1(l,k) is the first linear echo, X align(l-i,k) is the speaker speech signal after delay alignment, which is a frequency domain signal in complex form, Hf est1(i,k) is the foreground filter coefficient, Yest(l,k) is the second linear echo, Hf est(i,k) is the background filter coefficient, l is the frame index, k is the frequency band index, and L is the filter order. It should be noted that the above formula calculates the linear convolution in the complex domain. For example, the frame length corresponding to the speaker voice signal obtained by discrete Fourier transform of the time domain signal to be played from the speaker is set to 512 points, and the frame shift is 128 points. Then the effective frequency band is 257 points, and the value range of k is 0-256. For another example, if the filter order L of each frequency band is set to 6, the delay range that can be covered is 48ms.

[0042] Among them, the target residual signal is the processing result obtained after eliminating the linear echo of the microphone voice signal. In one embodiment, the first linear echo and the second linear echo can be respectively subtracted from the microphone voice signal to obtain the first residual signal and the second residual signal, and the first residual signal and the second residual signal are compared in terms of modulus, and the residual signal that satisfies the minimum modulus is determined as the target residual signal, wherein the smaller the modulus of the residual signal, the smaller the residual degree of the linear echo. In one embodiment, the first linear echo and the second linear echo can be respectively subtracted from the microphone voice signal to obtain the first residual signal and the second residual signal, and the modulus of the first residual signal and the second residual signal are respectively calculated and compared with the preset modulus threshold. When both the first residual signal and the second residual signal need to exceed the preset modulus threshold, the residual signal that satisfies the minimum modulus in the first residual signal and the second residual signal can be adjusted by a preset amplitude to obtain the target residual signal. When the first residual signal and / or the second residual signal do not exceed the preset modulus threshold, the residual signal that satisfies the minimum modulus is determined as the target residual signal.

[0043] Step S102: based on the microphone speech signal, the loudspeaker speech signal, the target residual signal and the second linear echo, update the first filter coefficient of the background filter, so that the background filter performs filtering processing on the loudspeaker speech signal corresponding to the next audio frame based on the updated first filter coefficient.

[0044] Among them, since the foreground filter is used as a coefficient backup for the background filter, it does not need to be updated frame by frame, while the background filter can dynamically update the coefficient frame by frame to better track the echo path changes and achieve a better echo cancellation effect. Specifically, the microphone voice signal, the speaker voice signal, the target residual signal and the second linear echo can be used as input information, and the first filter coefficient of the background filter can be updated using the least mean square algorithm, the recursive least squares algorithm, the proportional normalized least mean square algorithm, etc. In one embodiment, the proportional normalized least mean square algorithm can be used, and the relevant calculation formula is as follows:

[0045] absW(i,k)=abs(Hf est(i,k))i=0,...,L-1,

[0046] Wmax(k)=max(absW(i,k),0.01)*thre(k),

[0047]

[0048]

[0049] dHf(i,k)=conj(X align(l-i,k) )*(D(k)-Y est(k) )*g(i,k),

[0050] Hf′ est(i,k) =Hf est(i,k) +stepSize(k)*dHf(i,k),

[0051] Where abs(*) is the modulus value, Hf est(i,k) is the first filter coefficient of the background filter, max(*) is the maximum value, thre(k) is the input threshold, g(i,k) is the algorithm scale factor, stepSize(k) is the target step size, mu(k) is the input step size, dHf(i,k) is the error of the background filter, conj(*) is the calculated conjugate, X align(l-i,k) is the speaker speech signal after delay alignment, D(k) is the microphone speech signal, and Y est(k) is the second linear echo, Hf′ est(i,k) is the first updated filter coefficient, l is the frame index, k is the band index, L is the filter order, c 1 is a constant coefficient, for example, c 1 =1×e -3. In one embodiment, the microphone voice signal, the target residual signal and the preset step length formula can be used to calculate the aforementioned input step length mu(k). The specific preset step length formula can be obtained by the developer through adaptive design based on empirical data, and the input threshold thre(k) can be set as a constant threshold. In one embodiment, a neural network model for calculating the input step length and the input threshold can be provided, and the model input feature calculation is performed based on the speaker voice signal, the microphone voice signal, the target linear echo, the target residual signal and the first filter coefficient of the background filter, and the model input feature is input to the neural network model to output the input step length and the input threshold. In one embodiment, during the continuous audio frame processing, the preset step length formula and the neural network model are alternately used to calculate the input step length and the input threshold according to the preset rules. The specific preset rules can be to use different calculation methods for audio frames of different odd and even frame types, to determine the corresponding calculation method according to the modulo result of the frame number of the audio frame, etc., which are not limited in this application. It can be understood that the background filter can update the first filter coefficient in combination with the target residual signal corresponding to the current audio frame, and filter the speaker speech signal corresponding to the next audio frame based on the updated first filter coefficient, so as to better adapt to the change of the echo path and perform effective echo elimination.

[0052] Step S103: performing nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain a near-end speech signal.

[0053] Among them, the target residual signal is the processing result obtained after eliminating the linear echo of the microphone voice signal. However, in the actual voice interaction scenario, the physical characteristics of the acoustic device itself may have nonlinear distortion, or the sound may undergo complex physical phenomena such as scattering, diffraction and absorption, etc., and nonlinear echo may be introduced into the microphone voice signal. Therefore, it is necessary to further perform nonlinear echo elimination processing on the target residual signal according to the speaker voice signal to obtain a near-end voice signal. The near-end voice signal retains the audio information that the user expects to transmit to the other end through the microphone, and eliminates the influence of the far-end voice signal of the speaker. In one embodiment, the Volterra series can be used for nonlinear echo elimination. By estimating the coefficients of the Volterra series, the characteristics of the nonlinear echo path can be approximated and compensated. The specific implementation process can be: determine the order and memory length of the Volterra series according to the echo environment characteristics of the actual application scenario, use the microphone voice signal and the speaker voice signal, and estimate the coefficients of the Volterra series by the least squares method or other optimization algorithms, and finally calculate the estimated value of the nonlinear echo according to the estimated coefficients, subtract the estimated value from the microphone voice signal, and achieve the elimination of the nonlinear echo. In one embodiment, a neural network model can be used to model the nonlinear echo path between the microphone voice signal and the loudspeaker voice signal, so as to achieve the purpose of eliminating nonlinear echo. The specific implementation process can be: constructing a neural network model using a multi-layer perceptron or a recursive neural network, and determining the structure and parameters of the neural network model. The neural network model is trained based on microphone voice sample data, loudspeaker voice sample data and near-end voice label data, and the network weights of the neural network model are adjusted by using a back propagation algorithm until the loss function converges. The microphone voice signal and the loudspeaker voice signal are input into the trained neural network model to obtain a near-end voice signal.

[0054] In the above, by obtaining the microphone voice signal corresponding to the current audio frame and the speaker voice signal after delay alignment, the speaker voice signal is respectively input into the set foreground filter and background filter to obtain the first linear echo and the second linear echo, and the target residual signal is determined according to the first linear echo, the second linear echo and the microphone voice signal; based on the microphone voice signal, the speaker voice signal, the target residual signal and the second linear echo, the first filter coefficient of the background filter is updated, and the background filter is used to filter the speaker voice signal corresponding to the next audio frame based on the updated first filter coefficient; according to the speaker voice signal, the target residual signal is subjected to nonlinear echo cancellation processing to obtain the near-end voice signal. In the above scheme, by inputting the speaker voice signal into the set foreground filter and background filter, the linear component of the acoustic echo can be effectively estimated to obtain the first linear echo and the second linear echo. By determining the target residual signal according to the first linear echo, the second linear echo and the microphone voice signal, the linear component of the acoustic echo in the microphone voice signal can be effectively eliminated to obtain the target residual signal. By updating the first filter coefficient of the background filter frame by frame, the echo path change can be dynamically tracked to achieve a better effect of eliminating the acoustic echo. By performing nonlinear echo elimination processing on the target residual signal according to the speaker voice signal to obtain the near-end voice signal, the nonlinear component in the target residual signal can be further eliminated to achieve a better echo removal effect. The overall calculation of the audio echo processing process has a fast convergence speed and is not easy to diverge, which can ensure the quality of the output audio.

[0055] Figure 2 The present invention provides a flowchart of an audio echo processing method including a process of updating the first filter coefficient of the background filter. Figure 2 As shown, the following steps are included:

[0056] Step S201, obtain the microphone voice signal corresponding to the current audio frame and the speaker voice signal after delay alignment, input the speaker voice signal into the set foreground filter and background filter respectively, obtain the first linear echo and the second linear echo, and determine the target residual signal according to the first linear echo, the second linear echo and the microphone voice signal.

[0057] Step S202: Subtract the microphone speech signal from the target residual signal to obtain a target linear echo.

[0058] Among them, since the target residual signal is the processing result obtained after eliminating the linear echo of the microphone speech signal, the microphone speech signal and the target residual signal can be subtracted to obtain the final estimated target linear echo.

[0059] Step S203, determining the target filter coefficient according to the odd and even frame type of the current audio frame, the target linear echo, the microphone voice signal, the speaker voice signal, the target residual signal, the second linear echo and the set neural network model.

[0060] Among them, in this embodiment, a proportional normalized least mean square algorithm that needs to determine the input step size and input threshold is adopted. Since the calculation amount of the input step size is small only by using the preset step size formula, it is easy to diverge and converge slowly. Therefore, a neural network model can be added to participate in the calculation of the input step size and input threshold. On the basis of maintaining a reasonable amount of calculation, it has the characteristics of fast convergence speed and not easy to diverge of the neural network model, meeting the deployment requirements of the mobile terminal. Specifically, the preset step size formula and the neural network module can be used alternately according to the odd and even frame type of the current audio frame to calculate the input step size and input threshold. In one embodiment, if the current audio frame is an odd frame, the model input feature can be calculated based on the speaker voice signal, the microphone voice signal, the target linear echo, the target residual signal and the first filter coefficient of the background filter, and then the model input feature is input to the neural network model output input step size and input threshold. Finally, the microphone voice signal, the speaker voice signal, the target residual signal, the second linear echo, the input step size and the input threshold are used as the input information of the proportional normalized least mean square algorithm to calculate the target filter coefficient. In one embodiment, if the current audio frame is an even frame, the target linear echo and the target residual signal can be substituted into the preset step size formula to calculate the input step size, and the first filter coefficient of the background filter, the preset constant threshold, the input step size information, the speaker voice signal, the microphone voice signal, and the second linear echo are used as the input information of the proportional normalized least mean square algorithm to calculate the target filter coefficient. Of course, the aforementioned specific odd and even frames use the method of alternating the input step size and input threshold calculation only for exemplary descriptions, and developers can make adaptive adjustments based on the computing resource limitations and data convergence characteristics of the actual application scenario, which is not limited in this application.

[0061] Step S204: Update the first filter coefficient of the background filter to the target filter coefficient.

[0062] By updating the first filter coefficient of the background filter to the target filter coefficient, the first filter coefficient of the background filter can be dynamically updated frame by frame.

[0063] Step S205: Perform nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain a near-end speech signal.

[0064] As mentioned above, by adding the odd and even frame types of the current audio frame as reference information for specifically adjusting the target filter coefficients, and combining the calculation effect of the set neural network model, different calculation approaches can be combined to determine the target filter coefficients, thereby ensuring the calculation effect while increasing the robustness of the algorithm and reducing the amount of calculation.

[0065] Figure 3 The present invention provides a flowchart of an audio echo processing method including a process of determining a target filter coefficient of an odd-numbered audio frame. Figure 3 As shown, the following steps are included:

[0066] Step S301, obtain the microphone voice signal corresponding to the current audio frame and the speaker voice signal after delay alignment, input the speaker voice signal into the set foreground filter and background filter respectively, obtain the first linear echo and the second linear echo, and determine the target residual signal according to the first linear echo, the second linear echo and the microphone voice signal.

[0067] Step S302: Subtract the microphone speech signal from the target residual signal to obtain a target linear echo.

[0068] Step S303: when the current audio frame is an odd-numbered frame, a preset number of input feature values ​​are calculated according to the speaker speech signal, the microphone speech signal, the target linear echo, the target residual signal and the first filter coefficient of the background filter.

[0069] The specific calculation formula for calculating a preset number of input feature values ​​is as follows:

[0070] feat0=abs(X align(l,k) ),

[0071] feat1=abs(D(l,k)),

[0072] feat2=abs(Yo est(l,k) ),

[0073] feat3=abs(Eout(l,k)),

[0074] feat4=min(c 2 ,c 3 *feat2 / (feat3+c 4 )),

[0075] feat5~feat10=abs(Hf est(0,k) )~abs(Hf est(L-1,k) ),

[0076] The calculation of the input eigenvalue is performed according to the frequency band dimension. For example, if the frame length corresponding to the speaker voice signal obtained by discrete Fourier transform of the time domain signal to be played from the speaker is 512 points and the frame shift is 128 points, then the effective frequency band is 257 points. If the filter order is 6, that is, L=6, 11 eigenvalues ​​feat0~feat10 are calculated for each frequency band. abs(*) is the modulus value, X align(l,k) is the speaker speech signal after delay alignment, D(l,k) is the microphone speech signal, Yo est(l,k) is the target linear echo, Eout(l,k) is the target residual signal, c 2 、c 3 and c 4 is a constant coefficient, for example, c 2 =0.8, c 3 = 0.2 and c 4 =1×e -8 , Hf est(0,k) ~Hf est(L-1,k) Of course, developers can also select different frame lengths, frame shifts, filter orders and empirical data to make adaptive adjustments to the calculation formula, which is not limited in this application.

[0077] Step S304: input multiple input feature values ​​into the neural network model to obtain the first step length information and the first threshold information.

[0078] Among them, the neural network model can adopt a self-encoding structure. Figure 4 Schematic diagram of the structure of a neural network model for calculating the first step length information and the first threshold information provided in an embodiment of the present application, such as Figure 4 As shown, the neural network model includes a first linear layer, a first activation function layer, a timing modeling layer, a second linear layer, a second activation function layer, a third linear layer and a third activation function layer. Among them, the first linear layer is used to receive multiple input feature values, the first linear layer, the first activation function layer and the timing modeling layer are connected in sequence, the second linear layer and the third linear layer are connected to the timing modeling layer respectively, the second activation function layer is connected to the second linear layer, and the third activation function layer is connected to the third linear layer. The second activation function layer is used to output the first step length information, and the third activation function layer is used to output the first threshold information. Optionally, in order to reduce the amount of calculation introduced by the neural network model, the number of units in the first linear layer used for encoding can be set to 4, the number of units in the timing modeling layer can be set to 4, the number of layers can be set to 2, the timing modeling layer can be a gated recurrent unit layer, and the number of units in the second linear layer and the third linear layer used for decoding is set to 1. Of course, developers can also adjust the model structure according to the computing power limitations and model performance requirements of the actual application scenario, which is not limited in this application.

[0079] Step S305: determining a target filter coefficient according to the first filter coefficient of the background filter, the first threshold information, the first step length information, the speaker voice signal, the microphone voice signal and the second linear echo.

[0080] Among them, the proportional normalized least mean square algorithm is adopted in this embodiment, the first threshold information can be used as the input threshold of the algorithm, and the first step length information can be used as the input step length of the algorithm. The relevant formula for calculating the target filter coefficient has been shown in the aforementioned embodiment and will not be repeated in this application.

[0081] Step S306: Update the first filter coefficient of the background filter to the target filter coefficient.

[0082] Step S307: Perform nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain a near-end speech signal.

[0083] As mentioned above, by calculating and obtaining a preset number of multiple input feature values, sufficient feature information can be provided to the neural network model. When the current audio frame is an odd frame, the first step length information and the first threshold information are calculated by the neural network model, which can effectively perform step length prediction and threshold prediction of related algorithms for determining target filter coefficients. On the basis of ensuring a reasonable amount of calculation, the calculation converges quickly and is not easy to diverge.

[0084] Figure 5 A flowchart of an audio echo processing method including a process of determining target filter coefficients of even audio frames provided by an embodiment of the present application. Figure 5 As shown, the following steps are included:

[0085] Step 401, obtain the microphone speech signal corresponding to the current audio frame and the speaker speech signal after delay alignment, input the speaker speech signal into the set foreground filter and background filter respectively, obtain the first linear echo and the second linear echo, and determine the target residual signal according to the first linear echo, the second linear echo and the microphone speech signal.

[0086] Step S402: Subtract the microphone speech signal from the target residual signal to obtain a target linear echo.

[0087] Step S403: when the current audio frame is an odd-numbered frame, a preset number of input feature values ​​are calculated according to the speaker speech signal, the microphone speech signal, the target linear echo, the target residual signal and the first filter coefficient of the background filter.

[0088] Step S404: input multiple input feature values ​​into the neural network model to obtain the first step length information and the first threshold information.

[0089] Step S405: determining a target filter coefficient according to the first filter coefficient of the background filter, the first threshold information, the first step length information, the speaker voice signal, the microphone voice signal and the second linear echo.

[0090] Step S406: When the current audio frame is an even frame, obtain second step length information according to the target linear echo and the target residual signal.

[0091] Among them, the relevant formula for calculating the second step information is as follows:

[0092] feat4=min(c 5 ,c 6 *abs(Yo est(l,k) ) / (abs(Eout(l,k))+c 7 )),

[0093] Among them, Yo est(l,k) is the target linear echo, Eout(l,k) is the target residual signal, c 5 、c 6 and c 7 is a constant coefficient, for example, c 2 =0.5, c 3 = 0.1 and c 4 =1×e -8 Of course, developers can also make adaptive selections of different constant coefficients based on the specific effects of the application in determining the target filter coefficients, which is not limited in this application.

[0094] Step S407: determining a target filter coefficient according to the first filter coefficient of the background filter, the preset constant threshold, the second step length information, the speaker voice signal, the microphone voice signal and the second linear echo.

[0095] Among them, the proportional normalized least mean square algorithm is adopted in this embodiment, and the preset constant threshold can be used as the input threshold of the algorithm. For example, the preset constant threshold is 0.01, and the second step information can be used as the input step of the algorithm. The relevant formula for calculating the target filter coefficient has been shown in the aforementioned embodiment and will not be repeated in this application.

[0096] Step S408: Update the first filter coefficient of the background filter to the target filter coefficient.

[0097] Step S409: Perform nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain a near-end speech signal.

[0098] As mentioned above, when the current audio frame is an even frame, the preset step length formula is used to calculate the second step length information, which is reasonably combined with the method of using a neural network model to calculate the step length and threshold when the current audio frame is an odd frame, so as to ensure the robustness of the overall calculation to determine the target filter coefficient and appropriately reduce the amount of calculation.

[0099] Figure 6 A flowchart of an audio echo processing method including a process of determining a target residual signal is provided in an embodiment of the present application. Figure 6 As shown, the following steps are included:

[0100] Step S501: Acquire a microphone voice signal corresponding to a current audio frame and a speaker voice signal after delay alignment, and input the speaker voice signal into a set foreground filter and a set background filter respectively to obtain a first linear echo and a second linear echo.

[0101] Step S502: subtract the microphone speech signal from the first linear echo to obtain a first residual signal, subtract the microphone speech signal from the second linear echo to obtain a second residual signal, and determine the residual signal that satisfies the minimum modulus value between the first residual signal and the second residual signal as the target residual signal.

[0102] Among them, since the calculated first residual signal and the second residual signal are in complex form, the size of their modulus can directly reflect the amplitude characteristics of the residual signal. If the modulus of the residual signal is large, it can be regarded as the presence of a strong echo or interference that has not been eliminated. If the modulus of the residual signal is small, it can be regarded as a good echo elimination effect, and the residual signal is closer to the target speech signal. Therefore, the residual signal that satisfies the smallest modulus value in the first residual signal and the second residual signal can be selected as the target residual signal.

[0103] Step S503: based on the microphone speech signal, the loudspeaker speech signal, the target residual signal and the second linear echo, update the first filter coefficient of the background filter, so that the background filter performs filtering processing on the loudspeaker speech signal corresponding to the next audio frame based on the updated first filter coefficient.

[0104] Step S504: Perform nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain a near-end speech signal.

[0105] In the above, the residual signal that satisfies the minimum modulus value between the first residual signal and the second residual signal is determined as the target residual signal, avoiding the selection of the calculation result that may diverge the linear echo, ensuring the accuracy of the residual signal estimation, and being beneficial to providing accurate basic data for updating the first filter coefficient of the background filter and further calculating the near-end speech signal.

[0106] Figure 7The present invention provides a flowchart of an audio echo processing method including a process of adjusting the second filter coefficient of a foreground filter. Figure 7 As shown, the following steps are included:

[0107] Step S601: Acquire a microphone voice signal corresponding to a current audio frame and a speaker voice signal after delay alignment, and input the speaker voice signal into a set foreground filter and background filter respectively to obtain a first linear echo and a second linear echo.

[0108] Step S602: subtract the microphone speech signal from the first linear echo to obtain a first residual signal, subtract the microphone speech signal from the second linear echo to obtain a second residual signal, and determine the residual signal that satisfies the minimum modulus value between the first residual signal and the second residual signal as the target residual signal.

[0109] Step S603: obtain a signal smoothing value based on the microphone speech signal and the historical signal smoothing value, obtain a first residual signal smoothing value based on the first residual signal and the first historical residual signal smoothing value, and obtain a second residual signal smoothing value based on the second residual signal and the second historical residual signal smoothing value.

[0110] Among them, the relevant formula for calculating the signal smoothing value is as follows:

[0111] MSEmic(l,k)=α 1 *MSEmic(l-1,k)+(1-α 1 )*abs(D(l,k)) 2 ,

[0112] Where l is the frame index, k is the frequency band index, MSEmic(l,k) is the signal smoothing value, MSEmic(l-1,k) is the historical signal smoothing value, abs(*) is the modulus value, D(l,k) is the microphone speech signal, α 1 is the smoothing coefficient, for example, α 1 =0.97.

[0113] The relevant formula for calculating the smoothed value of the first residual signal is as follows:

[0114] MSEmain(l,k)=α 2 *MSEmain(l-1,k)+(1-α 2 )*abs(Eout1(l,k)) 2 ,

[0115] Wherein, l is the frame index, k is the frequency band index, MSEmain(l,k) is the smoothed value of the first residual signal, MSEmain(l-1,l) is the smoothed value of the first historical residual signal, abs(*) is the modulus value, Eout1(l,k) is the first residual signal, α 2 is the smoothing coefficient, for example, α 2 =0.97.

[0116] The relevant formula for calculating the smoothed value of the second residual signal is as follows:

[0117] MSEadapt(l,k)=α 3 *iMSEadapt(l-1,k)+(1-α 3 )*abs(Eout0(l,k)) 2 ,

[0118] Wherein, l is the frame index, k is the frequency band index, MSEadapt(l,k) is the second residual signal smoothing value, MSEadapt(l-1,k) is the second historical residual signal smoothing value, abs(*) is the modulus value, Eout0(l,k) is the second residual signal, α 3 is the smoothing coefficient, for example, α 3 =0.97.

[0119] Step S604: when the signal smoothing value is greater than the product of the second residual signal smoothing value and the first preset coefficient, and the second residual signal smoothing value is less than the product of the first residual signal smoothing value and the second preset coefficient, replace the second filter coefficient of the foreground filter with the first filter coefficient of the background filter.

[0120] Among them, the first residual signal smoothing value is calculated based on the first linear echo output by the foreground filter, and the second residual signal smoothing value is calculated based on the first linear echo output by the background filter. The first preset coefficient is greater than 1, for example, the first preset coefficient is 6. The second preset coefficient is less than 1, for example, the second preset coefficient is 0.5. If the signal smoothing value is greater than the product of the first residual signal smoothing value and the first preset coefficient, and the first residual signal smoothing value is less than the product of the second residual signal smoothing value and the second preset coefficient, it can be regarded that the output result of the background filter is relatively stable and the output result of the foreground filter is relatively divergent. The second filter coefficient of the foreground filter can be replaced by the first filter coefficient of the background filter to avoid the foreground filter losing the ability to effectively filter the input speaker voice signal.

[0121] Step S605: based on the microphone speech signal, the loudspeaker speech signal, the target residual signal and the second linear echo, update the first filter coefficient of the background filter, so that the background filter performs filtering processing on the loudspeaker speech signal corresponding to the next audio frame based on the updated first filter coefficient.

[0122] Step S606: Perform nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain a near-end speech signal.

[0123] In the above, by replacing the second filter coefficient of the foreground filter with the first filter coefficient of the background filter when the signal smoothing value is greater than the product of the second residual signal smoothing value and the first preset coefficient, and the second residual signal smoothing value is less than the product of the first residual signal smoothing value and the second preset coefficient, it is beneficial to ensure the filtering performance of the foreground filter and ensure the accuracy and reliability of the output first linear echo.

[0124] Figure 8 The present invention provides a flowchart of an audio echo processing method including a process of adjusting the first filter coefficient of a background filter. Figure 8 As shown, the following steps are included:

[0125] Step S701, obtaining a microphone voice signal corresponding to a current audio frame and a speaker voice signal after delay alignment, and inputting the speaker voice signal into a set foreground filter and a set background filter respectively to obtain a first linear echo and a second linear echo.

[0126] Step S702: subtract the microphone speech signal from the first linear echo to obtain a first residual signal, subtract the microphone speech signal from the second linear echo to obtain a second residual signal, and determine the residual signal that satisfies the minimum modulus value between the first residual signal and the second residual signal as the target residual signal.

[0127] Step S703: obtain a signal smoothing value based on the microphone speech signal and the historical signal smoothing value, obtain a first residual signal smoothing value based on the first residual signal and the first historical residual signal smoothing value, and obtain a second residual signal smoothing value based on the second residual signal and the second historical residual signal smoothing value.

[0128] Step S704: when the signal smoothing value is greater than the product of the second residual signal smoothing value and the first preset coefficient, and the second residual signal smoothing value is less than the product of the first residual signal smoothing value and the second preset coefficient, replace the second filter coefficient of the foreground filter with the first filter coefficient of the background filter.

[0129] Step S705: When the signal smoothing value is greater than the product of the first residual signal smoothing value and the third preset coefficient, and the first residual signal smoothing value is less than the product of the second residual signal smoothing value and the fourth preset coefficient, replace the first filter coefficient of the background filter with the second filter coefficient of the foreground filter.

[0130] Among them, the third preset coefficient is greater than 1, for example, the first preset coefficient is 8. The fourth preset coefficient is less than 1, for example, the fourth preset coefficient is 0.5. If the signal smoothing value is greater than the product of the first residual signal smoothing value and the third preset coefficient, and the first residual signal smoothing value is less than the product of the second residual signal smoothing value and the fourth preset coefficient, it can be regarded that the output result of the foreground filter is relatively stable, and the output result of the background filter is relatively divergent. The first filter coefficient of the background filter can be replaced by the second filter coefficient of the foreground filter to avoid the background filter losing the ability to effectively filter the input speaker voice signal.

[0131] Step S706: based on the microphone speech signal, the loudspeaker speech signal, the target residual signal and the second linear echo, update the first filter coefficient of the background filter, so that the background filter performs filtering processing on the loudspeaker speech signal corresponding to the next audio frame based on the updated first filter coefficient.

[0132] Step S707: Perform nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain a near-end speech signal.

[0133] In the above, by replacing the first filter coefficient of the background filter with the second filter coefficient of the foreground filter when the signal smoothing value is greater than the product of the first residual signal smoothing value and the third preset coefficient, and the first residual signal smoothing value is less than the product of the second residual signal smoothing value and the fourth preset coefficient, it is helpful to ensure the filtering performance of the background filter and ensure the accuracy and reliability of the output second linear echo.

[0134] Fig. 9 A schematic diagram of a process of performing audio processing using an audio echo processing method provided in an embodiment of the present application. Fig. 9As shown, after the foreground filter 101 receives and processes the speaker voice signal, it outputs the first linear echo to the signal processing module 103. After the background filter 102 receives and processes the speaker voice signal, it outputs the second linear echo to the signal processing module 103 and the filter coefficient updating module 104. After the signal processing module 103 receives and processes the first linear echo, the second linear echo and the microphone voice signal, it outputs the target residual signal and the target linear echo to the filter coefficient updating module 104. The filter coefficient updating module 104 includes a feature extraction module 105, a neural network model 106, a parameter calculation module 107 and a calculation updating module 108. When the current audio frame is an odd frame, the feature extraction module 105 processes the speaker voice signal, the microphone voice signal, the target linear echo, the target residual signal and the first filter coefficient of the current background filter and outputs multiple input feature values ​​to the neural network model 106. The neural network model 106 processes the received multiple input feature values ​​and outputs the first step length information and the first threshold information to the calculation updating module 108. The calculation updating module 108 determines the target filter coefficient based on the first filter coefficient, the first threshold information, the first step length information, the speaker voice signal, the microphone voice signal and the second linear echo of the background filter 102, and updates the first filter coefficient of the background filter 102 to the target filter coefficient. In the case where the current audio frame is an even frame, the parameter calculation module 107 processes the target linear echo and the target residual signal and outputs the second step size information and the preset constant threshold to the calculation update module 108. The calculation update module 108 determines the target filter coefficient based on the first filter coefficient of the background filter 102, the preset constant threshold, the second step size information, the speaker voice signal, the microphone voice signal and the second linear echo, and updates the first filter coefficient of the background filter 102 to the target filter coefficient. In addition, when the foreground filter 101 and the background filter 102 meet a specific threshold condition, the second filter coefficient of the foreground filter 101 can be replaced by the first filter coefficient of the background filter 102, or the first filter coefficient of the background filter 102 can be replaced by the second filter coefficient of the foreground filter 101.

[0135] As described above, by applying the audio echo processing method provided in the embodiment of the present application to perform real-time processing of audio signals in a voice interaction scenario, acoustic echo cancellation can be achieved, audio quality can be improved, and user voice interaction experience can be guaranteed.

[0136] Fig.10 This is a structural block diagram of an audio echo processing device provided in an embodiment of the present application. The device is configured to execute the audio echo processing method provided in the above embodiment and has the corresponding functional modules and beneficial effects of the execution method. Fig.10 As shown, the device comprises:

[0137] The residual signal determination module 201 is configured to obtain the microphone speech signal corresponding to the current audio frame and the speaker speech signal after delay alignment, input the speaker speech signal to the set foreground filter and background filter respectively, obtain the first linear echo and the second linear echo, and determine the target residual signal according to the first linear echo, the second linear echo and the microphone speech signal;

[0138] The filter coefficient updating module 202 is configured to update the first filter coefficient of the background filter based on the microphone speech signal, the loudspeaker speech signal, the target residual signal and the second linear echo, so that the background filter performs filtering processing on the loudspeaker speech signal corresponding to the next audio frame based on the updated first filter coefficient;

[0139] The near-end speech signal determination module 203 is configured to perform nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain the near-end speech signal.

[0140] In the above, by obtaining the microphone voice signal corresponding to the current audio frame and the speaker voice signal after delay alignment, the speaker voice signal is respectively input into the set foreground filter and background filter to obtain the first linear echo and the second linear echo, and the target residual signal is determined according to the first linear echo, the second linear echo and the microphone voice signal; based on the microphone voice signal, the speaker voice signal, the target residual signal and the second linear echo, the first filter coefficient of the background filter is updated, and the background filter is used to filter the speaker voice signal corresponding to the next audio frame based on the updated first filter coefficient; according to the speaker voice signal, the target residual signal is subjected to nonlinear echo cancellation processing to obtain the near-end voice signal. In the above scheme, by inputting the speaker voice signal into the set foreground filter and background filter, the linear component of the acoustic echo can be effectively estimated to obtain the first linear echo and the second linear echo. By determining the target residual signal according to the first linear echo, the second linear echo and the microphone voice signal, the linear component of the acoustic echo in the microphone voice signal can be effectively eliminated to obtain the target residual signal. By updating the first filter coefficient of the background filter frame by frame, the echo path change can be dynamically tracked to achieve a better effect of eliminating the acoustic echo. By performing nonlinear echo elimination processing on the target residual signal according to the speaker voice signal to obtain the near-end voice signal, the nonlinear component in the target residual signal can be further eliminated to achieve a better echo removal effect. The overall calculation of the audio echo processing process has a fast convergence speed and is not easy to diverge, which can ensure the quality of the output audio.

[0141] In a possible embodiment, the filter coefficient updating module 202 is further configured to:

[0142] Subtract the microphone speech signal from the target residual signal to obtain the target linear echo;

[0143] Determine a target filter coefficient according to an odd-even frame type of a current audio frame, a target linear echo, a microphone voice signal, a loudspeaker voice signal, a target residual signal, a second linear echo, and a set neural network model;

[0144] The first filter coefficient of the background filter is updated to the target filter coefficient.

[0145] In a possible embodiment, the filter coefficient updating module 202 is further configured to:

[0146] When the current audio frame is an odd-numbered frame, a preset number of input feature values ​​are calculated according to the speaker speech signal, the microphone speech signal, the target linear echo, the target residual signal and the first filter coefficient of the background filter;

[0147] Inputting a plurality of input feature values ​​into the neural network model to obtain first step length information and first threshold information;

[0148] A target filter coefficient is determined according to the first filter coefficient of the background filter, the first threshold information, the first step length information, the speaker voice signal, the microphone voice signal and the second linear echo.

[0149] In a possible embodiment, a coefficient determination module is further included, configured as follows:

[0150] When the current audio frame is an even frame, the second step length information is calculated according to the target linear echo and the target residual signal;

[0151] A target filter coefficient is determined according to the first filter coefficient of the background filter, a preset constant threshold, the second step length information, the speaker voice signal, the microphone voice signal, and the second linear echo.

[0152] In a possible embodiment, the residual signal determination module 201 is further configured to:

[0153] Subtracting the microphone speech signal from the first linear echo to obtain a first residual signal;

[0154] Subtracting the microphone speech signal from the second linear echo to obtain a second residual signal;

[0155] The residual signal that satisfies the minimum modulus value between the first residual signal and the second residual signal is determined as the target residual signal.

[0156] In a possible embodiment, it further includes a first coefficient adjustment module configured to:

[0157] A signal smoothing value is calculated based on the microphone speech signal and the historical signal smoothing value, a first residual signal smoothing value is calculated based on the first residual signal and the first historical residual signal smoothing value, and a second residual signal smoothing value is calculated based on the second residual signal and the second historical residual signal smoothing value;

[0158] When the signal smoothing value is greater than the product of the second residual signal smoothing value and the first preset coefficient, and the second residual signal smoothing value is less than the product of the first residual signal smoothing value and the second preset coefficient, the second filter coefficient of the foreground filter is replaced by the first filter coefficient of the background filter.

[0159] In a possible embodiment, a second coefficient adjustment module is further included, configured to:

[0160] When the signal smoothing value is greater than the product of the first residual signal smoothing value and the third preset coefficient, and the first residual signal smoothing value is less than the product of the second residual signal smoothing value and the fourth preset coefficient, the first filter coefficient of the background filter is replaced by the second filter coefficient of the foreground filter.

[0161] Fig.11 A schematic diagram of the structure of an audio echo processing device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the device includes a processor 301, a memory 302, an input device 303 and an output device 304; the number of processors 301 in the device can be one or more. Fig.11 A processor 301 is taken as an example; the processor 301, memory 302, input device 303 and output device 304 in the device can be connected by a bus or other means. Fig.11 The example of the connection via bus is taken. The memory 302, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the audio echo processing method in the embodiment of the present application. The processor 301 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 302, that is, implements the above-mentioned audio echo processing method. The input device 303 can be configured to receive input digital or character information, and generate key signal input related to user settings and function control of the device. The output device 304 may include a display device such as a display screen.

[0162] The embodiment of the present application also provides a non-volatile storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are configured to execute an audio echo processing method described in the above embodiment, which includes: obtaining a microphone voice signal and a speaker voice signal after delay alignment corresponding to a current audio frame, inputting the speaker voice signal into a set foreground filter and a background filter respectively, obtaining a first linear echo and a second linear echo, and determining a target residual signal according to the first linear echo, the second linear echo and the microphone voice signal; updating a first filter coefficient of the background filter based on the microphone voice signal, the speaker voice signal, the target residual signal and the second linear echo, so that the background filter performs filtering processing on the speaker voice signal corresponding to the next audio frame based on the updated first filter coefficient; and performing nonlinear echo cancellation processing on the target residual signal according to the speaker voice signal to obtain a near-end voice signal.

[0163] It is worth noting that in the above-mentioned embodiment of the audio echo processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not configured to limit the protection scope of the embodiments of the present application.

[0164] In some possible implementations, various aspects of the method provided by the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is configured to enable the computer device to execute the steps of the method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the audio echo processing method described in the embodiment of the present application. The program product may be implemented in any combination of one or more readable media.

Claims

1. An audio echo processing method, characterized in that: include: Acquire a microphone speech signal and a speaker speech signal after delay alignment corresponding to a current audio frame, input the speaker speech signal into a set foreground filter and a set background filter respectively, obtain a first linear echo and a second linear echo, and determine a target residual signal according to the first linear echo, the second linear echo and the microphone speech signal; Based on the microphone speech signal, the loudspeaker speech signal, the target residual signal and the second linear echo, updating the first filter coefficient of the background filter, so that the background filter performs filtering processing on the loudspeaker speech signal corresponding to the next audio frame based on the updated first filter coefficient; The target residual signal is subjected to nonlinear echo cancellation processing according to the loudspeaker speech signal to obtain a near-end speech signal.

2. The audio echo processing method according to claim 1, characterized in that: The updating of the first filter coefficient of the background filter based on the microphone voice signal, the loudspeaker voice signal, the target residual signal and the second linear echo comprises: Subtracting the microphone speech signal from the target residual signal to obtain a target linear echo; Determine a target filter coefficient according to the odd and even frame type of the current audio frame, the target linear echo, the microphone voice signal, the speaker voice signal, the target residual signal, the second linear echo and the set neural network model; The first filter coefficient of the background filter is updated to the target filter coefficient.

3. The audio echo processing method according to claim 2, characterized in that: The step of determining a target filter coefficient according to the odd-even frame type of the current audio frame, the target linear echo, the microphone voice signal, the speaker voice signal, the target residual signal, the second linear echo, and the set neural network model includes: When the current audio frame is an odd-numbered frame, a preset number of input feature values ​​are calculated according to the speaker voice signal, the microphone voice signal, the target linear echo, the target residual signal and the first filter coefficient of the background filter; Inputting the plurality of input feature values ​​into the neural network model to obtain first step length information and first threshold information; A target filter coefficient is determined according to the first filter coefficient of the background filter, the first threshold information, the first step length information, the speaker voice signal, the microphone voice signal and the second linear echo.

4. The audio echo processing method according to claim 3, characterized in that: Also includes: When the current audio frame is an even frame, calculating the second step length information according to the target linear echo and the target residual signal; A target filter coefficient is determined according to the first filter coefficient of the background filter, a preset constant threshold, the second step size information, the speaker voice signal, the microphone voice signal and the second linear echo.

5. The audio echo processing method according to claim 1, characterized in that: The determining a target residual signal according to the first linear echo, the second linear echo and the microphone speech signal comprises: Subtracting the microphone speech signal from the first linear echo to obtain a first residual signal; Subtracting the microphone speech signal from the second linear echo to obtain a second residual signal; The residual signal that satisfies the minimum modulus value between the first residual signal and the second residual signal is determined as a target residual signal.

6. The audio echo processing method according to claim 5, characterized in that: After determining the residual signal satisfying the minimum modulus value between the first residual signal and the second residual signal as the target residual signal, the method further includes: A signal smoothing value is calculated based on the microphone speech signal and the historical signal smoothing value, a first residual signal smoothing value is calculated based on the first residual signal and the first historical residual signal smoothing value, and a second residual signal smoothing value is calculated based on the second residual signal and the second historical residual signal smoothing value; When the signal smoothing value is greater than the product of the second residual signal smoothing value and the first preset coefficient, and the second residual signal smoothing value is less than the product of the first residual signal smoothing value and the second preset coefficient, the second filter coefficient of the foreground filter is replaced by the first filter coefficient of the background filter.

7. The audio echo processing method according to claim 6, characterized in that: Also includes: When the signal smoothing value is greater than the product of the first residual signal smoothing value and the third preset coefficient, and the first residual signal smoothing value is less than the product of the second residual signal smoothing value and the fourth preset coefficient, the first filter coefficient of the background filter is replaced by the second filter coefficient of the foreground filter.

8. An audio echo processing device, characterized in that: include: A residual signal determination module is configured to obtain a microphone voice signal corresponding to a current audio frame and a speaker voice signal after delay alignment, input the speaker voice signal into a set foreground filter and a set background filter respectively, obtain a first linear echo and a second linear echo, and determine a target residual signal according to the first linear echo, the second linear echo and the microphone voice signal; a filter coefficient updating module, configured to update a first filter coefficient of the background filter based on the microphone voice signal, the loudspeaker voice signal, the target residual signal and the second linear echo, so that the background filter performs filtering processing on the loudspeaker voice signal corresponding to the next audio frame based on the updated first filter coefficient; The near-end speech signal determination module is configured to perform nonlinear echo cancellation processing on the target residual signal according to the speaker speech signal to obtain a near-end speech signal.

9. An audio echo processing device, the device comprising: one or more processors; The storage device is configured to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the audio echo processing method according to any one of claims 1 to 7.

10. A non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions are configured to perform the audio echo processing method according to any one of claims 1 to 7 when executed by a computer processor.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the audio echo processing method according to any one of claims 1 to 7 is implemented.

Images