Nonlinear echo cancellation method, device and electronic equipment

Abstract

The present invention discloses a nonlinear echo cancellation method, device and electronic device, and relates to the technical field of echo cancellation. The method comprises: obtaining a signal to be echo-cancelled, converting the signal to be echo-cancelled from a time domain signal to a frequency domain signal; converting the frequency domain signal to a measurement domain signal, presetting a smoothing parameter according to whether the voice state of the near-end signal in the measurement domain changes, calculating a power spectrum parameter set according to each frequency point in the measurement domain, the measurement domain signal and the smoothing parameter, calculating a mutual interference coefficient according to the power spectrum parameter set, judging whether the current frame has an echo according to the mutual interference coefficient of each frequency point in the perceived measurement domain, performing nonlinear suppression according to the judgment result to obtain a nonlinear echo suppression signal based on the measurement domain; converting the measurement domain signal to a frequency domain signal, post-processing the frequency domain signal, and converting the post-processed signal to a time domain signal. The misjudgment rate of coherence caused by Gaussian white noise is reduced, and the calculation amount of the algorithm is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of echo cancellation, and in particular to a nonlinear echo cancellation method, device and electronic equipment. Background Art

[0002] In recent years, the development of the Internet has promoted the widespread application of real-time communication technologies such as Internet telephony and video conferencing. At the same time, its voice quality has also received increasing attention, and one of the most critical factors is the acoustic echo problem. In actual call equipment, the actual audio signal is played through the speaker and propagated through the air. Nonlinear distortion in the form of harmonic distortion and dynamic compression may occur due to device characteristics or environmental changes, resulting in nonlinear characteristics in the echo path. This nonlinear feature seriously affects the quality of the call.

[0003] Currently, the more mature and widely used echo cancellation algorithm in WebRTC, which is open sourced by Google, is used to reduce these nonlinear distortions. An adaptive filter is used to adjust the effect of echo cancellation, and a nonlinear processing module is introduced to deal with nonlinear problems. The principle is to convert the time domain signal into the Fourier transform domain, calculate the coherence of each frequency band, and determine whether there is an echo in the near-end signal.

[0004] However, the existing echo cancellation algorithms mentioned above have some drawbacks. For example, after adding Gaussian white noise, the randomness of the noise will introduce additional phase differences and amplitude fluctuations, resulting in a decrease in the calculated coherence coefficient. Especially when the signal-to-noise ratio (SNR) is low, the impact of noise is more significant, which may cause the coherence of the signal to drop significantly. Gaussian white noise has a uniform effect on each frequency component, so when calculating coherence, the noise may mask the phase consistency of certain frequency components in the speech signal, causing the coherence to become unreliable in these frequency ranges. Summary of the invention

[0005] The purpose of the invention is to reduce the misjudgment rate of coherence affected by Gaussian white noise in far-end signals, near-end signals and error signals, and to provide a nonlinear echo cancellation method, device and electronic equipment.

[0006] Technical solution:

[0007] According to one aspect of the present disclosure, a nonlinear echo cancellation method is provided, comprising:

[0008] Acquire a signal to be echo-cancelled, and convert the signal to be echo-cancelled from a time domain signal to a frequency domain signal; wherein the signal to be echo-cancelled includes a far-end signal, a near-end signal, and an error signal of a current frame;

[0009] The frequency domain signal is converted into a measurement domain signal, a smoothing parameter is pre-set according to whether the speech state of the near-end signal in the measurement domain changes, a power spectrum parameter set is calculated according to each frequency point in the measurement domain, the measurement domain signal and the smoothing parameter, and a mutual interference coefficient is calculated according to the power spectrum parameter set, wherein the mutual interference coefficient includes: a mutual interference coefficient between a far-end signal and a near-end signal, recorded as a far-end near-end mutual interference coefficient, and an error mutual interference coefficient between a near-end signal and an error signal, recorded as a near-end error mutual interference coefficient;

[0010] Judging whether the current frame has an echo according to the far-end near-end mutual interference coefficient and the near-end error mutual interference coefficient of each frequency point in the measurement domain, and performing nonlinear suppression according to the judgment result to obtain a nonlinear echo suppression signal based on the measurement domain;

[0011] The nonlinear echo suppression signal in the measurement domain is converted into a nonlinear echo suppression signal based on the frequency domain, and the nonlinear echo suppression signal based on the frequency domain is post-processed to obtain an output signal based on the frequency domain, and the output signal based on the frequency domain is converted into a time domain signal.

[0012] According to another aspect of the present disclosure, a nonlinear echo cancellation device is provided, which is applied to a terminal device, comprising:

[0013] An acquisition module is used to acquire the signal to be echo-cancelled and convert the signal to be echo-cancelled from a time domain signal to a frequency domain signal; wherein the signal to be echo-cancelled includes a far-end signal, a near-end signal and an error signal of a current frame;

[0014] A measurement domain module is used to convert a frequency domain signal into a measurement domain signal, pre-set a smoothing parameter according to whether a speech state of a near-end signal in the measurement domain changes, calculate a power spectrum parameter set according to each frequency point in the measurement domain, the measurement domain signal and the smoothing parameter, and calculate a mutual interference coefficient according to the power spectrum parameter set, wherein the mutual interference coefficient includes: a mutual interference coefficient between a far-end signal and a near-end signal, recorded as a far-end near-end mutual interference coefficient, and an error mutual interference coefficient between a near-end signal and an error signal, recorded as a near-end error mutual interference coefficient;

[0015] The elimination module is used to determine whether the current frame has an echo according to the far-end near-end mutual interference coefficient and the near-end error mutual interference coefficient of each frequency point in the measurement domain, and perform nonlinear suppression according to the judgment result to obtain a nonlinear echo suppression signal based on the measurement domain;

[0016] The conversion module is used to convert the nonlinear echo suppression signal in the measurement domain into a nonlinear echo suppression signal based on the frequency domain, and post-process the nonlinear echo suppression signal based on the frequency domain to obtain an output signal based on the frequency domain, and convert the output signal based on the frequency domain into a time domain signal.

[0017] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform any of the above-mentioned nonlinear echo cancellation methods.

[0019] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute any one of the above-mentioned nonlinear echo cancellation methods.

[0020] Beneficial effects: The far-end signal, near-end signal and error signal are preprocessed using compressed sensing theory, that is, the far-end signal, near-end signal and error signal in the frequency domain are converted into the far-end signal, near-end signal and error signal in the compressed sensing measurement domain, and then one or more coherences between these signals are further calculated, which not only reduces the misjudgment rate of coherence affected by Gaussian white noise in the far-end signal, near-end signal and error signal;

[0021] After the compression of the compressed sensing measurement matrix, the measurement domain signal length is smaller than the time domain signal length, which reduces the calculation amount of the algorithm;

[0022] Depending on whether the speech state of the near-end signal changes, different smoothing parameters are used to smooth the smoothed cross-power spectrum between the far-end signal and the near-end signal, the smoothed cross-power spectrum between the near-end signal and the error signal, the power spectrum of the far-end signal, the power spectrum of the near-end signal, and the power spectrum of the error signal. This can speed up convergence and avoid some detail fluctuations in the frequency spectrum. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of a nonlinear echo cancellation method according to embodiment 1 of the present invention;

[0024] Figure 2 It is a structural schematic diagram of a nonlinear echo elimination device according to embodiment 2 of the present invention. DETAILED DESCRIPTION

[0025] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Embodiment 1: Figure 1 This is a flowchart of a nonlinear echo elimination method provided in Embodiment 1 of the present disclosure.

[0027] In the above embodiment, the nonlinear echo cancellation method is applied to a terminal device with a communication function and / or a human-computer interaction function, wherein the terminal device may be a mobile phone, a tablet computer, etc. During the communication process of the terminal device, the input signal of the microphone includes a near-end signal, that is, a voice signal input by the user, and there are also linear echo signals and nonlinear echo signals. The purpose of the echo cancellation method is to solve the problem that the algorithm fails due to the existence of Gaussian white noise when the prior art uses coherence for nonlinear suppression, and proposes a coherence coefficient based on a compressed sensing measurement domain (hereinafter referred to as the "measurement domain"), which is used to determine whether there is an echo in the near-end signal of the current frame and perform nonlinear suppression, thereby solving the problem that the algorithm fails due to the existence of Gaussian white noise in the prior art, so as to obtain a relatively pure near-end signal.

[0028] like Figure 1 As shown, including:

[0029] Step S101, obtaining a signal to be echo-cancelled, and converting the signal to be echo-cancelled from a time domain signal to a frequency domain signal; wherein the signal to be echo-cancelled includes a far-end signal, a near-end signal, and an error signal of a current frame;

[0030] Preferably, the signal to be echo-cancelled may be converted from a time domain signal to a frequency domain signal by using Fourier transform;

[0031] As a preferred implementation, in the echo cancellation algorithm, for the signal to be echo-cancelled in the current frame, the signal to be echo-cancelled includes: the far-end signal , near-end signal and the error signal obtained after linear echo cancellation , where the signal length is , respectively, and perform Fourier transform to obtain the far-end signal based on the frequency domain , near-end signal and the error signal obtained after linear echo cancellation , where the signal length is .

[0032] Step S102, converting the frequency domain signal into a measurement domain signal, presetting a smoothing parameter according to whether the speech state of the near-end signal in the measurement domain changes, calculating a power spectrum parameter set according to each frequency point in the measurement domain, the measurement domain signal and the smoothing parameter, and calculating a mutual interference coefficient according to the power spectrum parameter set, wherein the mutual interference coefficient includes: a mutual interference coefficient between a far-end signal and a near-end signal, recorded as a far-end near-end mutual interference coefficient, and an error mutual interference coefficient between a near-end signal and an error signal, recorded as a near-end error mutual interference coefficient;

[0033] In the above embodiment, the compressed sensing theory is used to convert the error signal, far-end signal and near-end signal obtained after linear filtering in the echo cancellation algorithm from the frequency domain to the measurement domain, and then calculate the mutual coherence between the error signal, far-end signal and near-end signal based on the measurement domain, thereby avoiding the influence of Gaussian white noise;

[0034] Specifically, the far-end signal, near-end signal and error signal are preprocessed using compressed sensing theory, that is, the far-end signal, near-end signal and error signal in the frequency domain are converted into the far-end signal, near-end signal and error signal in the compressed sensing measurement domain, and then one or more coherences between these signals are further calculated, which not only reduces the misjudgment rate of the Gaussian white noise in the far-end signal, near-end signal and error signal that affects the coherence.

[0035] It should be noted that the basic principle of compressed sensing theory is that for compressible signals or signals that are sparse in a certain transform domain, low-dimensional projection can be performed, and then the compressed sensing reconstruction algorithm can be applied to restore the signal. Its mathematical model is shown in the following formula:

[0036] ; (1)

[0037] in, Indicates the length The one-dimensional original signal;

[0038] Indicates the length One-dimensional compressed sensing measurement signal;

[0039] Indicates size The measurement matrix.

[0040] Since the compressed sensing theory is based on signal The sparsity of the signal If it is sparse, then after projection, most of the projection coefficients are zero, and only a few are non-zero. The mathematical expression is shown in the following formula:

[0041] ; (2)

[0042] in, represents an orthogonal basis matrix;

[0043] Indicates the length The one-dimensional projection coefficients of are also called sparse signals.

[0044] Combining formula (1) and formula (2), we can get the following formula:

[0045] ; (3)

[0046] To restore the original signal , we need to solve the inverse of formula (3) to obtain the sparse signal By converting the inverse problem of formula (3) into a minimization problem The norm problem is solved by the following formula:

[0047] ; (4)

[0048] in represents the number of non-zero elements of the vector. However, solving formula (4) is an NP (Non-deterministic Polynomial) problem, which needs to be converted into The norm minimization problem is approximately solved to obtain formula (5):

[0049] ; (5)

[0050] Step S102 specifically includes:

[0051] Step S1021, mapping the frequency domain signal to the perception measurement domain through the compressed sensing theory, so as to convert the frequency domain signal into the measurement domain signal, as shown in the following formula:

[0052] ; (6)

[0053] ; (7)

[0054] ; (8)

[0055] in, Used to indicate the far-end signal based on the measurement domain. Used to represent far-end signals based on frequency domain;

[0056] Used to represent the near-end signal based on the measurement domain. Used to represent near-end signals based on frequency domain;

[0057] Used to represent the error signal obtained after linear echo cancellation based on the measurement domain, Used to represent the error signal obtained after linear echo cancellation based on frequency domain;

[0058] Indicates size The measurement matrix is Used to represent the signal length of the time domain signal, Used to represent the signal length of the measurement domain signal, ;

[0059] In the above embodiment, after the compression of the compressed sensing measurement matrix, the measurement domain signal length is made smaller than the frequency domain signal length required for calculation (half of the time domain signal length plus one), thereby reducing the amount of calculation of the algorithm.

[0060] Step S1022, as shown in the following formula, is calculated based on each frequency point of the sensing measurement domain, the measurement domain signal and a preset smoothing parameter:

[0061] A cross power spectrum between a far-end signal and a near-end signal based on the measurement domain is recorded as a far-end near-end cross power spectrum;

[0062] ; (9)

[0063] in, Indicates frame, Used to indicate the frequency points based on the measurement domain. ;

[0064] Used to indicate calculation of conjugation;

[0065] Used to represent the near-end and far-end cross-power spectra based on the measurement domain;

[0066] Used to represent smoothing parameters;

[0067] A cross power spectrum between a near-end signal and an error signal based on the measurement domain is denoted as a near-end error cross power spectrum;

[0068] ; (10)

[0069] in, Used to represent the near-end error cross power spectrum based on the measurement domain;

[0070] and power spectra of far-end signals, near-end signals and error signals based on the measurement domain;

[0071] ; (11)

[0072] ; (12)

[0073] ; (13)

[0074] in, Used to represent the power spectrum of the far-end signal based on the measurement domain;

[0075] Used to represent the power spectrum of the near-end signal based on the measurement domain;

[0076] Used to represent the power spectrum of the error signal based on the measurement domain.

[0077] Step S102 also includes:

[0078] Step S1023, using a speech endpoint detection algorithm to pre-set a corresponding smoothing parameter according to whether the speech state of the near-end signal changes, wherein the smoothing parameter includes a first smoothing parameter and a second smoothing parameter, and the first smoothing parameter is smaller than the second smoothing parameter;

[0079] Specifically, as shown in the following formula:

[0080] When the speech state of the near-end signal changes, setting a first smoothing parameter;

[0081] It should be noted that the voice state of the near-end signal changes in the following two situations:

[0082] Case 1: From the noise area to the speech area: The current frame of the near-end signal is a speech frame (there is near-end speech or far-end echo) and the previous frame is a silence frame (does not contain speech information);

[0083] Case 2: From the speech area to the noise area: when the current frame of the near-end signal is a silence frame and the previous frame is a speech frame;

[0084] When the speech state of the near-end signal does not change, setting a second smoothing parameter;

[0085] It should be noted that the speech state of the near-end signal does not change in the following two situations:

[0086] Case 1: Speech area: The current frame of the near-end signal is a speech frame and the previous frame is also a speech frame;

[0087] Case 2: Noise zone: The current frame of the near-end signal is a silent frame and the previous frame is also a silent frame;

[0088] As shown in the following formula:

[0089] ; (14)

[0090] in,

[0091] Used to represent smoothing parameters;

[0092] Used to represent the first smoothing parameter;

[0093] Used to represent the second smoothing parameter;

[0094] Used to indicate the speech endpoint detection result of the near-end signal.

[0095] In the above embodiment, when the speech state of the near-end signal changes, a smaller smoothing parameter is used to smooth the smoothed cross-power spectrum between the far-end signal and the near-end signal, the smoothed cross-power spectrum between the near-end signal and the error signal, the power spectrum of the far-end signal, the power spectrum of the near-end signal, and the power spectrum of the error signal to accelerate convergence, while a larger smoothing parameter is used in the speech area or the noise area to avoid fluctuations in some details in the frequency spectrum, which may cause musical noise in the output signal of the nonlinear suppression.

[0096] Step S102 includes:

[0097] S1024, calculating the mutual interference coefficient according to the power spectrum parameter set, specifically including:

[0098] The mutual interference coefficient of the far-end signal and the near-end signal is calculated according to the far-end near-end cross power spectrum, the power spectrum of the far-end signal, and the power spectrum of the near-end signal, and is recorded as the far-end near-end mutual interference coefficient, as shown in the following formula:

[0099] ; (15)

[0100] in, Used to represent the far-end and near-end mutual interference coefficient;

[0101] The mutual interference coefficient between the near-end signal and the error signal is calculated according to the near-end error cross power spectrum, the power spectrum of the near-end signal and the power spectrum of the error signal, and is recorded as the near-end error cross interference coefficient;

[0102] ; (16)

[0103] in, Used to represent the near-end error mutual interference coefficient.

[0104] Step S103, judging whether the current frame has an echo according to the far-end near-end mutual interference coefficient and the near-end error mutual interference coefficient of each frequency point in the perceptual measurement domain, and performing nonlinear suppression according to the judgment result to obtain a nonlinear echo suppression signal based on the measurement domain;

[0105] Step S103 specifically includes:

[0106] Step S1031, counting the number of far-end near-end mutual interference coefficients and near-end error mutual interference coefficients of the frequency points of the current frame in the measurement domain that meet the first preset condition, and when the number that meets the first preset condition is greater than the first preset number, determining that the current frame has no echo, and the echo suppression coefficient is the mutual interference coefficient of the near-end signal and the error signal, as shown in the following formula:

[0107] ; (17)

[0108] in, Used to represent the echo suppression coefficient;

[0109] As a preferred implementation, the first preset condition can be and , is a positive real number greater than 1;

[0110] Specifically, statistics are based on the frequency points in the measurement domain Satisfied and If num is greater than the first preset number, the current frame is judged to be echo-free, and the echo suppression coefficient for ( ).

[0111] Step S1032, counting the number of far-end near-end mutual interference coefficients and near-end error mutual interference coefficients of the frequency points of the current frame in the measurement domain that meet the second preset condition, and when the number that meets the second preset condition is greater than the second preset number, determining that the current frame has an echo, and setting the echo suppression coefficient according to the mutual interference coefficient of the far-end signal and the near-end signal, and the error mutual interference coefficient of the near-end signal and the error signal, as shown in the following formula:

[0112] ; (18)

[0113] As a preferred implementation, the second preset condition can be or , is a positive real number greater than 1;

[0114] Specifically, statistics are based on the frequency points in the measurement domain Satisfied or If num is greater than a certain number, the current frame is judged to contain echo, and the echo suppression coefficient .

[0115] Step S1033, nonlinear suppression is performed on the current frame according to the echo suppression coefficient to obtain a nonlinear echo suppression signal based on the measurement domain, as shown in the following formula:

[0116] ; (19)

[0117] in, Used to represent nonlinear echo suppression signals based on the measurement domain.

[0118] In the above embodiment, based on each frequency point in the compressed sensing measurement domain, the coherence of the far-end signal, the near-end signal and the error signal in the measurement domain is calculated, that is, the mutual coherence coefficient between the far-end signal and the near-end signal in the measurement domain, and the mutual coherence coefficient between the near-end signal and the error signal are calculated. Based on these coherence coefficients, it is determined whether the current near-end signal contains an echo signal, thereby calculating the corresponding echo suppression coefficient of each frequency band, and further suppressing the residual echo signal in the near-end signal.

[0119] Step S104, converting the nonlinear echo suppression signal in the measurement domain into a nonlinear echo suppression signal based on the frequency domain, post-processing the nonlinear echo suppression signal based on the frequency domain to obtain an output signal based on the frequency domain, and converting the output signal based on the frequency domain into a time domain signal;

[0120] Step S104 specifically includes:

[0121] Step S1041, using an orthogonal matching pursuit algorithm to convert the nonlinear echo signal based on the measurement domain into a nonlinear echo suppression signal based on the frequency domain;

[0122] Step S1042, post-processing the frequency-domain-based nonlinear echo suppression signal to obtain a frequency-domain-based output signal, as shown in the following formula:

[0123] ; (20)

[0124] in, It represents the conjugate transposition of the error signal E obtained after the linear echo cancellation based on the frequency domain;

[0125] Used to represent the output signal based on the frequency domain;

[0126] Used to represent frequency-domain based nonlinear echo suppression signals.

[0127] In the above embodiment, post-processing is performed by convolution of the error signal based on the frequency domain and the output result of the nonlinear suppression, so as to avoid speech loss caused by over-suppression of the nonlinear suppression algorithm.

[0128] Step S1043, converting the output signal based on the frequency domain into a time domain signal;

[0129] Preferably, an inverse Fourier transform may be used to convert the frequency domain-based output signal into a time domain signal;

[0130] Specifically, the frequency-domain output signal of the nonlinear echo suppression is inversely Fourier transformed , and perform overlap-addition to convert it into a time domain signal .

[0131] Example 2: Combined with Figure 2 , the nonlinear echo elimination device provided by the present invention is described.

[0132] Figure 2 is a schematic diagram of the structure of a nonlinear echo cancellation device provided in accordance with Embodiment 2 of the present disclosure. The nonlinear echo cancellation device is applied to a terminal device, such as Figure 2 As shown, including:

[0133] An acquisition module is used to acquire the signal to be echo-cancelled and convert the signal to be echo-cancelled from a time domain signal to a frequency domain signal; wherein the signal to be echo-cancelled includes a far-end signal, a near-end signal and an error signal of a current frame;

[0134] A measurement domain module is used to convert a frequency domain signal into a measurement domain signal, pre-set a smoothing parameter according to whether a speech state of a near-end signal in the measurement domain changes, calculate a power spectrum parameter set according to each frequency point in the measurement domain, the measurement domain signal and the smoothing parameter, and calculate a mutual interference coefficient according to the power spectrum parameter set, wherein the mutual interference coefficient includes: a mutual interference coefficient between a far-end signal and a near-end signal, recorded as a far-end near-end mutual interference coefficient, and an error mutual interference coefficient between a near-end signal and an error signal, recorded as a near-end error mutual interference coefficient;

[0135] The elimination module is used to judge whether the current frame has an echo according to the far-end near-end mutual interference coefficient and the near-end error mutual interference coefficient of each frequency point in the perception measurement domain, and perform nonlinear suppression according to the judgment result to obtain a nonlinear echo suppression signal based on the measurement domain;

[0136] The conversion module is used to convert the nonlinear echo suppression signal in the measurement domain into a nonlinear echo suppression signal based on the frequency domain, and post-process the nonlinear echo suppression signal based on the frequency domain to obtain an output signal based on the frequency domain, and convert the output signal based on the frequency domain into a time domain signal.

[0137] In the above embodiment, the far-end signal, the near-end signal and the error signal are preprocessed using the compressed sensing theory, that is, the far-end signal, the near-end signal and the error signal in the frequency domain are converted into the far-end signal, the near-end signal and the error signal in the compressed sensing measurement domain, and then one or more coherences between these signals are further calculated, which not only reduces the misjudgment rate of the coherence affected by the Gaussian white noise in the far-end signal, the near-end signal and the error signal;

[0138] After the compression of the compressed sensing measurement matrix, the measurement domain signal length is smaller than the time domain signal length, which reduces the calculation amount of the algorithm;

[0139] Depending on whether the speech state of the near-end signal changes, different smoothing parameters are used to smooth the smoothed cross-power spectrum between the far-end signal and the near-end signal, the smoothed cross-power spectrum between the near-end signal and the error signal, the power spectrum of the far-end signal, the power spectrum of the near-end signal, and the power spectrum of the error signal. This can speed up convergence and avoid some detail fluctuations in the spectrum.

[0140] The specific implementation of the nonlinear echo cancellation device provided in the embodiment of the present disclosure is consistent with the nonlinear echo cancellation method provided in the first embodiment, and will not be described in detail here.

[0141] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0142] Among them, an electronic device includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the nonlinear echo elimination method in the aforementioned aspect.

[0143] Among them, a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the nonlinear echo cancellation method in the aforementioned aspect.

[0144] A computer program product includes a computer program, and when the computer program is executed by a processor, the nonlinear echo cancellation method in the aforementioned aspect is implemented.

[0145] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0146] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0147] Any process or method description in a flowchart or otherwise described herein may be understood to represent a unit, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0148] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0149] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0150] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

Claims

1. A nonlinear echo cancellation method, characterized in that: include: Acquire a signal to be echo-cancelled, and convert the signal to be echo-cancelled from a time domain signal to a frequency domain signal; wherein the signal to be echo-cancelled includes a far-end signal, a near-end signal, and an error signal of a current frame; convert the frequency domain signal to a measurement domain signal, pre-set a smoothing parameter according to whether the voice state of the near-end signal in the measurement domain changes, calculate a power spectrum parameter set according to each frequency point in the measurement domain, the measurement domain signal, and the smoothing parameter, and calculate a mutual interference coefficient according to the power spectrum parameter set, wherein the mutual interference coefficient includes: a mutual interference coefficient between a far-end signal and a near-end signal, recorded as a far-end near-end mutual interference coefficient, and an error mutual interference coefficient between a near-end signal and an error signal, recorded as a near-end error mutual interference coefficient; wherein, The measurement domain signal is a compressed sensing measurement domain signal obtained based on the compressed sensing theory; judging whether the current frame has an echo according to the far-end near-end mutual interference coefficient and the near-end error mutual interference coefficient of each frequency point in the measurement domain, and performing nonlinear suppression according to the judgment result to obtain a nonlinear echo suppression signal based on the measurement domain; converting the nonlinear echo suppression signal in the measurement domain into a nonlinear echo suppression signal based on the frequency domain, and post-processing the nonlinear echo suppression signal based on the frequency domain to obtain an output signal based on the frequency domain, and converting the output signal based on the frequency domain into a time domain signal; wherein, the frequency domain signal is converted into a measurement domain signal, and a power spectrum parameter set is obtained according to each frequency point in the measurement domain, the measurement domain signal and a smoothing parameter calculation, specifically including: The frequency domain signal is mapped to the measurement domain through the compressed sensing theory to convert the frequency domain signal into a measurement domain signal, as shown in the following formula: ； ； ； in, Used to indicate the far-end signal based on the measurement domain. Used to represent far-end signals based on frequency domain; Used to represent the near-end signal based on the measurement domain. Used to represent near-end signals based on frequency domain; Used to represent the error signal obtained after linear echo cancellation based on the measurement domain, Used to represent the error signal obtained after linear echo cancellation based on frequency domain; Indicates size The measurement matrix is Used to represent the signal length of the time domain signal, Used to represent the signal length of the measurement domain signal, ; As shown in the following formula, it is calculated based on each frequency point of the measurement domain, the measurement domain signal and a preset smoothing parameter: a cross power spectrum between a far-end signal and a near-end signal based on the measurement domain, recorded as a far-end near-end cross power spectrum; ； in, Indicates frame, Used to indicate the frequency points based on the measurement domain. ; Used to indicate the calculation of conjugation; Used to represent the near-end and far-end cross-power spectra based on the measurement domain; Used to represent the smoothing parameter; based on the cross power spectrum between the near-end signal and the error signal in the measurement domain, it is recorded as the near-end error cross power spectrum; ； in, Used to represent the near-end error cross power spectrum based on the measurement domain; and the power spectrum of the far-end signal, the near-end signal and the error signal based on the measurement domain; ； ； ； in, Used to represent the power spectrum of the far-end signal based on the measurement domain; Used to represent the power spectrum of the near-end signal based on the measurement domain; A power spectrum for representing an error signal based on the measurement domain; The step of calculating the mutual interference coefficient according to the power spectrum parameter set specifically includes: The far-end near-end mutual interference coefficient is calculated according to the far-end near-end cross power spectrum, the power spectrum of the far-end signal and the power spectrum of the near-end signal, as shown in the following formula: ;in, Used to represent the far-end near-end mutual interference coefficient; calculate the near-end error mutual interference coefficient according to the near-end error cross power spectrum, the power spectrum of the near-end signal and the power spectrum of the error signal; ;in, Used to represent the near-end error mutual interference coefficient.

2. The nonlinear echo cancellation method according to claim 1, characterized in that: The step of presetting a smoothing parameter according to whether the voice state of the near-end signal in the measurement domain changes comprises: using a voice endpoint detection algorithm to pre-set a corresponding smoothing parameter according to whether the voice state of the near-end signal changes, wherein the smoothing parameter comprises a first smoothing parameter and a second smoothing parameter, and the first smoothing parameter is less than the second smoothing parameter; specifically, as shown in the following formula: When the speech state of the near-end signal changes, a first smoothing parameter is set; when the speech state of the near-end signal does not change, a second smoothing parameter is set; ； in, Used to represent smoothing parameters; Used to represent the first smoothing parameter; Used to represent the second smoothing parameter; Used to indicate the speech endpoint detection result of the near-end signal.

3. The nonlinear echo cancellation method according to claim 1, characterized in that: The method of judging whether the current frame has an echo according to the far-end near-end mutual interference coefficient and the near-end error mutual interference coefficient of each frequency point in the measurement domain, and performing nonlinear suppression according to the judgment result to obtain a nonlinear echo suppression signal based on the measurement domain specifically includes: counting the number of far-end near-end mutual interference coefficients and near-end error mutual interference coefficients of the frequency points of the current frame based on the measurement domain that meet a first preset condition, and when the number that meets the first preset condition is greater than the first preset number, determining that the current frame has no echo, and the echo suppression coefficient is the mutual interference coefficient of the near-end signal and the error signal, as shown in the following formula: ;in, Used to represent the echo suppression coefficient; count the number of far-end near-end mutual interference coefficients and near-end error mutual interference coefficients of the frequency point of the current frame based on the measurement domain that meet the second preset condition, and when the number that meets the second preset condition is greater than the second preset number, determine that the current frame has an echo, and set the echo suppression coefficient according to the mutual interference coefficient of the far-end signal and the near-end signal, and the error mutual interference coefficient of the near-end signal and the error signal, as shown in the following formula: ； The current frame is subjected to nonlinear suppression according to the echo suppression coefficient to obtain a nonlinear echo suppression signal based on the measurement domain, as shown in the following formula: ； in, Used to represent nonlinear echo suppression signals based on the measurement domain.

4. The nonlinear echo cancellation method according to claim 1, characterized in that: The method of converting the nonlinear echo suppression signal in the measurement domain into a nonlinear echo suppression signal based on the frequency domain, and post-processing the nonlinear echo suppression signal based on the frequency domain to obtain an output signal based on the frequency domain, specifically includes: using an orthogonal matching pursuit algorithm to convert the nonlinear echo suppression signal based on the measurement domain into a nonlinear echo suppression signal based on the frequency domain; The frequency-domain-based nonlinear echo suppression signal is post-processed to obtain a frequency-domain-based output signal, as shown in the following formula: ； in, It represents the conjugate transposition of the error signal E obtained after the linear echo cancellation based on the frequency domain; Used to represent the output signal based on the frequency domain; Used to represent frequency-domain based nonlinear echo suppression signals.

5. The nonlinear echo cancellation method according to claim 1, wherein: The signal to be echo-cancelled is converted from a time domain signal to a frequency domain signal by using Fourier transform; and / or an output signal based on the frequency domain is converted to a time domain signal by using inverse Fourier transform.

6. A nonlinear echo cancellation device, characterized in that: The invention is applied to a terminal device, and comprises: an acquisition module, which is used to acquire a signal to be echo-cancelled, and convert the signal to be echo-cancelled from a time domain signal to a frequency domain signal; wherein the signal to be echo-cancelled comprises a far-end signal, a near-end signal and an error signal of a current frame; a measurement domain module, which is used to convert a frequency domain signal to a measurement domain signal, and pre-set a smoothing parameter according to whether a voice state of a near-end signal in the measurement domain changes, and calculate a power spectrum parameter set according to each frequency point in the measurement domain, the measurement domain signal and the smoothing parameter, and calculate a mutual interference coefficient according to the power spectrum parameter set, wherein the mutual interference coefficient comprises: a mutual interference coefficient between a far-end signal and a near-end signal, which is recorded as a far-end near-end mutual interference coefficient, and the error mutual interference coefficient of the near-end signal and the error signal, recorded as the near-end error mutual interference coefficient, wherein the measurement domain signal is a compressed sensing measurement domain signal obtained based on the compressed sensing theory; an elimination module, used to judge whether the current frame has an echo according to the far-end near-end mutual interference coefficient and the near-end error mutual interference coefficient of each frequency point in the measurement domain, and perform nonlinear suppression according to the judgment result to obtain a nonlinear echo suppression signal based on the measurement domain; a conversion module, used to convert the nonlinear echo suppression signal in the measurement domain into a nonlinear echo suppression signal based on the frequency domain, and post-process the nonlinear echo suppression signal based on the frequency domain to obtain an output signal based on the frequency domain, and convert the output signal based on the frequency domain into a time domain signal; The step of converting the frequency domain signal into the measurement domain signal, and calculating the power spectrum parameter set according to each frequency point in the measurement domain, the measurement domain signal and the smoothing parameter, specifically includes: mapping the frequency domain signal to the measurement domain through compressed sensing theory to convert the frequency domain signal into the measurement domain signal, as shown in the following formula: ； ； ； in, Used to indicate the far-end signal based on the measurement domain. Used to represent far-end signals based on frequency domain; Used to represent the near-end signal based on the measurement domain. Used to represent near-end signals based on frequency domain; Used to represent the error signal obtained after linear echo cancellation based on the measurement domain, Used to represent the error signal obtained after linear echo cancellation based on frequency domain; Indicates size The measurement matrix is Used to represent the signal length of the time domain signal, Used to represent the signal length of the measurement domain signal, ; As shown in the following formula, based on each frequency point of the measurement domain, the measurement domain signal and the preset smoothing parameter, it is calculated: the cross power spectrum between the far-end signal and the near-end signal based on the measurement domain is recorded as the far-end near-end cross power spectrum; ； in, Indicates frame, Used to indicate the frequency points based on the measurement domain. ; Used to indicate the calculation of conjugation; Used to represent the near-end and far-end cross-power spectra based on the measurement domain; Used to represent smoothing parameters; A cross power spectrum between a near-end signal and an error signal based on the measurement domain is denoted as a near-end error cross power spectrum; ； in, Used to represent the near-end error cross power spectrum based on the measurement domain; and the power spectrum of the far-end signal, the near-end signal and the error signal based on the measurement domain; ； ； ； in, Used to represent the power spectrum of the far-end signal based on the measurement domain; Used to represent the power spectrum of the near-end signal based on the measurement domain; A power spectrum for representing an error signal based on the measurement domain; The calculating of the mutual interference coefficient according to the power spectrum parameter set specifically includes: calculating the far-end near-end mutual interference coefficient according to the far-end near-end cross power spectrum, the power spectrum of the far-end signal and the power spectrum of the near-end signal, as shown in the following formula: ； in, Used to represent the far-end near-end mutual interference coefficient; calculate the near-end error mutual interference coefficient according to the near-end error cross power spectrum, the power spectrum of the near-end signal and the power spectrum of the error signal; ； in, Used to represent the near-end error mutual interference coefficient.

7. An electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

Images