Echo cancellation method, apparatus, computer readable storage medium and terminal device

Abstract

The application belongs to the technical field of acoustic processing, and particularly relates to an echo cancellation method and device, a computer readable storage medium and a terminal device. The method comprises the following steps: obtaining a reference alignment signal and a microphone alignment signal; filtering the reference alignment signal and the microphone alignment signal by using a preset adaptive filter to obtain a filter output signal; performing speech compensation processing on the filter output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal; and performing nonlinear processing on the compensated speech signal to obtain an echo cancellation speech signal. Through the above method, speech compensation processing can be performed on the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal, so that the elimination of the speech signal in the echo cancellation process can be weakened, the loss of the speech signal can be reduced, and normal conversation with the user can be facilitated.

Description

Technical Field

[0001] This application belongs to the field of acoustic processing technology, and in particular relates to an echo cancellation method, apparatus, computer-readable storage medium and terminal equipment. Background Technology

[0002] In human-computer voice dialogue systems, the user's voice can be captured by a microphone, and the corresponding response can be broadcast through a speaker. To improve user experience, these two processes are usually performed simultaneously. However, if the user speaks while the voice is being broadcast, the captured result will be a mixture of the user's voice and the broadcast audio, leading to incorrect judgments and responses. Therefore, acoustic echo cancellation (AEC) is also required for the broadcast audio during the voice capture process.

[0003] However, using existing echo cancellation methods results in the loss of some of the user's voice after echo cancellation, making it impossible to accurately identify the user's speech content and affecting normal conversations with the user. Summary of the Invention

[0004] In view of this, embodiments of this application provide an echo cancellation method, apparatus, computer-readable storage medium, and terminal device to solve the problem that existing echo cancellation methods lose some of the user's speech, causing the terminal device to be unable to accurately identify the user's speech content and affecting normal dialogue with the user.

[0005] A first aspect of this application provides an echo cancellation method, which may include:

[0006] Obtain the reference alignment signal and the microphone alignment signal;

[0007] The reference alignment signal and the microphone alignment signal are filtered using a preset adaptive filter to obtain a filtered output signal;

[0008] Based on the reference alignment signal and the microphone alignment signal, the filtered output signal is subjected to speech compensation processing to obtain a compensated speech signal;

[0009] The compensated speech signal is subjected to nonlinear processing to obtain an echo-cancelled speech signal.

[0010] In one specific implementation of the first aspect, performing speech compensation processing on the filtered output signal based on the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal may include:

[0011] The reference alignment signal, the microphone alignment signal, and the filtered output signal are respectively subjected to frequency domain transformation to obtain the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal;

[0012] The filtering suppression component is determined based on the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal;

[0013] Extract the estimated speech component from the filtered and suppressed components;

[0014] The frequency domain compensation of the filtered output signal is performed based on the estimated speech components to obtain the compensated speech domain signal.

[0015] The compensated speech signal is obtained by performing a time-domain transformation on the compensated speech signal.

[0016] In one specific implementation of the first aspect, extracting the estimated speech component from the filtered suppression component may include:

[0017] The noise component is estimated by performing noise component estimation on the filtered and suppressed components to obtain the estimated noise component;

[0018] Echo component estimation is performed on the filtered and suppressed components to obtain the estimated echo components;

[0019] Based on the estimated noise component and the estimated echo component, the estimated speech component is extracted from the filtered suppression component.

[0020] In one specific implementation of the first aspect, estimating the echo components of the filtered suppression components to obtain estimated echo components may include:

[0021] Calculate the first correlation value between the reference alignment transformation signal and the microphone alignment transformation signal;

[0022] Calculate the second correlation value between the filtered output transformed signal and the microphone alignment transformed signal;

[0023] If the first correlation value is greater than a preset first threshold and the second correlation value is less than a preset second threshold, then the filtered suppression component and the estimated noise component are linearly fitted to obtain the estimated room impulse response and the estimated nonlinear component.

[0024] The estimated echo component is determined based on the estimated room impulse response and the estimated nonlinear component.

[0025] In one specific implementation of the first aspect, frequency domain transformations are performed on the reference alignment signal, the microphone alignment signal, and the filtered output signal to obtain the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal, respectively. This may include:

[0026] The reference alignment signal, the microphone alignment signal, and the filtered output signal are each framed to obtain a reference alignment signal frame, a microphone alignment signal frame, and a filtered output signal frame.

[0027] Short-time Fourier transforms are performed on the reference alignment signal frame, the microphone alignment signal frame, and the filtered output signal frame, respectively, to obtain the reference alignment transform signal, the microphone alignment transform signal, and the filtered output transform signal.

[0028] In one specific implementation of the first aspect, performing a time-domain transformation on the compensated speech signal to obtain the compensated speech signal may include:

[0029] Perform an inverse Fourier transform on the compensated speech audio domain signal to obtain the compensated speech time domain signal frame;

[0030] The compensated speech time-domain signal frames are subjected to frame stacking processing to obtain the compensated speech signal.

[0031] In one specific implementation of the first aspect, obtaining the reference alignment signal and the microphone alignment signal may include:

[0032] Acquire the reference input signal and the microphone input signal;

[0033] The reference input signal and the microphone input signal are time-domain aligned to obtain the reference aligned signal and the microphone aligned signal.

[0034] A second aspect of the embodiments of this application provides an echo cancellation device, which may include:

[0035] Alignment signal acquisition module, used to acquire reference alignment signal and microphone alignment signal;

[0036] An adaptive filtering module is used to filter the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal;

[0037] The voice compensation module is used to perform voice compensation processing on the filtered output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated voice signal;

[0038] The nonlinear processing module is used to perform nonlinear processing on the compensated speech signal to obtain an echo-cancelled speech signal.

[0039] In one specific implementation of the second aspect, the voice compensation module may include:

[0040] The frequency domain transformation submodule is used to perform frequency domain transformation on the reference alignment signal, the microphone alignment signal, and the filtered output signal respectively to obtain the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal;

[0041] The suppression component determination submodule is used to determine the filtering suppression component based on the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal;

[0042] The estimated speech component extraction submodule is used to extract the estimated speech component from the filtered and suppressed components;

[0043] The frequency domain compensation submodule is used to perform frequency domain compensation on the filtered output signal based on the estimated speech components to obtain a compensated speech domain signal.

[0044] The time-domain transformation submodule is used to perform time-domain transformation on the compensated speech audio domain signal to obtain the compensated speech signal.

[0045] In one specific implementation of the second aspect, the estimated speech component extraction submodule may include:

[0046] A noise component estimation unit is used to estimate the noise component of the filtered and suppressed component to obtain the estimated noise component.

[0047] The echo component estimation unit is used to estimate the echo components of the filtered and suppressed components to obtain the estimated echo components.

[0048] The estimated speech component extraction unit is used to extract the estimated speech component from the filtered suppression component based on the estimated noise component and the estimated echo component.

[0049] In one specific implementation of the second aspect, the estimated speech component extraction unit may include:

[0050] The first calculation subunit is used to calculate a first correlation value between the reference alignment transformation signal and the microphone alignment transformation signal;

[0051] The second calculation subunit is used to calculate a second correlation value between the filtered output transformed signal and the microphone alignment transformed signal;

[0052] A linear fitting subunit is used to perform linear fitting on the filtered suppression component and the estimated noise component if the first correlation value is greater than a preset first threshold and the second correlation value is less than a preset second threshold, so as to obtain the estimated room impulse response and the estimated nonlinear component.

[0053] An echo component estimation subunit is used to determine the estimated echo component based on the estimated room impulse response and the estimated nonlinear component.

[0054] In one specific implementation of the second aspect, the frequency domain transformation submodule may include:

[0055] The framing processing unit is used to framature the reference alignment signal, the microphone alignment signal and the filtered output signal respectively to obtain the reference alignment signal frame, the microphone alignment signal frame and the filtered output signal frame.

[0056] The Fourier transform unit is used to perform short-time Fourier transforms on the reference alignment signal frame, the microphone alignment signal frame, and the filtered output signal frame respectively to obtain the reference alignment transform signal, the microphone alignment transform signal, and the filtered output transform signal.

[0057] In one specific implementation of the second aspect, the time-domain transformation submodule may include:

[0058] The inverse Fourier transform unit is used to perform an inverse Fourier transform on the compensated speech audio domain signal to obtain a compensated speech time domain signal frame.

[0059] The frame stacking processing unit is used to perform frame stacking processing on the compensated speech time-domain signal frames to obtain the compensated speech signal.

[0060] In one specific implementation of the second aspect, the alignment signal acquisition module may include:

[0061] The input signal acquisition submodule is used to acquire the reference input signal and the microphone input signal;

[0062] The time-domain alignment submodule is used to perform time-domain alignment on the reference input signal and the microphone input signal to obtain the reference alignment signal and the microphone alignment signal.

[0063] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described echo cancellation methods.

[0064] A fourth aspect of this application provides a terminal device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described echo cancellation methods.

[0065] A fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to perform the steps of any of the above-described echo cancellation methods.

[0066] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment obtains a reference alignment signal and a microphone alignment signal; filters the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal; performs voice compensation processing on the filtered output signal based on the reference alignment signal and the microphone alignment signal to obtain a compensated voice signal; and performs nonlinear processing on the compensated voice signal to obtain an echo-cancelled voice signal. Through this application embodiment, voice compensation processing can be performed based on the reference alignment signal and the microphone alignment signal to obtain a compensated voice signal, thereby reducing the elimination of voice signal during echo cancellation, reducing voice signal loss, and facilitating normal dialogue between the terminal device and the user. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a schematic diagram of the AEC module in an embodiment of this application;

[0069] Figure 2 This is a flowchart of one embodiment of an echo cancellation method in this application.

[0070] Figure 3 This is a schematic diagram of the input and output of an adaptive filter;

[0071] Figure 4 This is a schematic diagram of the voice compensation module;

[0072] Figure 5 This is a structural diagram of one embodiment of an echo cancellation device according to the present application.

[0073] Figure 6 This is a schematic block diagram of a terminal device in an embodiment of this application. Detailed Implementation

[0074] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0075] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0076] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0077] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0078] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0079] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0080] In human-computer voice dialogue systems, the user's voice can be captured by a microphone, and the corresponding response can be broadcast through a speaker. To improve user experience, these two processes are usually performed simultaneously. However, if the user speaks while the voice is being broadcast, the captured result will be a mixture of the user's voice and the broadcast audio, leading to incorrect judgments and responses. Therefore, acoustic echo cancellation (AEC) is also required for the broadcast audio during the voice capture process.

[0081] However, using existing echo cancellation methods results in the loss of some of the user's voice after echo cancellation, causing the terminal device to be unable to accurately identify the user's speech content, thus affecting normal dialogue with the user.

[0082] In view of this, embodiments of this application provide an echo cancellation method, apparatus, computer-readable storage medium, and terminal device.

[0083] It should be noted that the subject of execution of the method in this application is a terminal device, which may include, but is not limited to, common computing devices in the prior art such as desktop computers, laptops, handheld computers, smart interactive screens, and robots.

[0084] Please see Figure 1 The echo cancellation method of this application can add a preset voice compensation module to the AEC module of the existing WebRTC framework, and use the voice compensation module to perform voice compensation processing on the filtered output signal, so as to more accurately recognize the user's speech content in the subsequent speech recognition process, thereby ensuring normal dialogue between the terminal device and the user.

[0085] Specifically, one embodiment of an echo cancellation method in this application may include... Figure 2 The steps shown are as follows:

[0086] Step S201: Obtain the reference alignment signal and the microphone alignment signal.

[0087] In this embodiment, after generating the corresponding response content and performing speech synthesis, the synthesized speech can be buffered in a preset buffer as a reference input signal during echo cancellation. During echo cancellation, the reference input signal x can be obtained through the buffer, and a microphone or other sound pickup device can be used to pick up the sound and obtain the microphone input signal y. The microphone input signal can consist of the user's speech s (the user's voice), ambient noise v, and the broadcast sound d (also known as the echo component) from the loudspeaker, i.e., y = (s + v) + d. Since the broadcast sound d captured by the sound pickup device is the result of the attenuation loss of the reference input signal x during acoustic propagation, and is related to factors such as room structure and the performance of the loudspeaker, we have d = x * h + d′. Here, h is the room impulse response, * is the convolution operation, and d′ is the nonlinear echo component.

[0088] It is important to understand that the reference input signal and the microphone input signal may not be aligned in the time domain. There may be a certain time delay between the reference input signal and the microphone input signal. If both are directly input into a preset adaptive filter (AF) for filtering, strong residual echo may occur, causing the adaptive filter to fail. Therefore, in this embodiment, the reference input signal x and the microphone input signal y can be aligned in the time domain to obtain a time-synchronized reference aligned signal x' and a microphone aligned signal y'.

[0089] Specifically, any common time-domain alignment method in the prior art can be used to align the reference input signal and the microphone input signal in the time domain. For example, the time delay of the reference input signal relative to the microphone input signal can be estimated by calculating the cross-correlation between the reference input signal and the microphone input signal, and then the two can be aligned in the time domain based on this time delay to obtain the reference-aligned signal and the microphone-aligned signal.

[0090] Step S202: Filter the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal.

[0091] In this embodiment, the preset adaptive filter can filter the reference alignment signal x' and the microphone alignment signal y' based on the correlation between the reference input signal x and the microphone input signal y, thereby suppressing the echo component in the microphone input signal and obtaining the filtered output signal s'.

[0092] Step S203: Based on the reference alignment signal and the microphone alignment signal, perform speech compensation processing on the filtered output signal to obtain the compensated speech signal.

[0093] It's important to understand that adaptive filters achieve good processing results when only the user is speaking (single-talk phase) or only the terminal device is broadcasting (echo phase). However, when the user speaks while the terminal device is broadcasting (dual-talk phase), both user speech and broadcast audio are present simultaneously. In this case, while adaptive filters can suppress the echo component (d), they also suppress some of the user's speech (s) and ambient noise (v). For details, please refer to [reference needed]. Figure 3 The output user voice s, ambient noise v, and broadcast sound d are compared with the input speaking sound. Environmental noise and broadcast audio In contrast, while the broadcast sound d is suppressed, the user's voice s and environmental noise v are also partially lost. This may cause the terminal device to be unable to accurately recognize the user's speech, affecting normal conversation with the user.

[0094] Therefore, in the embodiments of this application, the following can be utilized: Figure 4 The voice compensation module shown performs voice compensation processing on the filtered output signal to obtain a compensated voice signal. Specifically, step S203 may include the following process:

[0095] Step S2031: Perform frequency domain transformation on the reference alignment signal, microphone alignment signal and filter output signal respectively to obtain the reference alignment transformation signal, microphone alignment transformation signal and filter output transformation signal.

[0096] In this embodiment, the three time-domain signals—reference alignment signal, microphone alignment signal, and filtered output signal—can be converted to the linear frequency domain through frequency domain transformation to facilitate spectrum analysis. The frequency domain transformation may include framing and short-time Fourier transform.

[0097] Understandably, in practical applications, the input signal can be segmented into frames to obtain multiple analysis frames, and each analysis frame can be analyzed and processed frame by frame. In the embodiments of this application, the reference alignment signal x', the microphone alignment signal y', and the filtered output signal s' can be segmented into frames respectively to obtain the reference alignment signal frame. Microphone Alignment Signal Frame and filtered output signal frame Here, the next (NN) of the previous analysis frame can be used. s The current analysis frame is obtained by concatenating the sampled points with the currently input signal.

[0098]

[0099]

[0100]

[0101] In the formula, n is the sampling point number, k is the frame number, and N is the analysis frame length. S The number of sampling points for each frame of the input signal; where,

[0102] It should be noted that, to ensure the smoothness of the audio signal, the length of the analysis frame cannot be too large; here, the duration of each input signal frame can be set to 10 milliseconds. If the audio sampling rate is 16 kHz, then the number of sampling points N for each input signal frame is... S The value is 160, and the analysis frame length N is 256.

[0103] It is understood that the duration and audio sampling rate of each input signal frame can be customized and contextualized according to actual needs, and this application does not impose any restrictions on this.

[0104] Then, the reference alignment signal frames can be processed separately. Microphone Alignment Signal Frame and filtered output signal frame Performing a short-time Fourier transform yields the reference alignment transform signal X'(k,u), the microphone alignment transform signal Y'(k,u), and the filtered output transform signal S'(k,u), i.e.:

[0105]

[0106]

[0107]

[0108] Where u is the frequency point number, w(n) is a window function with a preset length of N, and j is the imaginary unit.

[0109] In this embodiment, a hybrid flat-top Hanning window can be used to window the input signal. The first and second segments of this window can form a window containing (NN) S The Hanning window has 1000 points, and the specific window function is as follows:

[0110]

[0111] It is understandable that other window functions can be set according to actual needs, and this application does not limit this.

[0112] Step S2032: Determine the filtering suppression component based on the reference alignment transformation signal, the microphone alignment transformation signal, and the filter output transformation signal.

[0113] The filtering suppression component is the signal suppressed by the adaptive filter. Specifically, the suppression component is [Y'(k,u)-S'(k,u)].

[0114] Step S2033: Extract the estimated speech component from the filtered and suppressed components.

[0115] It should be understood that an adaptive filter can suppress echo components, ambient noise, and some speech in the microphone input signal. Therefore, the filtering and suppression components can include noise components, echo components, and speech components. In this embodiment, the estimated speech component in the filtering and suppression components can be extracted based on the estimated noise component and the estimated echo component. Specifically, step S2033 can include the following process:

[0116] Step S2033a: Estimate the noise component of the filtered suppression component to obtain the estimated noise component.

[0117] In the embodiments of this application, the noise spectral density can be estimated for environmental noise to obtain the estimated noise spectral density.

[0118] It's important to understand that power spectral smoothing can be applied to the filtered suppression component to facilitate the analysis and estimation of noise components. Specifically, if the filtered suppression component is [Y'(k,u)-S'(k,u)], then its power spectral density P(k,u) is:

[0119] P(k, u) = |Y′(k, u)| 2 -|S′(k, u)| 2

[0120] For P(k,u), frequency domain smoothing yields:

[0121]

[0122] In the formula, P'(k,u) is the frequency-smoothed power spectral density of the power spectrum P(k,u) after frequency-domain smoothing, and α is the frequency-domain smoothing coefficient. Its value can be set according to actual needs and scenarios. This application does not limit it; here, it can preferably be set to 0.5.

[0123] Then, P'(k,u) is smoothed in the time domain to obtain the smoothed power spectral density P″(k,u):

[0124] P″(k,u)=β·P′(k,u)+(1-β)·P″(k-1,u)

[0125] Wherein, β is a preset first time-domain smoothing coefficient, the value of which can be set according to actual needs and contextualization, and this application does not limit it; here, it can preferably be set to 0.2.

[0126] It is important to understand that within a specific frequency band of a noisy speech signal, a large portion of the frame signals typically have low energy and contain significant noise components. Therefore, in this embodiment, a minimum temporal smoothed power spectral density can be searched for to perform noise spectrum estimation. Specifically, after obtaining r consecutive estimates of the temporal smoothed power density P″(k,u), the minimum temporal smoothed power spectral density P is updated. min "(k,u):"

[0127]

[0128]

[0129] Among them, P″ tmp (k,u) represents the minimum spectral estimate between frames, which can be considered as a temporary buffer variable, with an initial value of P″(0,u); r is the number of frames for inter-group state updates. The minimum power spectrum needs to be re-estimated every r frames to improve accuracy, and its value is related to the k-frame number, specifically:

[0130]

[0131] When the noise is estimated to have a power spectral density P V When the energy of a frequency point in (k,u) is relatively small, the noise estimate P at that frequency point can be obtained. V Update (k,u) if necessary, otherwise keep the original estimate, i.e.:

[0132]

[0133] Where γ is the smoothing factor. K P Minimum time-domain smoothed power spectral density P min The threshold for determining the relationship between "(k,u) and the time-domain smoothed power spectral density P"(k,u) can be set in a specific and contextual way according to actual needs, and this application does not limit it; here, it can preferably be set to 0.2.

[0134] Based on this, the estimated noise components can be obtained.

[0135] Step S2033b: Echo component estimation is performed on the filtered suppression component to obtain the estimated echo component.

[0136] It's important to understand that dual-talk detection can be performed based on the reference alignment transformation signal and the microphone alignment transformation signal to determine if the system is in an echo phase. If it is, echo component estimation can be performed on the filtered suppression components to obtain the estimated echo components. Specifically, when the correlation between the reference alignment transformation signal and the microphone alignment transformation signal is high, and the correlation between the filtered output transformation signal and the microphone alignment transformation signal is low, it can be assumed that the pickup device has not captured the user's speech, and this phase can be considered an echo phase.

[0137] In this embodiment of the application, a first correlation value |ρ| between the reference alignment transform signal and the microphone alignment transform signal can be calculated. XY | It can also calculate the second correlation value between the filtered output transformed signal and the microphone-aligned transformed signal |ρ SY | and determine whether it is in the echo stage based on the first and second relevance values. The calculation method for the relevance values ​​can be set according to the actual situation; this application does not specifically limit it. Here, the formulas for calculating the first and second relevance values ​​can be:

[0138]

[0139]

[0140] Here, Cov{·,·} represents the covariance operation between two objects, Var{·} represents the variance operation, and Corr{·,·} represents the correlation coefficient between the two objects. In the above calculation, the square is calculated first and then the square root is calculated to avoid the case where the correlation coefficient is negative.

[0141] If the first relevance value is |ρ XY |Greater than the preset first threshold ρ up And the second correlation value |ρ SY | Less than the preset second threshold ρ down If so, it can be considered that the system is in the echo phase. The first and second thresholds can be customized and set according to specific circumstances; this embodiment does not limit this.

[0142] During the echo phase, a linear fit can be performed on the filtered suppression component and the estimated noise component to obtain the estimated room impulse response and the estimated nonlinear component. Specifically, the filtered suppression component mainly contains most of the echo component and some noise components, thus:

[0143]

[0144] Where H(k,u) is the frequency domain room impulse response, and D′(k,u) is the spectrum of the nonlinear echo component. (Abbreviation) Then there is;

[0145] U(k,u)=X′(k,u)·H(k,u)+D′(k,u)

[0146] Here, the least squares method can be used to perform linear fitting on each frequency point u in the filtered suppression component and the estimated noise component, respectively, resulting in:

[0147]

[0148]

[0149] in, Let X′(k,u) be the average value of the accumulated frames at frequency point u. Let U′(k,u) be the average value of accumulated frames at frequency point u. To estimate the room's impulse response, To estimate the nonlinear components, the fitting residuals can then be obtained. for:

[0150]

[0151] The smaller the fitting residual, the better the fitting effect.

[0152] Based on the estimated room shock response and estimating nonlinear components The estimated echo components can be obtained as follows:

[0153] Step S2033c: Extract the estimated speech component from the filtered suppression component based on the estimated noise component and the estimated echo component.

[0154] In the embodiments of this application, the estimated speech component in the filtered suppression component can be extracted. Specifically:

[0155]

[0156] Where max{·} represents taking the maximum value, the above formula can avoid estimating speech components. It is a negative value.

[0157] Step S2034: Perform frequency domain compensation on the filtered output signal based on the estimated speech components to obtain the compensated speech domain signal.

[0158] In this embodiment of the application, the estimated speech components can be used as a basis. Frequency domain compensation is performed on the corresponding filtered output signal S′(k,u), and the specific formula is as follows:

[0159]

[0160] in, It is a compensation signal in the speech audio domain.

[0161] After that, you can... By performing time-domain smoothing, the compensated speech domain signal can be obtained.

[0162]

[0163] Wherein, λ is a preset second time-domain smoothing coefficient, the value of which can be set according to actual needs and contextualization, and this application does not limit it; here, it can preferably be set to 0.3.

[0164] Step S2035: Perform time-domain transformation on the compensated speech audio domain signal to obtain the compensated speech signal.

[0165] In this embodiment, a time-domain transform can be performed on the smoothed compensated speech audio domain signal to obtain the compensated speech signal. Specifically, an inverse Fourier transform can be performed on the compensated speech audio domain signal to obtain the compensated speech time-domain signal frame.

[0166]

[0167] To avoid spectral leakage, the compensated speech time-domain signal frame can be windowed and then overlaid with the previous compensated speech time-domain signal frame, i.e.:

[0168]

[0169] In the formula, w(n) is a preset window function, and a hybrid flat-top Hanning window can be used here. During frame overlay processing, the preceding (NN) frames of the current compensated speech time-domain signal are... S The values ​​of the ) points are compared with the next (NN) points of the previous compensated speech time-domain signal frame. S The values ​​of the first N points are superimposed; the first N points of the current compensated speech time-domain signal frame are superimposed. S Output the point values, and then (NN) S The ) point values ​​are used for the frame stacking of the next compensated speech time-domain signal frame, so the output result s′ of the current compensated speech time-domain signal frame is... out for:

[0170]

[0171] Based on this, the compensated speech signal s′ can be obtained. out .

[0172] Step S204: Perform nonlinear processing on the compensated speech signal to obtain the echo-cancelled speech signal.

[0173] In this embodiment of the application, after obtaining the compensated speech signal, a preset nonlinear processing module can be used to perform nonlinear processing on the compensated speech signal to eliminate residual nonlinear echoes and obtain echo-cancelled speech signal s″.

[0174] It should be noted that after obtaining the echo-cancelled speech signal, speech recognition can be performed on the echo-cancelled speech signal according to the processing flow in the existing technology, and corresponding response content can be generated based on the recognition result to continue the subsequent human-computer dialogue.

[0175] In summary, this application embodiment obtains a reference alignment signal and a microphone alignment signal; filters the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal; performs voice compensation processing on the filtered output signal based on the reference alignment signal and the microphone alignment signal to obtain a compensated voice signal; and performs nonlinear processing on the compensated voice signal to obtain an echo-cancelled voice signal. Through this application embodiment, voice compensation processing can be performed based on the reference alignment signal and the microphone alignment signal to obtain a compensated voice signal, thereby reducing the elimination of voice signal during echo cancellation, minimizing voice signal loss, and facilitating normal dialogue with users.

[0176] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0177] Corresponding to the echo cancellation method described in the above embodiments, Figure 5 This diagram illustrates a structural diagram of an embodiment of an echo cancellation device provided in this application.

[0178] In this embodiment of the application, an echo cancellation device may include:

[0179] Alignment signal acquisition module 501 is used to acquire reference alignment signal and microphone alignment signal;

[0180] The adaptive filtering module 502 is used to filter the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal;

[0181] The voice compensation module 503 is used to perform voice compensation processing on the filtered output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated voice signal;

[0182] The nonlinear processing module 504 is used to perform nonlinear processing on the compensated speech signal to obtain an echo-cancelled speech signal.

[0183] In one specific implementation of this application embodiment, the voice compensation module may include:

[0184] The frequency domain transformation submodule is used to perform frequency domain transformation on the reference alignment signal, the microphone alignment signal, and the filtered output signal respectively to obtain the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal;

[0185] The suppression component determination submodule is used to determine the filtering suppression component based on the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal;

[0186] The estimated speech component extraction submodule is used to extract the estimated speech component from the filtered and suppressed components;

[0187] The frequency domain compensation submodule is used to perform frequency domain compensation on the filtered output signal based on the estimated speech components to obtain a compensated speech domain signal.

[0188] The time-domain transformation submodule is used to perform time-domain transformation on the compensated speech audio domain signal to obtain the compensated speech signal.

[0189] In one specific implementation of this application embodiment, the estimated speech component extraction submodule may include:

[0190] A noise component estimation unit is used to estimate the noise component of the filtered and suppressed component to obtain the estimated noise component.

[0191] The echo component estimation unit is used to estimate the echo components of the filtered and suppressed components to obtain the estimated echo components.

[0192] The estimated speech component extraction unit is used to extract the estimated speech component from the filtered suppression component based on the estimated noise component and the estimated echo component.

[0193] In one specific implementation of this application embodiment, the estimated speech component extraction unit may include:

[0194] The first calculation subunit is used to calculate a first correlation value between the reference alignment transformation signal and the microphone alignment transformation signal;

[0195] The second calculation subunit is used to calculate a second correlation value between the filtered output transformed signal and the microphone alignment transformed signal;

[0196] A linear fitting subunit is used to perform linear fitting on the filtered suppression component and the estimated noise component if the first correlation value is greater than a preset first threshold and the second correlation value is less than a preset second threshold, so as to obtain the estimated room impulse response and the estimated nonlinear component.

[0197] An echo component estimation subunit is used to determine the estimated echo component based on the estimated room impulse response and the estimated nonlinear component.

[0198] In one specific implementation of this application embodiment, the frequency domain transformation submodule may include:

[0199] The framing processing unit is used to framature the reference alignment signal, the microphone alignment signal and the filtered output signal respectively to obtain the reference alignment signal frame, the microphone alignment signal frame and the filtered output signal frame.

[0200] The Fourier transform unit is used to perform short-time Fourier transforms on the reference alignment signal frame, the microphone alignment signal frame, and the filtered output signal frame respectively to obtain the reference alignment transform signal, the microphone alignment transform signal, and the filtered output transform signal.

[0201] In one specific implementation of this application embodiment, the time-domain transformation submodule may include:

[0202] The inverse Fourier transform unit is used to perform an inverse Fourier transform on the compensated speech audio domain signal to obtain a compensated speech time domain signal frame.

[0203] The frame stacking processing unit is used to perform frame stacking processing on the compensated speech time-domain signal frames to obtain the compensated speech signal.

[0204] In one specific implementation of this application embodiment, the alignment signal acquisition module may include:

[0205] The input signal acquisition submodule is used to acquire the reference input signal and the microphone input signal;

[0206] The time-domain alignment submodule is used to perform time-domain alignment on the reference input signal and the microphone input signal to obtain the reference alignment signal and the microphone alignment signal.

[0207] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0208] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0209] Figure 6 A schematic block diagram of a terminal device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0210] like Figure 6 As shown, the terminal device 6 in this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the various echo cancellation method embodiments described above, for example... Figure 2 Steps S201 to S204 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5 The functions of modules 501 to 504 are shown.

[0211] For example, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 62 in the terminal device 6.

[0212] Those skilled in the art will understand that Figure 6 This is merely an example of terminal device 6 and does not constitute a limitation on terminal device 6. It may include more or fewer components than shown, or combine certain components, or different components. For example, terminal device 6 may also include input / output devices, network access devices, buses, etc.

[0213] The processor 60 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0214] The memory 61 can be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. The memory 61 can also be an external storage device of the terminal device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 6. Furthermore, the memory 61 can include both internal and external storage units of the terminal device 6. The memory 61 is used to store the computer program and other programs and data required by the terminal device 6. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0215] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0216] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0217] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0218] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0219] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0220] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0221] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0222] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An echo cancellation method, characterized in that, include: Obtain the reference alignment signal and the microphone alignment signal; The reference alignment signal and the microphone alignment signal are filtered using a preset adaptive filter to obtain a filtered output signal; Frequency domain transformations are performed on the reference alignment signal, the microphone alignment signal, and the filtered output signal to obtain the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal, respectively; the filtering suppression component is determined based on the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal. The noise component is estimated by performing noise component estimation on the filtered and suppressed components to obtain the estimated noise component; Echo component estimation is performed on the filtered and suppressed components to obtain the estimated echo components; Based on the estimated noise component and the estimated echo component, extract the estimated speech component from the filtered suppression component; The frequency domain compensation of the filtered output signal is performed based on the estimated speech components to obtain the compensated speech domain signal. The compensated speech signal is transformed in the time domain to obtain the compensated speech signal; The compensated speech signal is subjected to nonlinear processing to obtain an echo-cancelled speech signal.

2. The echo cancellation method according to claim 1, characterized in that, Echo component estimation is performed on the filtered and suppressed components to obtain the estimated echo components, including: Calculate the first correlation value between the reference alignment transformation signal and the microphone alignment transformation signal; Calculate the second correlation value between the filtered output transformed signal and the microphone alignment transformed signal; If the first correlation value is greater than a preset first threshold and the second correlation value is less than a preset second threshold, then the filtered suppression component and the estimated noise component are linearly fitted to obtain the estimated room impulse response and the estimated nonlinear component. The estimated echo component is determined based on the estimated room impulse response and the estimated nonlinear component.

3. The echo cancellation method according to claim 1, characterized in that, Frequency domain transformations are performed on the reference alignment signal, the microphone alignment signal, and the filtered output signal, respectively, to obtain the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal, including: The reference alignment signal, the microphone alignment signal, and the filtered output signal are each framed to obtain a reference alignment signal frame, a microphone alignment signal frame, and a filtered output signal frame. Short-time Fourier transforms are performed on the reference alignment signal frame, the microphone alignment signal frame, and the filtered output signal frame, respectively, to obtain the reference alignment transform signal, the microphone alignment transform signal, and the filtered output transform signal.

4. The echo cancellation method according to claim 3, characterized in that, The compensated speech signal is obtained by performing a time-domain transformation on the compensated speech signal, including: Perform an inverse Fourier transform on the compensated speech audio domain signal to obtain the compensated speech time domain signal frame; The compensated speech time-domain signal frames are subjected to frame stacking processing to obtain the compensated speech signal.

5. The echo cancellation method according to any one of claims 1 to 4, characterized in that, Acquire the reference alignment signal and microphone alignment signal, including: Acquire the reference input signal and the microphone input signal; The reference input signal and the microphone input signal are time-domain aligned to obtain the reference aligned signal and the microphone aligned signal.

6. An echo cancellation device, characterized in that, include: The alignment signal acquisition module is used to acquire the reference alignment signal and the microphone alignment signal; An adaptive filtering module is used to filter the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal; The voice compensation module is used to perform frequency domain transformation on the reference alignment signal, the microphone alignment signal, and the filtered output signal respectively to obtain the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal; and to determine the filtering suppression component based on the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal. The noise component is estimated by performing noise component estimation on the filtered and suppressed components to obtain the estimated noise component; Echo component estimation is performed on the filtered and suppressed components to obtain the estimated echo components; Based on the estimated noise component and the estimated echo component, extract the estimated speech component from the filtered suppression component; The frequency domain compensation of the filtered output signal is performed based on the estimated speech components to obtain the compensated speech domain signal. The compensated speech signal is transformed in the time domain to obtain the compensated speech signal; The nonlinear processing module is used to perform nonlinear processing on the compensated speech signal to obtain an echo-cancelled speech signal.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the echo cancellation method as described in any one of claims 1 to 5.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the echo cancellation method as described in any one of claims 1 to 5.

Images