Echo cancellation method and apparatus, computer readable storage medium, and terminal device

Abstract

An echo cancellation method and apparatus, a computer readable storage medium, and a terminal device. The method comprises: acquiring a reference alignment signal and a microphone alignment signal (S201); using a preset adaptive filter to filter the reference alignment signal and the microphone alignment signal to obtain a filtered output signal (S202); on the basis of the reference alignment signal and the microphone alignment signal, performing speech compensation processing on the filtered output signal to obtain a compensated speech signal (S203); and performing nonlinear processing on the compensated speech signal to obtain a speech signal having undergone echo cancellation (S204). According to the method, speech compensation processing can be performed on the basis of the reference alignment signal and the microphone alignment signal to obtain the compensated speech signal, thereby weakening the cancellation of a speech signal during echo cancellation, reducing the loss of the speech signal, and facilitating the normal conversation with a user.

Description

Echo cancellation method, device, computer-readable storage medium, and terminal device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 31, 2023, with application number 202311129179.6 and invention name “Echo Cancellation Method, Device, Computer-Readable Storage Medium and Terminal Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application belongs to the field of acoustic processing technology, and in particular relates to an echo cancellation method, apparatus, computer-readable storage medium, and terminal device. Background Art

[0003] In a human-computer voice dialogue system, the user's voice can be captured by a pickup device, and the corresponding reply content can be broadcasted by a speaker device. To improve the user experience, these two processes are usually carried out simultaneously. However, if the user speaks during the voice broadcast, the user's voice and the broadcast audio will be mixed, resulting in incorrect judgment and response. Therefore, during the voice pickup process, Acoustic Echo Cancellation (AEC) is also required for the broadcast audio.

[0004] However, after the echo cancellation is performed using the echo cancellation method in the prior art, part of the user's speech voice will be lost, resulting in an inability to accurately recognize the user's speech content, affecting the normal conversation with the user. Technical issues

[0005] In view of this, the embodiments of the present application provide an echo cancellation method, apparatus, computer-readable storage medium, and terminal device to solve the problem that the echo cancellation method in the prior art will lose some of the user's speaking voice, resulting in the terminal device being unable to accurately recognize the user's speaking content, affecting normal conversation with the user. Technical Solutions

[0006] A first aspect of an embodiment of the present application provides an echo cancellation method, which may include:

[0007] obtaining a reference alignment signal and a microphone alignment signal;

[0008] Filtering the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal;

[0009] performing speech compensation processing on the filtered output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal;

[0010] The compensated speech signal is subjected to nonlinear processing to obtain an echo-cancelled speech signal.

[0011] In a specific implementation of the first aspect, performing speech compensation processing on the filtered output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal may include:

[0012] Performing frequency domain transformation on the reference alignment signal, the microphone alignment signal, and the filtered output signal respectively to obtain a reference alignment transformed signal, a microphone alignment transformed signal, and a filtered output transformed signal;

[0013] determining a filter suppression component based on the reference alignment transformed signal, the microphone alignment transformed signal, and the filtered output transformed signal;

[0014] extracting an estimated speech component from the filtered suppression component;

[0015] Performing frequency domain compensation on the filtered output signal according to the estimated speech component to obtain a compensated speech frequency domain signal;

[0016] Performing a time domain transform on the compensated speech frequency domain signal to obtain the compensated speech signal.

[0017] In a specific implementation of the first aspect, extracting the estimated speech component from the filtered suppression component may include:

[0018] performing noise component estimation on the filtered suppression component to obtain an estimated noise component;

[0019] performing echo component estimation on the filtered suppression component to obtain an estimated echo component;

[0020] The estimated speech component in the filtered suppression component is extracted based on the estimated noise component and the estimated echo component.

[0021] In a specific implementation of the first aspect, performing echo component estimation on the filtered suppression component to obtain the estimated echo component may include:

[0022] Calculating a first correlation value between the reference alignment transformation signal and the microphone alignment transformation signal;

[0023] calculating a second correlation value between the filtered output transformed signal and the microphone alignment transformed signal;

[0024] If the first correlation value is greater than a preset first threshold and the second correlation value is less than a preset second threshold, linearly fitting the filtered suppression component and the estimated noise component to obtain an estimated room impulse response and an estimated nonlinear component;

[0025] The estimated echo component is determined according to the estimated room impulse response and the estimated nonlinear component.

[0026] In a specific implementation of the first aspect, performing frequency domain transform on the reference alignment signal, the microphone alignment signal, and the filtered output signal to obtain a reference alignment transformed signal, a microphone alignment transformed signal, and a filtered output transformed signal may include:

[0027] framing the reference alignment signal, the microphone alignment signal, and the filtered output signal to obtain a reference alignment signal frame, a microphone alignment signal frame, and a filtered output signal frame;

[0028] Short-time Fourier transform is performed on the reference alignment signal frame, the microphone alignment signal frame and the filtered output signal frame respectively to obtain the reference alignment transformation signal, the microphone alignment transformation signal and the filtered output transformation signal.

[0029] In a specific implementation of the first aspect, performing a time domain transform on the compensated speech signal to obtain the compensated speech signal may include:

[0030] Performing an inverse Fourier transform on the compensated speech audio-domain signal to obtain a compensated speech time-domain signal frame;

[0031] Perform frame stacking processing on the compensated speech time domain signal frame to obtain the compensated speech signal.

[0032] In a specific implementation of the first aspect, obtaining the reference alignment signal and the microphone alignment signal may include:

[0033] Obtain a reference input signal and a microphone input signal;

[0034] The reference input signal and the microphone input signal are time-domain aligned to obtain the reference aligned signal and the microphone aligned signal.

[0035] A second aspect of the embodiments of the present application provides an echo cancellation device, which may include:

[0036] an alignment signal acquisition module, configured to acquire a reference alignment signal and a microphone alignment signal;

[0037] an adaptive filtering module, configured to filter the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal;

[0038] a speech compensation module, configured to perform speech compensation processing on the filtered output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal;

[0039] The nonlinear processing module is used to perform nonlinear processing on the compensated speech signal to obtain an echo-cancelled speech signal.

[0040] In a specific implementation of the second aspect, the voice compensation module may include:

[0041] a frequency domain transformation submodule, configured to perform frequency domain transformation on the reference alignment signal, the microphone alignment signal, and the filtered output signal, respectively, to obtain a reference alignment transformation signal, a microphone alignment transformation signal, and a filtered output transformation signal;

[0042] a suppression component determination submodule, configured to determine a filtering suppression component based on the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal;

[0043] An estimated speech component extraction submodule, configured to extract the estimated speech component from the filtered suppression component;

[0044] a frequency domain compensation submodule, configured to perform frequency domain compensation on the filtered output signal according to the estimated speech component to obtain a compensated speech frequency domain signal;

[0045] The time domain transformation submodule is configured to perform time domain transformation on the compensated speech audio domain signal to obtain the compensated speech signal.

[0046] In a specific implementation of the second aspect, the estimated speech component extraction submodule may include:

[0047] a noise component estimation unit, configured to perform noise component estimation on the filtered suppression component to obtain an estimated noise component;

[0048] an echo component estimation unit, configured to perform echo component estimation on the filtered suppression component to obtain an estimated echo component;

[0049] The estimated speech component extraction unit is configured to extract the estimated speech component from the filtered suppression component based on the estimated noise component and the estimated echo component.

[0050] In a specific implementation of the second aspect, the estimated speech component extraction unit may include:

[0051] A first calculation subunit, configured to calculate a first correlation value between the reference alignment transformation signal and the microphone alignment transformation signal;

[0052] a second calculation subunit, configured to calculate a second correlation value between the filtered output transformation signal and the microphone alignment transformation signal;

[0053] a linear fitting subunit, configured to perform linear fitting on the filtered suppression component and the estimated noise component to obtain an estimated room impulse response and an estimated nonlinear component if the first correlation value is greater than a preset first threshold and the second correlation value is less than a preset second threshold;

[0054] The estimated echo component determination subunit is configured to determine the estimated echo component according to the estimated room impulse response and the estimated nonlinear component.

[0055] In a specific implementation of the second aspect, the frequency domain transformation submodule may include:

[0056] A framing processing unit, configured to frame the reference alignment signal, the microphone alignment signal, and the filtered output signal, respectively, to obtain a reference alignment signal frame, a microphone alignment signal frame, and a filtered output signal frame;

[0057] The Fourier transform unit is used to perform short-time Fourier transform on the reference alignment signal frame, the microphone alignment signal frame and the filtered output signal frame respectively to obtain the reference alignment transformation signal, the microphone alignment transformation signal and the filtered output transformation signal.

[0058] In a specific implementation of the second aspect, the time domain transformation submodule may include:

[0059] an inverse Fourier transform unit, configured to perform an inverse Fourier transform on the compensated speech audio-domain signal to obtain a compensated speech time-domain signal frame;

[0060] The frame stacking processing unit is used to perform frame stacking processing on the compensated speech time domain signal frame to obtain the compensated speech signal.

[0061] In a specific implementation of the second aspect, the alignment signal acquisition module may include:

[0062] An input signal acquisition submodule, used to obtain a reference input signal and a microphone input signal;

[0063] The time domain alignment submodule is configured to perform time domain alignment on the reference input signal and the microphone input signal to obtain the reference alignment signal and the microphone alignment signal.

[0064] A third aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned echo cancellation methods are implemented.

[0065] A fourth aspect of an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned echo cancellation methods when executing the computer program.

[0066] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the steps of any one of the above-mentioned echo cancellation methods. Beneficial effects

[0067] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application obtain a reference alignment signal and a microphone alignment signal; use a preset adaptive filter to filter the reference alignment signal and the microphone alignment signal to obtain a filtered output signal; perform voice compensation processing on the filtered output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated voice signal; perform nonlinear processing on the compensated voice signal to obtain an echo-canceled voice signal. Through the embodiments of the present application, voice compensation processing can be performed according to the reference alignment signal and the microphone alignment signal to obtain a compensated voice signal, thereby weakening the elimination of the voice signal during the echo cancellation process, reducing the loss of the voice signal, and facilitating normal conversation between the terminal device and the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0069] FIG1 is a schematic diagram of an AEC module in an embodiment of the present application;

[0070] FIG2 is a flow chart of an embodiment of an echo cancellation method in an embodiment of the present application;

[0071] FIG3 is a schematic diagram of the input and output of the adaptive filter;

[0072] FIG4 is a schematic diagram of a speech compensation module;

[0073] FIG5 is a structural diagram of an embodiment of an echo cancellation device according to an embodiment of the present application;

[0074] FIG6 is a schematic block diagram of a terminal device in an embodiment of the present application. Modes for Carrying Out the Invention

[0075] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0076] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0077] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0078] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0079] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0080] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0081] In a human-computer voice dialogue system, the user's voice can be captured by a pickup device, and the corresponding reply content can be broadcasted by a speaker device. To improve the user experience, these two processes are usually carried out simultaneously. However, if the user speaks during the voice broadcast, the user's voice and the broadcast audio will be mixed, resulting in incorrect judgment and response. Therefore, during the voice pickup process, Acoustic Echo Cancellation (AEC) is also required for the broadcast audio.

[0082] However, after echo cancellation is performed using the echo cancellation method in the prior art, part of the user's speaking voice will be lost, resulting in the terminal device being unable to accurately recognize the user's speaking content, affecting the normal conversation with the user.

[0083] In view of this, embodiments of the present application provide an echo cancellation method, apparatus, computer-readable storage medium, and terminal device.

[0084] It should be noted that the execution subject of the method of the present application is a terminal device, specifically, it may include but is not limited to desktop computers, notebooks, PDAs, smart interactive screens, robots and other common computing devices in the prior art.

[0085] Please refer to Figure 1. The echo cancellation method of the present application can add a preset voice compensation module to the AEC module of the existing WebRTC framework, and use the voice compensation module to perform voice compensation processing on the filtered output signal, so that in the subsequent voice recognition processing process, the user's speech content can be more accurately recognized to ensure normal conversation between the terminal device and the user.

[0086] Specifically, an embodiment of an echo cancellation method in the embodiment of the present application may include the steps shown in FIG2 :

[0087] Step S201: Acquire a reference alignment signal and a microphone alignment signal.

[0088] In an embodiment of the present application, after generating the corresponding reply content and performing speech synthesis, the synthesized speech can be cached in a preset buffer as a reference input signal for the echo cancellation process. During echo cancellation, the reference input signal x can be obtained through the buffer, and a microphone or other sound pickup device can be used to pick up the sound to obtain a microphone input signal y. The microphone input signal can be composed of the user's voice s (the user's speaking voice), the ambient noise v, and the announcement sound d of the speaker device (also known as the echo portion), that is, y = (s + v) + d. Since the announcement sound d captured by the sound pickup device is the result of attenuation loss of the reference input signal x during acoustic propagation, it is related to factors such as the room structure and the performance of the speaker device. Therefore, d = x*h + d′; where h is the room impulse response, * is the convolution operation, and d′ is the nonlinear echo component.

[0089] It should be understood that the reference input signal and the microphone input signal may not be aligned in the time domain. There may be a certain time delay between the reference input signal and the microphone input signal. If both are directly input into the preset adaptive filter (AF) for filtering, a strong residual echo may appear, causing the adaptive filter to fail. Therefore, in an embodiment of the present application, the reference input signal x and the microphone input signal y can be aligned in the time domain to obtain a reference alignment signal x' and a microphone alignment signal y' that are synchronized in the time domain.

[0090] Specifically, the reference input signal and the microphone input signal can be time-domain aligned using any common time-domain alignment method known in the art. For example, the time delay of the reference input signal relative to the microphone input signal can be estimated by calculating the cross-correlation between the reference input signal and the microphone input signal. The time delay can then be used to time-domain align the two signals to obtain a reference-aligned signal and a microphone-aligned signal.

[0091] Step S202: Filter the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal.

[0092] In an embodiment of the present application, the preset adaptive filter can filter the reference alignment signal x' and the microphone alignment signal y' based on the correlation between the reference input signal x and the microphone input signal y, thereby suppressing the echo part in the microphone input signal and obtaining a filtered output signal s'.

[0093] Step S203: Perform speech compensation processing on the filtered output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal.

[0094] It should be understood that when only the user is speaking (single-talk phase) or only the terminal device is broadcasting (echo phase), the use of adaptive filters can achieve better processing effects; but when the user speaks while the terminal device is broadcasting (dual-talk phase), the user voice and the broadcast sound will coexist. At this time, the adaptive filter can suppress the echo part d, but also suppresses part of the user's speaking sound s and environmental noise v. For details, please refer to Figure 3. The output user voice s, environmental noise v and broadcast sound d are the same as the input speaking sound. Environmental noise and announcement sound In comparison, the announcement sound d is suppressed, but at the same time the user voice s and environmental noise v are also partially lost, which may cause the terminal device to be unable to accurately recognize the user's speech content, affecting the normal conversation with the user.

[0095] Therefore, in the embodiment of the present application, the voice compensation module shown in FIG4 can be used to perform voice compensation processing on the filtered output signal to obtain a compensated voice signal. Specifically, step S203 may include the following process:

[0096] Step S2031 : performing frequency domain transformation on the reference alignment signal, the microphone alignment signal, and the filtered output signal respectively to obtain a reference alignment transformed signal, a microphone alignment transformed signal, and a filtered output transformed signal.

[0097] In an embodiment of the present application, the three time domain signals, namely the reference alignment signal, the microphone alignment signal and the filtered output signal, can be converted to a linear frequency domain by frequency domain transformation to facilitate spectrum analysis. The frequency domain transformation may include framing and short-time Fourier transform.

[0098] It is understandable that in practical applications, the input signal can be framed to obtain multiple analysis frames, and each analysis frame can be analyzed and processed frame by frame. In the embodiment of the present application, the reference alignment signal x', the microphone alignment signal y' and the filtered output signal s' can be framed to obtain the reference alignment signal frame Microphone alignment signal frame And filter output signal frame Here, the back (NN) of the previous analysis frame can be s ) sampling points are spliced ​​with the current input signal to obtain the current analysis frame:

[0099] Where n is the sampling point number, k is the frame number, N is the analysis frame length, and N S is the number of sampling points of the input signal per frame;

[0100] It should be noted that in order to ensure the stability of the speech signal, the length of the analysis frame cannot be too large. Here, the duration of each frame of input signal can be set to 10 milliseconds. If the audio sampling rate is 16 kHz, the sampling point N of each frame of input signal is S is 160, and the analysis frame length N is 256.

[0101] It is understandable that the duration of each frame of the input signal and the audio sampling rate can be specific and situational settings according to actual needs, and this application does not limit this.

[0102] Afterwards, the reference alignment signal frames can be Microphone alignment signal frame And filter output signal frame Perform short-time Fourier transform to obtain the reference alignment transformation signal X'(k,u), the microphone alignment transformation signal Y'(k,u) and the filtered output transformation signal S'(k,u), namely:

[0103] Wherein, u is the frequency point number, w(n) is the preset window function with a length of N, and j is the imaginary unit.

[0104] In the embodiment of the present application, a hybrid flat-top Hanning window can be used to perform windowing processing on the input signal, and the first and second segments of the window can just form a window containing (NN S ) points in the Hanning window. The specific window function is as follows:

[0105] It is understandable that other window functions may be set according to actual needs, and this application does not limit this.

[0106] Step S2032: Determine a filtering suppression component according to the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal.

[0107] The filtered suppression component is a signal suppressed by the adaptive filter. Specifically, the suppression component is [Y'(k,u)-S'(k,u)].

[0108] Step S2033: extract the estimated speech component from the filtered suppression component.

[0109] It should be understood that the adaptive filter can suppress the echo portion, ambient noise, and part of the speech in the microphone input signal. Therefore, the filtered suppression component may include a noise component, an echo component, and a speech component. In the embodiment of the present application, the estimated speech component in the filtered suppression component can be extracted based on the estimated noise component and the estimated echo component. Specifically, step S2033 may include the following process:

[0110] Step S2033a: perform noise component estimation on the filtered suppression component to obtain an estimated noise component.

[0111] In the embodiment of the present application, the noise spectral density of the environmental noise may be estimated to obtain the noise estimation spectral density.

[0112] It should be understood that the power spectrum of the filter suppression component can be smoothed to facilitate the analysis and estimation of the noise component. Specifically, the filter suppression component is [Y'(k,u)-S'(k,u)], and its power spectrum density P(k,u) is: P(k,u)=|Y′(k,u)| 2 -|S′(k,u)| 2

[0113] Perform frequency domain smoothing on P(k,u), and we can get:

[0114] Wherein, P'(k,u) is the frequency domain smoothed power spectrum density after the power spectrum P(k,u) is frequency domain smoothed, α is the frequency domain smoothing coefficient, and its value can be specific and situational according to actual needs. This application does not limit this; here, it can be preferably set to 0.5.

[0115] After that, P'(k,u) is smoothed in the time domain to obtain the time domain smoothed power spectrum density P″(k,u): P″(k,u)=β·P′(k,u)+(1-β)·P″(k-1,u)

[0116] Among them, β is a preset first time domain smoothing coefficient, and its value can be specific and situational according to actual needs, and this application does not limit this; here, it can be preferably set to 0.2.

[0117] It should be understood that in a specific frequency band of a noisy speech signal, most of the frame signals usually have low energy and contain more noise components. Therefore, in the embodiment of the present application, the minimum time-domain smoothed power spectrum density can be searched to perform noise spectrum estimation. Specifically, after obtaining r consecutive estimates of the time-domain smoothed power density P″(k,u), the minimum time-domain smoothed power spectrum density P can be updated. min ″(k,u):

[0118] Among them, P tmp (k,u) is the minimum spectrum estimate between frames, which can be regarded as a temporary cache variable. Its initial value is P″(0,u); r is the number of inter-group state update frames. The minimum power spectrum is re-estimated every r frames to improve accuracy. Its value is related to the k frame number, specifically:

[0119] When the noise estimation power spectral density P V When the frequency energy in (k,u) is small, the noise estimation value P of the frequency can be V (k,u) is updated, otherwise, the original estimated value is maintained, that is:

[0120] Among them, γ is the smoothing factor, K P is the minimum time domain smoothed power spectral density P min The specific value of the judgment threshold of ″(k,u) and the time-domain smoothed power spectrum density P″(k,u) can be specific and situationally set according to actual needs, and this application does not limit this; here, it can be preferably set to 0.2.

[0121] Based on this, the estimated noise component can be obtained

[0122] Step S2033b: perform echo component estimation on the filtered suppression component to obtain an estimated echo component.

[0123] It should be understood that double talk detection can be performed based on the reference alignment transform signal and the microphone alignment transform signal to determine whether the current state is the echo phase. If the current state is the echo phase, the echo component of the filtered suppression component can be estimated to obtain an estimated echo component. Specifically, when the reference alignment transform signal and the microphone alignment transform signal have a high correlation, and the filtered output transform signal and the microphone alignment transform signal have a low correlation, it can be determined that the sound pickup device has not captured the user's voice, and the current phase can be considered the echo phase.

[0124] In the embodiment of the present application, a first correlation value |ρ| between the reference alignment transformation signal and the microphone alignment transformation signal can be calculated. XY |, a second correlation value |ρ can also be calculated between the filtered output transformation signal and the microphone alignment transformation signal SY |, and determine whether it is in the echo stage based on the first correlation value and the second correlation value. The calculation method of the correlation value can be set according to actual conditions, and this application does not specifically limit this. Here, the calculation formula of the first correlation value and the second correlation value can be:

[0125] Where Cov{·,·} represents the covariance of two objects, Var{·} represents the variance, and Corr{·,·} represents the correlation coefficient of two objects. In the above calculation, performing the square root before the square root calculation can avoid the possibility of negative correlation values.

[0126] If the first correlation value |ρ XY | is greater than the preset first threshold ρ up , and the second correlation value |ρ SY | is less than the preset second threshold ρ down , it can be considered that the echo phase is in progress. The first threshold and the second threshold can be set specifically and contextually according to actual conditions, and the present application embodiment does not limit this.

[0127] In the echo stage, the filtered suppression component and the estimated noise component can be linearly fitted to obtain the estimated room impulse response and the estimated nonlinear component. Specifically, the filtered suppression component mainly contains most of the echo component and some noise components, so:

[0128] Where H(k,u) is the room impulse response in the frequency domain, and D′(k,u) is the spectrum of the nonlinear echo component. Then there is; U(k,u)=X′(k,u)·H(k,u)+D′(k,u)

[0129] Here, the least squares method can be used to perform linear fitting on each frequency point u in the filtered suppression component and the estimated noise component, respectively, as follows:

[0130] in, is the average value of X′(k,u) accumulated at frequency point u, is the average value of the accumulated frames of U′(k,u) at frequency point u, To estimate the room impulse response, To estimate the nonlinear component; then, the fitting residual can be obtained for:

[0131] If the fitting residual is smaller, the fitting effect is better.

[0132] Based on the estimated room impulse response and estimate the nonlinear components The estimated echo component is

[0133] Step S2033c: extract the estimated speech component in the filtered suppression component based on the estimated noise component and the estimated echo component.

[0134] In the embodiment of the present application, the estimated speech component in the filtered suppression component can be extracted Specifically:

[0135] Among them, max{·} represents the maximum value. The above formula can avoid estimating the speech component. is a negative value.

[0136] Step S2034: Perform frequency domain compensation on the filtered output signal according to the estimated speech component to obtain a compensated speech frequency domain signal.

[0137] In the embodiment of the present application, the estimated speech component The corresponding filtered output signal S′(k,u) is compensated in the frequency domain. The specific formula is:

[0138] in, is the compensated speech frequency domain signal.

[0139] Afterwards, you can Perform time domain smoothing to obtain the compensated speech frequency domain signal after time domain smoothing

[0140] Among them, λ is a preset second time domain smoothing coefficient, and its value can be specific and situational according to actual needs, and this application does not limit this; here, it can be preferably set to 0.3.

[0141] Step S2035: Perform time domain transformation on the compensated speech signal to obtain a compensated speech signal.

[0142] In the embodiment of the present application, the smoothed compensation speech frequency domain signal can be transformed into the time domain to obtain the compensation speech signal. Specifically, the compensation speech frequency domain signal can be transformed into the time domain to obtain the compensation speech time domain signal frame.

[0143] To avoid spectrum leakage, the compensated speech time domain signal frame can be windowed and then overlapped with the previous compensated speech time domain signal frame, that is:

[0144] Where w(n) is the preset window function, and a hybrid flat-top Hanning window can be used here. In the frame stacking process, the previous (NN) of the current compensated speech time domain signal frame S) point value and the last (NN) of the last compensated speech time domain signal frame S ) point values ​​are superimposed; the first N of the current compensation speech time domain signal frame S Point value output, then (NN S ) point values ​​are used for the next compensation of the speech time domain signal frame frame stack, so the output result s′ of the current compensation speech time domain signal frame out for:

[0145] Based on this, the compensated speech signal s′ can be obtained out .

[0146] Step S204: Perform nonlinear processing on the compensated speech signal to obtain an echo-cancelled speech signal.

[0147] In the embodiment of the present application, after obtaining the compensated speech signal, a preset nonlinear processing module may be used to perform nonlinear processing on the compensated speech signal to eliminate the residual nonlinear echo and obtain the echo-cancelled speech signal s″.

[0148] It should be noted that after obtaining the echo-cancelled voice signal, voice recognition can be performed on the echo-cancelled voice signal according to the processing flow in the prior art, and corresponding reply content can be generated according to the recognition result to continue to complete the subsequent human-computer dialogue.

[0149] In summary, the embodiments of the present application obtain a reference alignment signal and a microphone alignment signal; use a preset adaptive filter to filter the reference alignment signal and the microphone alignment signal to obtain a filtered output signal; perform speech compensation processing on the filtered output signal based on the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal; and perform nonlinear processing on the compensated speech signal to obtain an echo-canceled speech signal. Through the embodiments of the present application, speech compensation processing can be performed based on the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal, thereby weakening the elimination of the speech signal during the echo cancellation process, reducing the loss of the speech signal, and facilitating normal conversations with users.

[0150] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0151] Corresponding to the echo cancellation method described in the above embodiment, FIG5 shows a structural diagram of an embodiment of an echo cancellation device provided in an embodiment of the present application.

[0152] In an embodiment of the present application, an echo cancellation device may include:

[0153] An alignment signal acquisition module 501 is configured to acquire a reference alignment signal and a microphone alignment signal;

[0154] An adaptive filtering module 502 is configured to filter the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal;

[0155] a speech compensation module 503, configured to perform speech compensation processing on the filtered output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal;

[0156] The non-linear processing module 504 is configured to perform non-linear processing on the compensated speech signal to obtain an echo-cancelled speech signal.

[0157] In a specific implementation of the embodiment of the present application, the voice compensation module may include:

[0158] a frequency domain transformation submodule, configured to perform frequency domain transformation on the reference alignment signal, the microphone alignment signal, and the filtered output signal, respectively, to obtain a reference alignment transformation signal, a microphone alignment transformation signal, and a filtered output transformation signal;

[0159] a suppression component determination submodule, configured to determine a filtering suppression component based on the reference alignment transformation signal, the microphone alignment transformation signal, and the filtered output transformation signal;

[0160] An estimated speech component extraction submodule, configured to extract the estimated speech component from the filtered suppression component;

[0161] a frequency domain compensation submodule, configured to perform frequency domain compensation on the filtered output signal according to the estimated speech component to obtain a compensated speech frequency domain signal;

[0162] The time domain transformation submodule is configured to perform time domain transformation on the compensated speech audio domain signal to obtain the compensated speech signal.

[0163] In a specific implementation of the embodiment of the present application, the estimated speech component extraction submodule may include:

[0164] a noise component estimation unit, configured to perform noise component estimation on the filtered suppression component to obtain an estimated noise component;

[0165] an echo component estimation unit, configured to perform echo component estimation on the filtered suppression component to obtain an estimated echo component;

[0166] The estimated speech component extraction unit is configured to extract the estimated speech component from the filtered suppression component based on the estimated noise component and the estimated echo component.

[0167] In a specific implementation of the embodiment of the present application, the estimated speech component extraction unit may include:

[0168] A first calculation subunit, configured to calculate a first correlation value between the reference alignment transformation signal and the microphone alignment transformation signal;

[0169] a second calculation subunit, configured to calculate a second correlation value between the filtered output transformation signal and the microphone alignment transformation signal;

[0170] a linear fitting subunit, configured to perform linear fitting on the filtered suppression component and the estimated noise component to obtain an estimated room impulse response and an estimated nonlinear component if the first correlation value is greater than a preset first threshold and the second correlation value is less than a preset second threshold;

[0171] The estimated echo component determination subunit is configured to determine the estimated echo component according to the estimated room impulse response and the estimated nonlinear component.

[0172] In a specific implementation of the embodiment of the present application, the frequency domain transformation submodule may include:

[0173] A framing processing unit, configured to frame the reference alignment signal, the microphone alignment signal, and the filtered output signal, respectively, to obtain a reference alignment signal frame, a microphone alignment signal frame, and a filtered output signal frame;

[0174] The Fourier transform unit is used to perform short-time Fourier transform on the reference alignment signal frame, the microphone alignment signal frame and the filtered output signal frame respectively to obtain the reference alignment transformation signal, the microphone alignment transformation signal and the filtered output transformation signal.

[0175] In a specific implementation of the embodiment of the present application, the time domain transformation submodule may include:

[0176] an inverse Fourier transform unit, configured to perform an inverse Fourier transform on the compensated speech audio-domain signal to obtain a compensated speech time-domain signal frame;

[0177] The frame stacking processing unit is used to perform frame stacking processing on the compensated speech time domain signal frame to obtain the compensated speech signal.

[0178] In a specific implementation of the embodiment of the present application, the alignment signal acquisition module may include:

[0179] An input signal acquisition submodule, used to obtain a reference input signal and a microphone input signal;

[0180] The time domain alignment submodule is configured to perform time domain alignment on the reference input signal and the microphone input signal to obtain the reference alignment signal and the microphone alignment signal.

[0181] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0182] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0183] Figure 6 shows a schematic block diagram of a terminal device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0184] As shown in FIG6 , the terminal device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the aforementioned echo cancellation method embodiments, such as steps S201 to S204 shown in FIG2 . Alternatively, when the processor 60 executes the computer program 62, it implements the functions of the modules / units in the aforementioned apparatus embodiments, such as the functions of modules 501 to 504 shown in FIG5 .

[0185] Exemplarily, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 62 in the terminal device 6.

[0186] Those skilled in the art will understand that Figure 6 is merely an example of the terminal device 6 and does not constitute a limitation on the terminal device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device 6 may also include input and output devices, network access devices, buses, etc.

[0187] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0188] The memory 61 can be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. The memory 61 can also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 6. Furthermore, the memory 61 can also include both an internal storage unit of the terminal device 6 and an external storage device. The memory 61 is used to store the computer program and other programs and data required by the terminal device 6. The memory 61 can also be used to temporarily store data that has been output or is about to be output.

[0189] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0190] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0191] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0192] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0193] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0194] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0195] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0196] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An echo cancellation method, characterized in that: include: obtaining a reference alignment signal and a microphone alignment signal; Filtering the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal; According to the reference alignment signal and the microphone alignment signal, performing speech compensation processing on the filtered output signal to obtain a compensated speech signal; The compensated speech signal is processed nonlinearly to obtain an echo-cancelled speech signal.

2. The echo cancellation method according to claim 1, characterized in that: According to the reference alignment signal and the microphone alignment signal, the filtered output signal is subjected to speech compensation processing to obtain a compensated speech signal, including: Performing frequency domain transformation on the reference alignment signal, the microphone alignment signal and the filtered output signal respectively to obtain a reference alignment transformation signal, a microphone alignment transformation signal and a filtered output transformation signal; determining a filter suppression component based on the reference alignment transformation signal, the microphone alignment transformation signal, and the filter output transformation signal; extracting an estimated speech component from the filtered suppression component; Performing frequency domain compensation on the filtered output signal according to the estimated speech component to obtain a compensated speech frequency domain signal; Performing a time domain transformation on the compensated speech frequency domain signal to obtain the compensated speech signal.

3. The echo cancellation method according to claim 2, characterized in that: Extracting the estimated speech component in the filtering suppression component comprises: performing noise component estimation on the filtered suppression component to obtain an estimated noise component; performing echo component estimation on the filtered suppression component to obtain an estimated echo component; The estimated speech component in the filtering suppression component is extracted based on the estimated noise component and the estimated echo component.

4. The echo cancellation method according to claim 3, characterized in that: Estimating the echo component of the filtered suppression component to obtain an estimated echo component includes: Calculating a first correlation value between the reference alignment transformation signal and the microphone alignment transformation signal; Calculating a second correlation value between the filtered output transformation signal and the microphone alignment transformation signal; If the first correlation value is greater than a preset first threshold value, and the second correlation value is less than a preset second threshold value, linearly fitting the filtered suppression component and the estimated noise component to obtain an estimated room impulse response and an estimated nonlinear component; The estimated echo component is determined according to the estimated room impulse response and the estimated nonlinear component.

5. The echo cancellation method according to claim 2, characterized in that: Performing frequency domain transformation on the reference alignment signal, the microphone alignment signal and the filter output signal respectively to obtain a reference alignment transformation signal, a microphone alignment transformation signal and a filter output transformation signal, including: Respectively framing the reference alignment signal, the microphone alignment signal and the filtered output signal to obtain a reference alignment signal frame, a microphone alignment signal frame and a filtered output signal frame; The reference alignment signal frame, the microphone alignment signal frame and the filter output signal frame are respectively subjected to short-time Fourier transform to obtain the reference alignment transformation signal, the microphone alignment transformation signal and the filter output transformation signal.

6. The echo cancellation method according to claim 5, characterized in that: Performing a time domain transformation on the compensated speech frequency domain signal to obtain the compensated speech signal comprises: Performing inverse Fourier transform on the compensated speech audio domain signal to obtain a compensated speech time domain signal frame; Perform frame stacking processing on the compensated speech time domain signal frame to obtain the compensated speech signal.

7. The echo cancellation method according to any one of claims 1 to 6, characterized in that: Acquire reference alignment signals and microphone alignment signals, including: Obtain a reference input signal and a microphone input signal; The reference input signal and the microphone input signal are aligned in time domain to obtain the reference alignment signal and the microphone alignment signal.

8. An echo cancellation device, characterized in that: include: An alignment signal acquisition module, used to acquire a reference alignment signal and a microphone alignment signal; An adaptive filtering module, used for filtering the reference alignment signal and the microphone alignment signal using a preset adaptive filter to obtain a filtered output signal; A speech compensation module, used for performing speech compensation processing on the filtered output signal according to the reference alignment signal and the microphone alignment signal to obtain a compensated speech signal; The non-linear processing module is used to perform non-linear processing on the compensated speech signal to obtain an echo-cancelled speech signal.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the echo cancellation method according to any one of claims 1 to 7 are implemented.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the echo cancellation method according to any one of claims 1 to 7 are implemented.

Images