Echo cancellation method, electronic device, computer program product and storage medium

By using the target filter to simulate the echo signal in the echo cancellation technology and determining the convergence step in combination with the error signal, the problems of poor stability and insufficient adaptability in the prior art are solved, and a more efficient echo cancellation effect is achieved.

CN119091902APending Publication Date: 2024-12-06ANT BLOCKCHAIN TECHNOLOGY (SHANGHAI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411315846.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing echo cancellation techniques have problems such as poor stability, slow convergence, sensitivity to noise, lack of flexibility and insufficient adaptability, especially in environments where signal characteristics change rapidly or noise levels are high.

Method used

The remote signal is simulated by the target filter to obtain the echo estimate signal, and the target convergence step is determined based on the error signal, and the filter is adjusted to achieve dynamic adjustment, improve convergence speed and stability, and reduce the impact of noise.

Benefits of technology

It improves the stability and convergence speed of the filter, enhances the adaptability to signal changes, reduces the sensitivity of noise to the filter, and improves the overall performance of echo cancellation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091902A_ABST
    Figure CN119091902A_ABST
Patent Text Reader

Abstract

One or more embodiments of the present specification provide an echo cancellation method, an electronic device, a computer program product and a storage medium. The method comprises the steps that echo signal simulation is carried out on a target far-end signal segment in a target far-end signal through a target filter, an echo estimation signal is obtained, and the target far-end signal is an audio signal played through an audio playing device in a target environment; filtering the echo estimation signal in a target near-end signal segment to obtain an error signal, the target near-end signal segment being a sound signal collected by an audio collection device in a target environment and aligned with a target far-end signal segment; determining a target convergence step length based on the echo estimation signal and the error signal; and adjusting the convergence step size of the target filter to the target convergence step size to obtain a target step size filter, and performing echo cancellation on the target near-end signal segment through the target step size filter to obtain a pure near-end signal segment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of audio processing technology, and more particularly, to an echo cancellation method, electronic device, computer program product, and storage medium. Background Art

[0002] Acoustic Echo Cancelling eliminates the noise generated by the feedback path between the microphone and the speaker due to the air through sound wave interference. In layman's terms, echo cancellation is to eliminate the echo emitted by the machine itself and propagated in the environment without affecting the sound transmitted from the outside. For example, in a duplex call scenario, the sound emitted by the speaker from the far end is eliminated. Otherwise, the sound emitted by the speaker will be collected by the microphone and mixed with the sound signal generated by the near-end speaker and transmitted to the far end, which will cause the sound collected by the far end to contain the audio transmitted by itself. This echo is a fatal difference in experience for some communication devices.

[0003] In the related art, the far-end signal is filtered by an adaptive filter whose step size is dynamically adjusted according to the error signal to simulate the echo generated in the current environment. Next, the simulated echo is subtracted from the near-end signal to obtain a processed signal, which is finally determined as a pure near-end signal without echo.

[0004] However, this method has the following disadvantages when performing echo cancellation:

[0005] 1. Poor stability: Relying solely on the error signal to adjust the step size may cause the filter to be unstable. If the error signal changes greatly in a short period of time, it may cause the step size to change frequently and significantly, thus affecting the stability of the filter.

[0006] 2. Slow convergence. Simply relying on the error signal to adjust the step size may lead to slower convergence. In some cases, when the error signal is small, the filter may take longer to converge to the optimal solution.

[0007] 3. Too sensitive to noise. The error signal may contain noise components. Adjusting the step size only according to the error signal may make the filter too sensitive to noise, causing the update of the filter coefficients to be affected by the noise, thereby affecting the performance of the filter.

[0008] 4. Lack of flexibility. Considering only the error signal may cause the filter to perform poorly in some cases, especially in environments where the signal characteristics change rapidly or the noise level is high.

[0009] 5. Poor adaptability to signal changes. If the filter relies only on the error signal to adjust the step size, it may be difficult to adapt to rapid changes in signal characteristics, especially in double-talk (both parties talking at the same time) or when the background noise level changes greatly.

[0010] It can be seen that the echo cancellation method in the related art has the above technical problems. Summary of the invention

[0011] In view of this, one or more embodiments of the present specification provide an echo cancellation method, an electronic device, a computer program product, and a storage medium to overcome the technical problems existing in the related art.

[0012] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions:

[0013] According to a first aspect of one or more embodiments of this specification, a method for echo cancellation is provided, including:

[0014] Performing echo signal simulation on a target far-end signal segment in a target far-end signal through a target filter to obtain an echo estimation signal, wherein the target far-end signal is an audio signal played through an audio playback device in a target environment;

[0015] filtering the echo estimation signal in a target near-end signal segment to obtain an error signal, wherein the target near-end signal segment is a sound signal collected by an audio collection device in the target environment and aligned with the target far-end signal segment;

[0016] Determining a target convergence step size based on the echo estimation signal and the error signal;

[0017] The convergence step size of the target filter is adjusted to the target convergence step size to obtain a target step size filter, and the target near-end signal segment is echo-cancelled by the target step size filter to obtain a pure near-end signal segment.

[0018] According to a second aspect of one or more embodiments of this specification, an electronic device is provided, including:

[0019] processor;

[0020] a memory for storing processor-executable instructions;

[0021] The processor implements the method described in the first aspect above by running the executable instructions.

[0022] According to a third aspect of one or more embodiments of this specification, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0023] According to a fourth aspect of one or more embodiments of the present specification, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0024] It can be seen that in the technical solution of the present specification, the target convergence step length corresponding to the target far-end signal segment is determined based on the echo estimation signal and the error signal, and the convergence step length can be dynamically adjusted to improve the convergence speed and stability of the filter. In addition, although the error signal is still used as one of the considerations in the process of determining the target convergence step length, the echo estimation signal is also added as one of the considerations while retaining the error signal. Therefore, even if the noise components that may be contained in the error signal still cause noise to the target filter, the weight of the influence of the noise components on the step length can be reduced through the newly added echo estimation signal, thereby achieving the purpose of reducing the influence of the noise components that may be contained in the error signal on the target filter and the sensitivity of the target filter to noise. At the same time, by determining the target convergence step length and adjusting the target filter in real time The convergence step size of is the target convergence step size, so that the target step size filter determined in real time can better adapt to the echo characteristics of the echo corresponding to the target far-end signal segment, especially when the signal characteristics change, it can quickly adjust to adapt to the new echo path, thereby improving the overall performance of echo elimination for the target near-end signal segment (that is, flexibility and adaptability to signal changes), and thus can effectively overcome the problems of poor stability, slow convergence speed, excessive sensitivity to noise, lack of flexibility and poor adaptability to signal changes in the adaptive filter for far-end signal filtering that only uses the error signal to dynamically adjust the step size in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a schematic diagram of an echo cancellation process of a dual recording system provided by an exemplary embodiment.

[0026] Figure 2 It is a flow chart of an echo cancellation method provided by an exemplary embodiment.

[0027] Figure 3 is a flowchart of an echo cancellation method provided by another exemplary embodiment.

[0028] Figure 4 This is a logical diagram of echo cancellation slicing provided by an exemplary embodiment.

[0029] Figure 5 It is a structural schematic diagram of a device provided by an exemplary embodiment.

[0030] Figure 6 It is a block diagram of an echo cancellation device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0031] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, execution and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] Figure 1 The following is a schematic diagram of an echo cancellation process of a dual recording system provided by an exemplary embodiment. The dual recording system is a system that uses artificial intelligence technology to achieve automatic management and service of audio and video recording (dual recording). It is widely used in industries including but not limited to: finance, insurance, etc. The dual recording system can ensure the compliance and transparency of the sales and service processes. Figure 1 As shown, the dual recording system may include: a loudspeaker (i.e., an optional audio playback device) and a microphone (i.e., an optional audio acquisition device) provided in the current environment, and the far-end signal (i.e., the audio signal to be played) may be transmitted to the dual recording system, and then the far-end signal will be transferred to the loudspeaker in the specified current environment, and the far-end signal will be played through the loudspeaker, thereby generating a corresponding broadcast sound, so that the personnel in the current environment can hear the broadcast sound, and after hearing the broadcast sound, the personnel can respond through human voice (recorded as environmental human voice). Optionally, when the sender of the above-mentioned far-end signal inputs text data, the text data can be converted and processed by TTS (text to speech, Text ToSpeech) to obtain the far-end signal. However, when the broadcast sound is played in the current environment, an echo will be generated after being blocked by walls and other obstacles in the current environment, and the echo will be collected by the microphone together with the above-mentioned environmental human voice to obtain a near-end signal as the original sound of the scene. It can be seen that the near-end signal will contain echoes, which will lead to the problem that the ambient human voice to be collected is unclear. Therefore, it is necessary to perform echo cancellation (ie, AEC, Acoustic Echo Cancellation) on the near-end signal.

[0033] In the related art, echo cancellation is performed in the following manner: the far-end signal is filtered by an adaptive filter whose step size is dynamically adjusted according to the error signal to simulate the echo generated in the current environment (i.e., the echo estimation signal). Then, the echo generated by the simulation is subtracted from the near-end signal to obtain a processed signal, and the processed signal is finally identified as a pure near-end signal containing only the ambient human voice and no echo (i.e., the audio without TTS). Furthermore, after obtaining the pure near-end signal, ASR (Automatic Speech Recognition) speech transcription can be performed on it to automatically convert the above-mentioned pure near-end signal from a speech signal to text data.

[0034] However, when using this method for echo cancellation, the adaptive filter method that dynamically adjusts the step size only according to the error signal may have the following disadvantages:

[0035] 1) Poor stability. Relying solely on the error signal to adjust the step size may cause the filter to be unstable. If the error signal changes greatly in a short period of time, it may cause the step size to change frequently and significantly, thus affecting the stability of the filter.

[0036] 2) Slow convergence. Simply relying on the error signal to adjust the step size may lead to slower convergence. In some cases, when the error signal is small, the filter may take longer to converge to the optimal solution.

[0037] 3) Too sensitive to noise. The error signal may contain noise components. Adjusting the step size only according to the error signal may make the filter too sensitive to noise, causing the update of the filter coefficients to be affected by the noise, thereby affecting the performance of the filter.

[0038] 4) Lack of flexibility: considering only the error signal may cause the filter to perform poorly in some cases, especially in environments with rapidly changing signal characteristics or high noise levels.

[0039] 5) Poor adaptability to signal changes. If the filter relies solely on the error signal to adjust the step size, it may have difficulty adapting to rapid changes in signal characteristics, especially in cases of double talk (both parties speaking at the same time) or large changes in background noise levels.

[0040] In view of the above problems in the related art, this specification provides an improved echo cancellation method, which is described in detail below with reference to the accompanying drawings.

[0041] Figure 2 FIG. 1 is a flow chart of an echo cancellation method provided by an exemplary embodiment. Figure 2 As shown, the method may include the following steps:

[0042] Step S202 , performing echo signal simulation on a target far-end signal segment in a target far-end signal through a target filter to obtain an echo estimation signal, wherein the target far-end signal is an audio signal played through an audio playback device in a target environment.

[0043] In this embodiment, the target far-end signal may be an audio signal played by an audio playback device in the target environment, that is, an audio signal from the far end. The target far-end signal may be an audio signal determined by the far-end device according to the sound signal received by the near end (that is, the target environment) to respond to the sound signal, or it may be an audio signal sent by the far-end user. The target far-end signal segment may be a signal segment corresponding to the time domain length processed by the target filter each time in the target far-end signal. Optionally, the time domain length of the target far-end signal segment processed by the filter each time may be selected according to the accuracy to be filtered, which is not limited here. When the target far-end signal segment is played by the audio playback device in the target environment, an echo will be formed in the target environment. The target filter here may be a Kalman filter or an adaptive minimum mean square filter, which is used to simulate the acoustic characteristics of the path from the audio playback device to the audio acquisition device. The target filter is used to learn and replicate the transmission characteristics in the acoustic environment, so that the filtered far-end signal, that is, the echo estimation signal, can simulate the echo generated in the actual environment. After the target far-end signal segment is subjected to echo signal simulation through the target filter, an echo estimation signal for simulating the echo generated by the target far-end signal segment reflected back through the acoustic path in the target environment can be obtained.

[0044] As an optional implementation, step S102 may be implemented by simulating the target far-end signal segment in the target far-end signal through the target filter to obtain the echo estimation signal through the following steps:

[0045] Determine a target filtering mode corresponding to the target remote signal segment according to energy change information corresponding to the target remote signal segment, wherein the filtering mode includes: linear filtering and nonlinear filtering;

[0046] Simulating the target far-end signal segment through multiple candidate filters corresponding to the target filtering mode to obtain a candidate echo estimation signal output by each candidate filter;

[0047] A candidate echo estimation signal with the largest signal energy among all candidate echo estimation signals is determined as the echo estimation signal.

[0048] In this embodiment, after obtaining the target far-end signal segment, the energy change information of the target far-end signal segment can be determined, that is, the energy distribution of the signal at different time points can be determined. After obtaining the energy change information, it can be determined whether the energy change information is a linear change or a nonlinear change. When it is determined that the energy change information is a linear change, the target filtering mode is linear filtering, and when it is determined that the energy change information is a nonlinear change, the target filtering mode is nonlinear filtering. After determining the target filtering mode, multiple candidate filters corresponding to the target filtering mode can be determined. If the target filtering mode is linear filtering, the corresponding multiple candidate filters can be filters applied to linear filtering, such as: Kalman filter or adaptive minimum mean square filter applied to linear filtering; if the target filtering mode is nonlinear filtering, the corresponding multiple candidate filters can be filters applied to nonlinear filtering, such as: Kalman filter or adaptive minimum mean square filter applied to nonlinear filtering. After determining the multiple candidate filters corresponding to the target filtering mode, the target far-end signal segment can be simulated by each candidate filter to obtain the candidate echo estimation signal output by each candidate filter. After all candidate echo estimation signals are determined, the optimal value of all candidate echo estimation signals is taken and used as the echo estimation signal. In this embodiment, the candidate echo estimation signal with the largest signal energy among all candidate echo estimation signals is determined as the echo estimation signal, that is, the signal energies of all candidate echo estimation signals are compared (for example, the total signal energy or the average signal energy is compared), and then the candidate echo estimation signal with the largest signal energy is determined as the echo estimation signal.

[0049] Figure 3 FIG. 1 is a flow chart of an echo cancellation method provided by another exemplary embodiment. Figure 3 As shown, as an optional implementation, before filtering the echo estimation signal in the target near-end signal segment to obtain the error signal in step S204, the following steps may be performed in advance to determine the target near-end signal segment:

[0050] Step S302: acquiring a near-end signal to be processed in a target environment.

[0051] In this embodiment, a microphone or other audio collection device can be used to collect sound signals in the target environment to obtain a near-end signal to be processed, which includes the sound of the user speaking in the target environment and the echo caused by the target far-end signal after being played in the target environment.

[0052] Step S304: determine a designated near-end signal corresponding to the target far-end signal from the near-end signals to be processed.

[0053] After acquiring the near-end signal to be processed, the signal segment corresponding to the target far-end signal can be found in the near-end signal to be processed according to the timestamp and content of the target far-end signal. This signal generally contains at least the echo caused by the target far-end signal after being played in the target environment.

[0054] As an optional implementation, a designated near-end signal corresponding to a target far-end signal is determined in the near-end signal to be processed, including: determining the time interval between the start timestamp and the end timestamp of the target far-end signal as the original time interval of the target far-end signal; advancing the start timestamp of the original time interval according to a preset duration, and delaying the end timestamp of the original time interval to obtain an extended time interval corresponding to the original time interval; and determining the audio segment corresponding to the extended time interval in the near-end signal to be processed as the designated near-end signal. In this embodiment, the target far-end signal may be a continuous audio signal, the start timestamp of the target far-end signal may be the start playback time of the target far-end signal, the end timestamp may be the end playback time of the target far-end signal, and the time interval between the start timestamp and the end timestamp is determined as the original time interval of the target far-end signal. For example, Figure 4 is a schematic diagram of an echo cancellation slice logic provided by an exemplary embodiment, such as Figure 4 As shown, Figure 4 When the remote signal played between moments ab in is the target remote signal, moment a is the starting timestamp, moment b is the ending timestamp, and [a, b] is the original time interval corresponding to the target remote signal. After determining the original time interval of the target remote signal, since there may be a situation where the callback of the target remote signal from the remote end is relatively slow, which will cause the starting timestamp of the target remote signal to be later than the actual starting moment of the audio playback device playing the target remote signal, the starting timestamp can be advanced by a preset time length. In addition, since there will be an echo after the target remote signal is played in the target environment, and the echo is obviously lagging behind the ending timestamp of the target remote signal, the ending timestamp can be delayed by a preset time length. By advancing the starting timestamp of the original time interval by the preset time length, and delaying the ending timestamp of the original time interval, an extended time interval corresponding to the original time interval is obtained. For example, such as Figure 4 As shown, Figure 4For example, the far-end signal played between moments ab in the video is the target far-end signal. When the preset duration is 300ms, after the original time interval corresponding to [a, b] is extended, the extended time period that can be obtained is [a-300ms, b+300ms]. After the time interval to be extended is determined, the target far-end signal may generate an echo within the extended time interval. Therefore, the audio segment corresponding to the extended time interval in the near-end signal to be processed can be determined as the designated near-end signal, that is, the designated near-end signal is an audio signal that may contain the echo generated by the target far-end signal. Through the method of this embodiment, the designated near-end signal that may have an echo can be effectively determined in the near-end signal to be processed, so that the echo can be completely eliminated later, which is beneficial to improving the purity of the processed audio signal.

[0055] Step S306: align the target far-end signal and the designated near-end signal by determining the cross-correlation coefficient between the target far-end signal and the designated near-end signal, and determine the target near-end signal corresponding to the target far-end signal in the designated near-end signal.

[0056] After determining the target far-end signal and the designated near-end signal, the mutual correlation coefficient can be determined by a direct calculation method or a fast calculation method based on Fourier fast transform. The mutual correlation coefficient is an indicator of the similarity between the two signals, which can help determine the exact time delay of the target far-end signal reaching the microphone. The time delay refers to the time required for the sound wave generated by the target far-end signal after being processed by the speaker to propagate to the microphone from leaving the speaker. After calculating the above time delay, the time delay can be used to align the target far-end signal and the designated near-end signal, that is, the time axis of the target far-end signal is adjusted by the time delay so that the target far-end signal is aligned with the designated near-end signal in time, so as to ensure that the subsequent target step filter can accurately simulate the acoustic path from the audio playback device to the audio acquisition device, and the target far-end signal and its corresponding echo signal can correspond to the same time period. Then, after the target far-end signal and the designated near-end signal are aligned, the signal corresponding to the same time period as the target far-end signal can be determined in the designated near-end signal and determined as the target near-end signal. The target near-end signal at least includes the echo part aligned with the target far-end signal.

[0057] Step S308: taking a signal segment in the target near-end signal that meets a preset time domain length and has not been subjected to echo cancellation as a target near-end signal segment, wherein the preset time domain length is consistent with the time domain length of the target far-end signal segment.

[0058] After determining the target near-end signal, the original signal segment that has not been subjected to echo cancellation processing can be determined in the target near-end signal, and then the signal segment with the earliest timing and a duration of a preset time domain length can be determined as the target near-end signal segment in the original signal segment. Through the above steps, it can be ensured that the target far-end signal and its corresponding echo signal are aligned in time, thereby improving the accuracy and effectiveness of echo cancellation.

[0059] Step S204: Filter the echo estimation signal in the target near-end signal segment to obtain an error signal, wherein the target near-end signal segment is a sound signal collected by an audio collection device in the target environment and aligned with the target far-end signal segment.

[0060] As is known from the foregoing, when the target far-end signal is played in the target environment, an echo will be generated. Therefore, when the audio collection device (e.g., a microphone) in the target environment performs audio collection, the target near-end signal obtained may contain the voice of the user speaking in the target environment and the reverberation sound generated by the target far-end signal in the target environment. Based on this, in the target near-end signal segment aligned with the target far-end signal segment, the echo estimation signal corresponding to the target near-end signal segment can be filtered out to obtain an error signal. Here, the alignment of the target near-end signal segment with the target far-end signal segment refers to aligning the target near-end signal segment with the target far-end signal segment in time, so as to ensure that the target far-end signal segment and its corresponding target near-end signal segment correspond to the same time period in time.

[0061] Step S206: determining a target convergence step size based on the echo estimation signal and the error signal.

[0062] In this embodiment, the target convergence step size is determined based on the echo estimation signal and the error signal. That is, the target convergence step size is determined based on the echo estimation signal and the error signal. The target convergence step size is a step size factor for updating the coefficients of the target filter, which determines the amplitude of each update of the filter coefficients.

[0063] As an optional implementation, step S206 determines the target convergence step based on the echo estimation signal and the error signal, including the following steps: determining the echo energy of the echo estimation signal and the error energy of the error signal; and determining the energy ratio between the echo energy and the error energy as the target convergence step.

[0064] In this embodiment, the echo energy E echo It can be calculated as follows: first calculate the sum of squares of the echo estimation signal, then take the average of the sum of squares, and record the average as the echo energy. The echo energy can be calculated using the following formula:

[0065]

[0066] Wherein, e(n) is the value of the echo estimation signal at time n, and N is the length of the echo estimation signal.

[0067] Error energy E error It can be calculated as follows: first calculate the square sum of the error energy, then take the average of the square sum, and record the average as the echo energy. The error energy can be calculated using the following formula:

[0068]

[0069] Wherein, e'(n) is the value of the error signal at time n, and N is the length of the error signal.

[0070] After obtaining the echo energy and the error energy, the target convergence step corresponding to the target filter can be determined by calculating the energy ratio between the echo energy and the error energy, that is, the convergence step of the target filter is adjusted to: echo energy / error energy (that is, E echo / E error ). If the echo energy decreases and the error energy increases, that is, the energy ratio becomes smaller, it means that double talk or divergence occurs. At this time, the step size is reduced. Otherwise, the step size is increased. Double talk refers to the phenomenon that both the near-end and far-end voice signals are transmitted in the same time period in the voice communication system. In this case, the echo canceller needs to distinguish and process the near-end voice and far-end echo signals separately, which is quite difficult. If not handled properly, problems such as residual echo, near-end voice loss or distortion may occur. An excellent echo cancellation algorithm needs to maintain high echo cancellation performance in the double talk state without affecting the normal transmission of the near-end voice. Divergence means that the echo cancellation algorithm loses stability and accuracy during double talk and begins to over-adapt or incorrectly cancel the voice signal of the near-end speaker. This means that the algorithm not only tries to eliminate the far-end echo signal, but also incorrectly weakens or eliminates some near-end voice signals, resulting in a significant decrease in call quality, or even the near-end speaker cannot be heard at all. Therefore, through the method of this embodiment, the target convergence step corresponding to the target filter is determined according to the energy ratio between the echo energy and the error energy, which can effectively avoid the impact of double talk or divergence on the filtering of the filter, thereby improving the effectiveness of the filter.

[0071] Step S208: adjusting the convergence step size of the target filter to the target convergence step size to obtain a target step size filter, and performing echo cancellation on the target near-end signal segment through the target step size filter to obtain a pure near-end signal segment.

[0072] After determining the target convergence step size, the step size factor of the coefficients of the target filter can be updated according to the target convergence step size, so as to obtain an updated target step size filter, and the target step size filter can be applied again to the target near-end signal segment to eliminate the remaining echo components, so as to obtain a pure near-end signal segment, that is, a local signal without echo.

[0073] As an optional implementation, when the target filter is a linear filter, the target near-end signal echo cancellation by the target step filter in the aforementioned step S208 can be implemented by the following steps: the target near-end signal echo cancellation is performed by the first linear filter to obtain a first candidate near-end signal segment, and the target near-end signal echo cancellation is performed by the second linear filter to obtain a second candidate near-end signal segment; the first candidate near-end signal segment and the second candidate near-end signal segment with smaller energy are determined as the pure near-end signal segment. That is, when the target filter is a linear filter, the target step filter is a new linear filter after the step factor of the linear filter coefficient is updated according to the target convergence step. Optionally, the linear filter can adopt a Kalman filter and an adaptive minimum mean square filter, and then, the first linear filter can be a Kalman filter applied to linear filtering, and the second linear filter can be an adaptive minimum mean square filter applied to linear filtering. In addition, when the first linear filter can be an adaptive minimum mean square filter applied to linear filtering, the second linear filter is a Kalman filter applied to linear filtering, that is, the first linear filter and the second linear filter are two different linear filters in the Kalman filter and the adaptive minimum mean square filter applied to linear filtering. Moreover, in this embodiment, the target near-end signal is echo-eliminated by the first linear filter to obtain a first candidate near-end signal segment, and the target near-end signal is echo-eliminated by the second linear filter to obtain a second candidate near-end signal segment, so that different results can be obtained after the two linear filters perform echo elimination on the target near-end signal. After obtaining the first candidate near-end signal segment and the second candidate near-end signal segment, in order to obtain a better suppression effect on the echo signal, the one with smaller energy among the first candidate near-end signal segment and the second candidate near-end signal segment is determined as the pure near-end signal segment, that is, the group with the smaller signal after the two linear filtering is taken as the final actual output.

[0074] As an optional implementation, the aforementioned echo cancellation of the target near-end signal by the target step filter can be achieved by the following steps when the target filter is a nonlinear filter to obtain a pure near-end signal segment: convert the target near-end signal segment from the time domain to the frequency domain to obtain a frequency domain signal; determine the proportion of the target far-end signal segment in the error signal after propagation by the target convergence step; suppress the echo component in the partial frequency domain signal corresponding to each frequency band by the target step filter corresponding to each frequency band in the frequency domain signal to obtain a suppressed signal, wherein the coefficient of the target step filter corresponding to each frequency band is obtained by adjusting the proportion; convert the suppressed signal from the frequency domain to the time domain to obtain a pure near-end signal segment. In this embodiment, the target near-end signal segment can be converted from the time domain to the frequency domain by using a fast Fourier transform (FFT) to obtain a corresponding frequency domain signal. By converting the target near-end signal segment from the time domain to the frequency domain, the signal can be decomposed into a combination of different frequency components for subsequent processing. The target convergence step has been determined by the above embodiment, and then the proportion of the target far-end signal segment in the error signal after propagation can be determined according to the target convergence step, and the proportion can be used to reflect the contribution of the target far-end signal segment to the error signal, that is, the proportion of the residual echo. That is to say, the target convergence step obtained in the above embodiment estimates the proportion of the target far-end signal segment in the error signal after propagation, that is, the power spectral density of the residual echo, and in this embodiment, the target convergence step is directly used as the proportion. After obtaining the frequency domain signal, the frequency domain signal can be divided into multiple frequency bands, each frequency band contains frequency components within a certain range, and the frequency components contained in each frequency band can be realized by selecting a suitable bandwidth and center frequency, and the signal in each frequency band is recorded as a partial frequency domain signal. Furthermore, when the nonlinear filter is a nonlinear exponential function, in each frequency band, the coefficient of a nonlinear exponential function is adjusted by the proportion of the residual echo, and a target step size filter corresponding to each frequency band is obtained, so that the echo component in the partial frequency domain signal of each frequency band is suppressed by the target step size filter corresponding to each frequency band, and a partial suppressed signal corresponding to each frequency band is obtained, and then the partial suppressed signal of each frequency band can be recombined into a complete suppressed signal. After obtaining the suppressed signal, the suppressed signal can be converted from the frequency domain back to the time domain using the Inverse Fast Fourier Transform (IFFT), so that the obtained time domain signal can be a pure near-end signal segment processed by echo suppression.

[0075] This specification determines the target convergence step length corresponding to the target far-end signal segment based on the echo estimation signal and the error signal, and can realize dynamic adjustment of the convergence step length, so as to improve the convergence speed and stability of the filter. In addition, although the error signal is still used as one of the considerations in the process of determining the target convergence step length, the echo estimation signal is added as one of the considerations while retaining the error signal. Therefore, even if the noise components that may be contained in the error signal still cause noise to the target filter, the weight of the influence of the noise components on the step length can be reduced through the newly added echo estimation signal, thereby achieving the purpose of reducing the influence of the noise components that may be contained in the error signal on the target filter and the sensitivity of the target filter to noise. At the same time, by determining the target convergence step length and adjusting the target filter in real time. The convergence step size of is the target convergence step size, so that the target step size filter determined in real time can better adapt to the echo characteristics of the echo corresponding to the target far-end signal segment, especially when the signal characteristics change, it can quickly adjust to adapt to the new echo path, thereby improving the overall performance of echo elimination for the target near-end signal segment (that is, flexibility and adaptability to signal changes), and thus can effectively overcome the problems of poor stability, slow convergence speed, excessive sensitivity to noise, lack of flexibility, and poor adaptability to signal changes in the adaptive filter for far-end signal filtering that only uses the error signal to dynamically adjust the step size in the related art.

[0076] As an optional implementation, the method further includes the following steps: determining an actual error signal corresponding to the error signal; adjusting the coefficients of a linear filter according to the error signal and the actual error signal to obtain an optimized linear filter, wherein the linear filter is used to filter the target near-end signal segment according to the echo estimation signal to obtain the error signal. In this embodiment, the actual error signal corresponding to the error signal can be determined by subtracting the pure near-end signal segment from the near-end signal segment. Then, the coefficients of the linear filter can be adjusted according to the error signal and the actual error signal until the linear filter can estimate an improved error signal that is closer to the actual error signal relative to the error signal according to the target near-end signal segment, until the improved error signal becomes stable. Through the method of this embodiment, the performance of the linear filter can be continuously optimized based on the error signal and the actual error signal, thereby achieving a better echo cancellation effect.

[0077] As an optional implementation, for example, in the double recording scenario during local offline sales, there is often the problem of sales staff illegally directing the speech to consumers. All the clean near-end signal fragments in the offline sales scenario can be obtained by the method of any of the aforementioned embodiments, and the clean near-end signal can be combined. After obtaining the clean near-end signal, ASR (Automatic Speech Recognition) speech transcription can also be performed on it to automatically convert the above-mentioned clean near-end signal from a speech signal to text data. The text data is detected for prohibited speech, and it is judged whether the salesperson illegally directs the speech to the consumer based on the detected prohibited speech.

[0078] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0079] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0080] Figure 5 is a schematic structural diagram of a device provided by an exemplary embodiment. Figure 5 At the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, a memory 508, and a non-volatile memory 510, and may also include hardware required for other functions. One or more embodiments of this specification may be implemented based on software, such as the processor 502 reading the corresponding computer program from the non-volatile memory 510 into the memory 508 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0081] Please refer to Figure 6 , the echo cancellation device can be applied to Figure 5The device shown in the figure is used to implement the technical solution of this specification. The echo cancellation device may include:

[0082] The echo signal simulation module 61 is used to perform echo signal simulation on a target remote signal segment in a target remote signal through a target filter to obtain an echo estimation signal, wherein the target remote signal is an audio signal played through an audio playback device in a target environment;

[0083] A filtering module 62, configured to filter out the echo estimation signal in the target near-end signal segment to obtain an error signal, wherein the target near-end signal segment is a sound signal collected by an audio collection device in a target environment and aligned with the target far-end signal segment;

[0084] A step length determination module 63, configured to determine a target convergence step length based on the echo estimation signal and the error signal;

[0085] The echo cancellation module 64 is used to adjust the convergence step size of the target filter to the target convergence step size to obtain a target step size filter, and perform echo cancellation on the target near-end signal segment through the target step size filter to obtain a pure near-end signal segment.

[0086] As an optional implementation manner, the echo cancellation device further includes:

[0087] An acquisition module, used for acquiring near-end signals to be processed in a target environment;

[0088] A first determination module, configured to determine a designated near-end signal corresponding to a target far-end signal from among the near-end signals to be processed;

[0089] an alignment module, configured to align the target far-end signal and the designated near-end signal by determining a mutual correlation coefficient between the target far-end signal and the designated near-end signal, and determine a target near-end signal corresponding to the target far-end signal in the designated near-end signal;

[0090] The second determination module is used to use a signal segment in the target near-end signal that meets a preset time domain length and has not undergone echo cancellation as a target near-end signal segment, wherein the preset time domain length is consistent with the time domain length of the target far-end signal segment.

[0091] As an optional implementation manner, the first determining module is configured to:

[0092] Determine the time interval between the start timestamp and the end timestamp of the target remote signal as the original time interval of the target remote signal;

[0093] The starting timestamp of the original time interval is advanced according to the preset duration, and the ending timestamp of the original time interval is delayed to obtain an extended time interval corresponding to the original time interval;

[0094] An audio segment corresponding to the extended time interval in the near-end signal to be processed is determined as the designated near-end signal.

[0095] As an optional implementation, the step length determination module 63 is configured to:

[0096] determining an echo energy of the echo estimation signal and an error energy of the error signal;

[0097] The energy ratio between the echo energy and the error energy is determined as the target convergence step size.

[0098] As an optional implementation, when the target filter is a linear filter, the echo cancellation module 64 is configured as follows:

[0099] Performing echo cancellation on the target near-end signal through a first linear filter to obtain a first candidate near-end signal segment, and performing echo cancellation on the target near-end signal through a second linear filter to obtain a second candidate near-end signal segment;

[0100] The one with smaller energy among the first candidate near-end signal segment and the second candidate near-end signal segment is determined as a pure near-end signal segment.

[0101] As an optional implementation, when the target filter is a nonlinear filter, the echo cancellation module 64 is configured as follows:

[0102] Convert the target near-end signal segment from the time domain to the frequency domain to obtain a frequency domain signal;

[0103] The proportion of the target far-end signal segment in the error signal after propagation is determined by the target convergence step size;

[0104] For a portion of the frequency domain signal corresponding to each frequency band of the frequency domain signal, the coefficient of the nonlinear filter is adjusted according to the proportion to obtain an adjusted nonlinear filter corresponding to each frequency band;

[0105] According to a one-to-one correspondence, the echo component in each frequency band is suppressed by each adjusted nonlinear filter to obtain a suppressed signal;

[0106] The suppressed signal is converted from the frequency domain to the time domain to obtain a pure near-end signal segment.

[0107] As an optional implementation, the echo signal simulation module 61 is configured as follows:

[0108] Determine a target filtering mode corresponding to the target remote signal according to energy change information corresponding to the target remote signal segment, wherein the filtering mode includes: linear filtering and nonlinear filtering;

[0109] Simulating a target far-end signal segment in the target far-end signal through a plurality of candidate filters corresponding to the target filtering mode, and obtaining a candidate echo estimation signal output by each candidate filter;

[0110] A candidate echo estimation signal with the largest signal energy among all candidate echo estimation signals is determined as the echo estimation signal.

[0111] As an optional implementation, the echo cancellation device further includes an adjustment module, and the adjustment module is configured to:

[0112] determining an actual error signal corresponding to the error signal;

[0113] According to the error signal and the actual error signal, the coefficient of the linear filter is adjusted to obtain an optimized linear filter, wherein the linear filter is used to filter the target near-end signal segment according to the echo estimation signal to obtain the error signal.

[0114] Based on the same concept as the above method, this specification also provides an electronic device, including: a processor; a memory for storing processor executable instructions; wherein the processor implements the steps of the method described in any of the above embodiments by running the executable instructions.

[0115] Based on the same concept as the above method, this specification also provides a computer-readable storage medium on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0116] Based on the same concept as the above method, this specification also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.

Claims

1. An echo cancellation method, comprising: Performing echo signal simulation on a target far-end signal segment in a target far-end signal through a target filter to obtain an echo estimation signal, wherein the target far-end signal is an audio signal played through an audio playback device in a target environment; filtering the echo estimation signal in a target near-end signal segment to obtain an error signal, wherein the target near-end signal segment is a sound signal collected by an audio collection device in the target environment and aligned with the target far-end signal segment; Determining a target convergence step size based on the echo estimation signal and the error signal; The convergence step size of the target filter is adjusted to the target convergence step size to obtain a target step size filter, and the target near-end signal segment is echo-cancelled by the target step size filter to obtain a pure near-end signal segment.

2. The method according to claim 1, before simulating the target far-end signal segment by the target filter to obtain the echo estimation signal, the method further comprises: Acquire a near-end signal to be processed in the target environment; Determining a designated near-end signal corresponding to the target far-end signal from the near-end signals to be processed; By determining a mutual correlation coefficient between the target far-end signal and the designated near-end signal, the target far-end signal and the designated near-end signal are aligned, and the target near-end signal corresponding to the target far-end signal is determined in the designated near-end signal; A signal segment in the target near-end signal that meets a preset time domain length and is not subjected to echo cancellation is used as the target near-end signal segment, wherein the preset time domain length is consistent with the time domain length of the target far-end signal segment.

3. The method according to claim 2, wherein determining the designated near-end signal corresponding to the target far-end signal from the near-end signal to be processed comprises: Determine the time interval between the start timestamp and the end timestamp of the target remote-end signal as the original time interval of the target remote-end signal; Advancing the start timestamp of the original time interval according to a preset duration, and delaying the end timestamp of the original time interval, to obtain an extended time interval corresponding to the original time interval; An audio segment corresponding to the extended time interval in the near-end signal to be processed is determined as the designated near-end signal.

4. The method according to claim 1, wherein determining a target convergence step size based on the echo estimation signal and the error signal comprises: determining an echo energy of the echo estimation signal and an error energy of the error signal; An energy ratio between the echo energy and the error energy is determined as the target convergence step size.

5. The method according to claim 1, wherein when the target filter is a linear filter, performing echo cancellation on the target near-end signal through the target step-size filter to obtain a clean near-end signal segment comprises: Performing echo cancellation on the target near-end signal through a first linear filter to obtain a first candidate near-end signal segment, and performing echo cancellation on the target near-end signal through a second linear filter to obtain a second candidate near-end signal segment; The one with smaller energy among the first candidate near-end signal segment and the second candidate near-end signal segment is determined as the pure near-end signal segment.

6. The method according to claim 1, wherein when the target filter is a nonlinear filter, performing echo cancellation on the target near-end signal through the target step-size filter to obtain a pure near-end signal segment comprises: Convert the target near-end signal segment from the time domain to the frequency domain to obtain a frequency domain signal; Determine the proportion of the target far-end signal segment in the error signal after propagation through the target convergence step size; Suppressing the echo component in the partial frequency domain signal corresponding to each frequency band through a target step size filter corresponding to each frequency band in the frequency domain signal, to obtain a suppressed signal, wherein the coefficient of the target step size filter corresponding to each frequency band is obtained by adjusting according to the proportion; The suppressed signal is converted from the frequency domain to the time domain to obtain the pure near-end signal segment.

7. The method according to claim 1, wherein simulating the target far-end signal segment in the target far-end signal by using the target filter to obtain the echo estimation signal comprises: Determining a target filtering mode corresponding to the target remote-end signal segment according to energy change information corresponding to the target remote-end signal segment, wherein the filtering mode includes: linear filtering and nonlinear filtering; Simulating a target far-end signal segment through a plurality of candidate filters corresponding to the target filtering mode to obtain a candidate echo estimation signal output by each candidate filter; A candidate echo estimation signal with the largest signal energy among all candidate echo estimation signals is determined as the echo estimation signal.

8. The method according to claim 1, further comprising: determining an actual error signal corresponding to the error signal; According to the error signal and the actual error signal, coefficients of a linear filter are adjusted to obtain an optimized linear filter, wherein the linear filter is used to perform filtering of a target near-end signal segment according to the echo estimation signal to obtain an error signal.

9. An electronic device, comprising: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

10. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.