An adaptive echo path modeling method and system for echo suppression

Abstract

The application provides a self-adaptive echo path modeling method and system for echo suppression, which is applied to the field of audio signal processing in network communication; the method comprises the following steps: receiving a far-end audio signal and collecting a near-end microphone signal containing echo; a first adaptive filter with a large step size and a short filter and a second adaptive filter with a small step size and a long filter are used in parallel to process the far-end signal; a network state parameter representing network delay jitter is obtained; the fusion weight of the outputs of the two filters is dynamically adjusted according to the parameter; a predicted echo signal is generated by weighted fusion; and the predicted echo signal is subtracted from the near-end signal to obtain an echo-free output; through the above scheme, the stability and adaptability of echo suppression can be effectively improved while the fast tracking ability and steady-state accuracy of echo path modeling are taken into account in the case of severe network delay jitter.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing technology, and in particular to an adaptive echo path modeling method and system for echo suppression. Background Technology

[0002] In the field of audio signal processing, echo cancellation technology is widely used in VoIP, video conferencing systems and smart voice devices to eliminate acoustic echoes formed when distant voice is played through a speaker and picked up by a nearby microphone.

[0003] Traditional echo suppression methods primarily rely on a single adaptive filter to model the echo path, adjusting the filter coefficients using the least mean square (LMS) or normalized least mean square (NLMS) algorithm to predict and cancel the echo signal. However, in real-world network communication environments, packet loss and time-of-arrival jitter cause the echo path's delay characteristics to change rapidly over time. Traditional single models struggle to balance convergence speed and steady-state accuracy, easily leading to filter divergence or tracking lag, resulting in a significant increase in residual echo. Furthermore, when loudspeakers operate at high volumes, their physical nonlinear distortion and the complex reflections of the room's acoustic environment introduce nonlinear echo components. Conventional linear filtering models cannot effectively model these nonlinear characteristics, further limiting echo suppression performance. Simultaneously, highly correlated interference signals such as music in the far-end signal and asynchronous sampling clocks at the transceiver end also severely impact the stability and convergence of adaptive algorithms.

[0004] Therefore, existing technologies still suffer from technical defects such as insufficient echo path modeling accuracy and weak adaptability when dealing with multiple challenges such as dynamic network conditions, nonlinear distortion and complex signal interference, making it difficult to meet the needs of high-quality real-time communication.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides an adaptive echo path modeling method and system for echo suppression, aiming to solve the problems of poor adaptability of echo path modeling in the face of severe network communication latency jitter in the prior art, and the difficulty of traditional single adaptive filters in balancing fast tracking capability and steady-state modeling accuracy. By adopting a fast and slow dual-model parallel processing architecture combined with a dynamic weighted fusion mechanism based on network state parameters, the present invention achieves efficient tracking and accurate modeling of time-varying echo paths, thereby improving the stability and robustness of the echo suppression system.

[0007] This invention provides an adaptive echo path modeling method for echo suppression, applied to sound analysis and processing of network communication devices, including: Receives far-end audio signals and acquires near-end microphone signals including echoes; The far-end audio signal is processed in parallel using a first adaptive filter and a second adaptive filter, wherein the step size parameter of the first adaptive filter is greater than that of the second adaptive filter, and the filter length of the first adaptive filter is less than that of the second adaptive filter. Obtain network state parameters that characterize network communication latency jitter. Based on the network state parameters, determine the fusion weights assigned to the output signals of the first adaptive filter and the second adaptive filter; The output signals of the first adaptive filter and the second adaptive filter are weighted and fused to generate a predicted echo signal. The predicted echo signal is subtracted from the near-end microphone signal to obtain the echo-canceled output audio signal.

[0008] In some optional embodiments, network state parameters include network latency jitter values.

[0009] In some optional embodiments, the step of determining the fusion weights assigned to the output signals of the first adaptive filter and the second adaptive filter based on network state parameters includes: Based on the network state parameters, the fusion weights are initially determined; The first convergence error of the first adaptive filter and the second convergence error of the second adaptive filter are obtained in parallel. Based on the comparison between the first convergence error and the second convergence error, the initially determined fusion weights are adjusted to obtain the final fusion weights.

[0010] In some optional embodiments, the method further includes: Calculate the stability index of the coefficient vector of the second adaptive filter within a preset time window; When the stability index exceeds the preset threshold, the near-end microphone signal is converted from the time domain to the frequency domain, and the converted frequency domain signal is compensated.

[0011] In some optional embodiments, the method further includes: The preset threshold is dynamically adjusted based on the energy of the residual error signal generated by the second adaptive filter.

[0012] In some optional embodiments, the method further includes: The nonlinear distortion residual components of the output audio signal after echo cancellation are separated and suppressed.

[0013] In some optional embodiments, the step of separating and suppressing the residual components of nonlinear distortion includes: The far-end audio signal is input into the second adaptive filter to generate the second predicted linear echo signal; The second predicted linear echo signal is subtracted from the echo-eliminated output audio signal to separate the nonlinear distortion residual component.

[0014] In some optional embodiments, the step of separating and suppressing the residual components of nonlinear distortion further includes: The nonlinear distortion residual components of the output audio signal after echo elimination are all converted to the time-frequency domain. The gain adjustment coefficient is determined based on the energy ratio of the residual nonlinear distortion component to the energy of the echo-eliminating output audio signal in the time-frequency unit. The time-frequency domain representation of the echo-canceled output audio signal is processed using a gain adjustment coefficient.

[0015] In some optional embodiments, the method further includes: The echo-eliminated output audio signal is used as the input signal to update the coefficient vectors of the first adaptive filter and the second adaptive filter.

[0016] In some alternative embodiments, the method further includes: using a speech separation model to separate the human voice stream signal from the far-end audio signal before processing the far-end audio signal in parallel using a first adaptive filter and a second adaptive filter; The step of processing the far-end audio signal in parallel using the first adaptive filter and the second adaptive filter includes: processing the human voice stream signal in parallel using the first adaptive filter and the second adaptive filter.

[0017] In some optional embodiments, the speech separation model is a deep learning-based neural network model.

[0018] In some optional embodiments, the method further includes: estimating a clock drift factor by calculating the rate of change of the phase difference between the far-end audio signal and the near-end microphone signal at a specific frequency point over time before processing the far-end audio signal in parallel using a first adaptive filter and a second adaptive filter; and resampling the far-end audio signal using a fractional delay filter based on the clock drift factor to generate a resampled far-end audio signal. The step of processing the far-end audio signal in parallel using a first adaptive filter and a second adaptive filter includes: processing the resampled far-end audio signal in parallel using a first adaptive filter and a second adaptive filter.

[0019] In some alternative embodiments, the method is applied to an audio device including a speaker, and the method further includes: processing the far-end audio signal by a speaker nonlinear predistortion module to generate a pre-compensated signal before processing the far-end audio signal in parallel using a first adaptive filter and a second adaptive filter; The steps of processing the far-end audio signal in parallel using the first adaptive filter and the second adaptive filter are as follows: processing the pre-compensation signal in parallel using the first adaptive filter and the second adaptive filter, and sending the pre-compensation signal to the speaker for playback.

[0020] In some alternative embodiments, the step size parameter of the first adaptive filter is dynamically adjusted in a positive correlation with its respective network state parameter.

[0021] In some alternative embodiments, the step of determining the fusion weights includes: When the network state parameters indicate that the network communication delay jitter is increasing, the fusion weight of the output signal allocated to the first adaptive filter is increased, and the fusion weight of the output signal allocated to the second adaptive filter is decreased.

[0022] This invention provides an adaptive echo path modeling system for echo suppression, which is applied to the sound analysis and processing of network communication devices, including: The signal receiving module is configured to receive far-end audio signal acquisition from a near-end microphone signal that includes echo; The parallel filtering module includes a first adaptive filter and a second adaptive filter, and is configured to process the far-end audio signal in parallel using the first adaptive filter and the second adaptive filter. The step size parameter of the first adaptive filter is greater than the step size parameter of the second adaptive filter, and the filter length of the first adaptive filter is less than the filter length of the second adaptive filter. The network status acquisition module is configured to acquire network status parameters that characterize the network communication delay jitter status. The weighted fusion module is connected to the parallel filtering module and the network state acquisition module, and is configured to determine the fusion weights of the output signals of the first adaptive filter and the second adaptive filter based on the network state parameters obtained by the network state acquisition module, and perform weighted fusion on the output signals of the first adaptive filter and the second adaptive filter to generate a predicted echo signal. The echo cancellation module is configured to subtract the predicted echo signal from the near-end microphone signal to obtain the echo-cancelled output audio signal.

[0023] In some optional embodiments, the weighted fusion module is further configured to adjust the fusion weights based on the convergence error of the first adaptive filter when determining the fusion weights according to the network state parameters.

[0024] In some optional embodiments, the system further includes a nonlinear processing module configured to: generate a second predicted linear echo signal using a second adaptive filter, separate nonlinear distortion residual components from the echo-cancelled output audio signal, and suppress the echo-cancelled output audio signal based on the energy characteristics of the nonlinear distortion residual components in the time-frequency domain.

[0025] In some optional embodiments, the system further includes a speech separation module located between the signal receiving module and the parallel filtering module, and configured to: separate the human voice stream signal from the far-end audio signal received by the signal receiving module, and provide the human voice stream signal to the parallel filtering module as a reference signal for processing by the parallel filtering module.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention.

[0027] The adaptive echo path modeling method and system for echo suppression of the present invention have the following beneficial effects: This invention effectively addresses the time-varying echo path problem caused by network latency jitter by configuring adaptive filters with different step sizes and filter lengths in parallel and combining them with a mechanism that dynamically adjusts the fusion weights based on network state parameters. While ensuring steady-state suppression accuracy, it significantly improves the model's ability to track rapidly changing environments, thereby enhancing the stability and robustness of echo cancellation and improving the quality of voice communication under complex network conditions. Attached Figure Description

[0028] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0029] Figure 1 This is a flowchart of an adaptive echo path modeling method for echo suppression according to an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram of an adaptive echo path modeling system for echo suppression according to an embodiment of the present invention. Detailed Implementation

[0031] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0032] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0033] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined. Therefore, the actual execution order may change depending on the specific circumstances.

[0034] In audio signal processing, echoes originate from distant speech signals being played through speakers, propagating in physical space, and being picked up by near-end microphones, forming a copy with time delay and amplitude attenuation compared to the original signal. To eliminate this echo, a mathematical model capable of simulating the actual acoustic path in real time needs to be constructed. Typically, an adaptive filter is used to convolve the distant signal to generate a predicted echo signal, which is then subtracted from the microphone's received signal. Traditional methods rely on a single filter structure, which presents an inherent contradiction between convergence speed and steady-state accuracy when dealing with time-varying delay jitter in network transmission: a larger step size can accelerate the tracking of dynamic paths but leads to increased steady-state errors; a smaller step size, while improving modeling accuracy, struggles to respond to rapidly changing channel characteristics. To address this contradiction, two filters with different dynamic characteristics are introduced and run in parallel, respectively handling fast response and high-precision modeling. The outputs of these two filters are then weighted and fused based on externally observable network state parameters, improving the system's adaptability to instantaneous path changes without sacrificing steady-state performance. The weight allocation mechanism dynamically adjusts based on network latency jitter, prioritizing the use of short, fast-responding filter outputs in high-jitter environments and favoring long, finely modeled filter outputs in low-jitter environments. This achieves stable and accurate estimation of time-varying echo paths. This approach effectively mitigates the model mismatch problem caused by the limited update rate in traditional single-model architectures in dynamic environments, improving the overall robustness and stability of the echo suppression system.

[0035] It should be understood that, attached Figure 1 Each step in the flowchart shown can be explained by the appendix. Figure 2 This is achieved through one or more functional modules in the system shown.

[0036] like Figure 1 As shown, this embodiment of the invention provides an adaptive echo path modeling method for echo suppression, applied to the sound analysis and processing of network communication devices. The method includes the following steps: Step S100: Receive the far-end audio signal and acquire the near-end microphone signal containing the echo.

[0037] In one embodiment, the remote audio signal is received in the form of data packets through a network interface and decoded to restore it into a time-domain digital audio stream; at the same time, the local audio hardware acquires a mixed signal that is played by the speaker, propagates through physical space, and is picked up by the microphone. This signal contains linear and nonlinear echo components as well as possible near-end speech and environmental noise.

[0038] Step S200: Process the far-end audio signal in parallel using the first adaptive filter and the second adaptive filter.

[0039] In this implementation, the first adaptive filter is configured with a large step size and a short filter length to achieve fast response; the second adaptive filter is configured with a small step size and a long filter length to ensure high-precision modeling. The two filters operate independently based on the same reference signal, each outputting a corresponding echo prediction signal. In one implementation, both filters employ a Normalized Least Mean Square (NLMS) algorithm, with their tap coefficients updated online over time to approximate the impulse response of the actual echo path. In other alternative implementations, affine projection (APA), recursive least squares (RLS), or other adaptive algorithms suitable for real-time signal processing can be used as alternatives.

[0040] Step S300: Obtain network state parameters that characterize the network communication delay jitter state.

[0041] This parameter reflects the degree of instability in the arrival time of data packets in the current communication link. In one embodiment, the system extracts an indicator that can quantify the dynamic characteristics of the network by parsing control information in the transport layer protocol or directly analyzing the timestamp sequence of received data packets. In other optional implementations, this network state parameter can be indirectly obtained by monitoring the rate of change of time intervals between consecutive data packets, packet loss rate trends, or round-trip delay fluctuations.

[0042] Step S400: Determine the fusion weights assigned to the output signals of the first adaptive filter and the second adaptive filter based on the network state parameters.

[0043] The fusion weights are set based on the changing trends of the acquired network state parameters. When the network state exhibits significant instability, the first adaptive filter is given higher output weights; when the network state tends to be stable, the dominant role of the second adaptive filter is enhanced. This decision-making process aims to balance the trade-off between the model's tracking speed and steady-state accuracy. In some other alternative implementations, the fusion weights can be generated based on a pre-set lookup table, a piecewise linear function, or a nonlinear mapping relationship obtained through machine learning training.

[0044] Step S500: Weighted fusion of the output signals of the first adaptive filter and the second adaptive filter to generate a predicted echo signal.

[0045] The time-domain echo estimates output by the two filters are linearly combined according to the weights determined in step S400 to form the final comprehensive prediction result. This predicted signal reproduces the actual acoustic path response from the loudspeaker to the microphone as accurately as possible in the time domain, providing a basis for subsequent echo cancellation.

[0046] Step S600: Subtract the predicted echo signal from the near-end microphone signal to obtain the echo-cancelled output audio signal.

[0047] The original microphone signal is subtracted point by point from the predicted echo signal generated in step S500 to obtain the initially purified voice output. This signal serves as the basis for further processing and can be used for call uploading or local playback.

[0048] There is a close collaborative working mechanism among the aforementioned technical features. Specifically, the first adaptive filter, with its fast convergence characteristics, addresses abrupt changes in path delay caused by the network, while the second adaptive filter focuses on accurately characterizing the complex reverberation characteristics under steady-state conditions. The fusion of the two outputs is not a fixed-ratio superposition, but rather dynamically adjusted based on the externally observable network state, thus achieving robust modeling of time-varying echo paths. This hierarchical, division-of-labor, and on-demand scheduling architecture allows the system to effectively adapt to drastically changing communication conditions while maintaining high modeling accuracy, solving the technical bottleneck of traditional single models that struggle to balance dynamic response and steady-state performance. By introducing a weighting mechanism based on network state feedback, the entire modeling process possesses adaptive decision-making capabilities for real-world network environments, significantly improving echo suppression stability in high-jitter scenarios.

[0049] Through the above scheme, this embodiment can effectively track and accurately model fast time-varying echo paths in complex network environments, and significantly improve the dynamic adaptability of the system without sacrificing steady-state suppression performance, thereby providing clearer and more stable voice communication quality.

[0050] In one specific implementation, the network state parameter includes a network latency jitter value. Specifically, this network latency jitter value is obtained by real-time analysis of changes in the arrival time intervals of network data packets at the receiving end. First, the system extracts timestamp information from the receiver's report of the Real-time Transmission Control Protocol (RTCP) or directly monitors the arrival time sequence of remote audio data packets. Then, the standard deviation or mean absolute deviation of the arrival time differences between adjacent data packets is calculated as an indicator to quantify the degree of network latency jitter. This jitter value is periodically updated in milliseconds and input to the weighted fusion module for dynamically adjusting the filter output weights.

[0051] In other alternative implementations, network latency jitter can also be achieved through range statistics within a sliding window, latency change trends processed by exponentially weighted moving average (EWMA), or a composite jitter scoring mechanism combining packet loss rate and out-of-order rate. The above different calculation methods can be flexibly selected according to the actual configuration of the communication protocol stack and the performance of the hardware platform, and all of them can effectively reflect the time-varying characteristics of the network transmission process.

[0052] Through the above scheme, this embodiment can provide an objective and quantifiable network state representation method, enabling the echo path modeling process to have a stronger ability to perceive network dynamic changes, thereby improving the accuracy and response speed of adaptive filter fusion decision-making and enhancing the robustness of the system in complex network environments.

[0053] In one specific implementation, based on the above embodiments, the process of determining the fusion weights according to network state parameters further includes a technique for fine-tuning the initially determined weights. Specifically, firstly, the fusion weights are initially set based on the acquired network state parameters. For example, when the network latency jitter value is high, the first adaptive filter is given a higher initial weight to enhance the system's response to path abrupt changes. Then, the first convergence error of the first adaptive filter and the second convergence error of the second adaptive filter are monitored in parallel. The convergence error is obtained by calculating the residual between the predicted echo signal output by the corresponding filter and the near-end microphone signal. Next, the energy levels of the two convergence errors are compared. If the energy of the first convergence error is significantly lower than that of the second convergence error, it indicates that the current fast model fits the actual echo path more accurately. In this case, the fusion weights allocated to the output signal of the first adaptive filter are further increased. Conversely, if the second convergence error is smaller, the weight of the fast model is appropriately reduced so that the slow model plays a dominant role in a stable state. This adjustment process can be implemented using a scaling factor or lookup table, or the error energy ratio can be converted into a weight correction amount by designing a nonlinear mapping function.

[0054] In some other alternative implementations, the comparison of convergence errors can be statistically processed within a sliding time window to improve the stability of the decision; or, the error comparison results can be used to generate discrete adjustment instructions through a logic judgment module to trigger a step-by-step adjustment of the weights; in addition, the final determination of the weights can also introduce a hysteresis mechanism to prevent frequent switching near the critical point of the network state.

[0055] In a preferred embodiment, the weighted fusion module employs a two-stage, two-level decision-making mechanism when determining the fusion weights. This mechanism not only relies on external network indicators but also incorporates feedback from the internal modeling results, thereby achieving more precise dynamic adjustments.

[0056] Phase 1: Preliminary Determination Based on Network State Parameters. As previously stated, the weighted fusion module first sets the fusion weights based on the network state parameters (such as network latency jitter) obtained from the network state acquisition module. This preliminary setting follows the principle that when the network latency jitter is high, the first adaptive filter (fast model) is initially assigned a higher weight to enhance the system's response to path changes.

[0057] Phase Two: Fine-tuning Based on Convergence Error. After initially determining the fusion weights, the weighted fusion module further refines these weights. Specifically, the weighted fusion module obtains the first convergence error of the first adaptive filter and the second convergence error of the second adaptive filter from the parallel filtering module in parallel. The convergence error is obtained by calculating the residual between the predicted echo signal output by the corresponding filter and the near-end microphone signal. Next, the weighted fusion module compares the energy levels of the two convergence errors. If the energy of the first convergence error is significantly lower than that of the second convergence error, it indicates that the current fast model fits the actual echo path more accurately. In this case, the weights allocated to the output signal of the first adaptive filter are further increased based on the initially determined fusion weights. Conversely, if the second convergence error is smaller, the weights of the fast model are appropriately reduced, allowing the slow model to play a dominant role in a stable state.

[0058] This two-stage decision-making mechanism, which combines "macro-control (network state)" and "micro-calibration (convergence error)," makes the allocation of fusion weights more precise and real-time, greatly enhancing the adaptability and overall stability of the echo suppression system in complex time-varying environments.

[0059] Through the above scheme, this embodiment can dynamically optimize the fusion strategy of the two models according to the actual modeling effect. On the basis of macroscopic control based on network state parameters, it further combines the convergence performance of the filter itself to achieve microscopic calibration, thereby improving the accuracy and real-time performance of fusion weight allocation and enhancing the adaptability and overall stability of the echo suppression system in complex time-varying environments.

[0060] In one specific implementation, the method further includes: The stability index of the coefficient vector of the second adaptive filter within a preset time window is calculated. Specifically, the dynamic characteristics are quantified by monitoring the degree of change of the coefficient vector between consecutive time frames. The difference between the coefficient vector at the current moment and the coefficient vector at the previous reference moment is calculated, and the norm of the difference vector is obtained and normalized to the norm of the current coefficient vector, thereby obtaining a dimensionless stability index.

[0061] The stability index is compared with a preset threshold. When the stability index exceeds the preset threshold, it is determined that the echo path has undergone a drastic change that exceeds the normal time-varying range, and a compensation mechanism is triggered.

[0062] The near-end microphone signal is converted from the time domain to the frequency domain and decomposed into a time-frequency representation using a short-time Fourier transform. In the frequency domain, block processing techniques are used to quickly estimate the transfer function and compensate for echo paths to cope with sudden changes in the acoustic environment. After compensation is completed and the path stabilizes, the system reverts to its original time-domain dual-model parallel processing mode.

[0063] In an alternative embodiment, the invention also provides a supplementary mechanism for addressing significant echo path shifts caused by abrupt changes in the physical environment (e.g., user movement or device relocation). This mechanism is implemented collaboratively by a parallel filtering module and a frequency domain compensation processing logic.

[0064] First, the parallel filtering module continuously monitors the coefficient vector of the second adaptive filter (slow model). And calculate its time window. The stability index within the L2 norm. In a specific implementation, the stability index can be quantified using the normalized L2-norm variation rate, calculated as follows:

[0065] in, In the first The stability index at each sampling time. Let be the coefficient vector of the second adaptive filter at the current time. For the previous reference time ( The coefficient vector (before each sampling point), This represents the L2 norm of a vector. The value is a dimensionless index used to quantify the relative degree of change in a coefficient vector.

[0066] Next, the calculated stability index will be... Compare with a preset threshold. When detected... If the preset threshold is exceeded, it is determined that the echo path has undergone a drastic change that exceeds the normal time-varying range, and the frequency domain compensation mechanism is triggered.

[0067] Upon triggering, the system converts the near-end microphone signal from the time domain to the frequency domain, typically using a Short-Time Fourier Transform (STFT) to decompose it into a time-frequency representation. Within the frequency domain, a block processing method based on cross-power spectrum is employed for fast path compensation. Specifically, by calculating the cross-power spectrum of the far-end audio signal and the near-end microphone signal, as well as the auto-power spectrum of the far-end audio signal, the frequency domain transfer function of the current echo path is quickly estimated. Based on this, a compensation filter (e.g., a Wiener filter) is designed to process the frequency domain representation of the near-end microphone signal.

[0068] Once frequency domain compensation is complete and the stability index is monitored... Once the temperature drops below the preset threshold, indicating that the new physical path has stabilized, the system reverts to its original time-domain dual-model parallel processing mode.

[0069] In other alternative implementations, the stability index can be calculated based on different types of vector distance metrics, including but not limited to Euclidean distance, cosine similarity, or weighted Lp norm, where the weights can be configured according to the frequency response distribution; the preset threshold can be determined through offline calibration or adaptively set according to the statistical characteristics during system operation; frequency domain compensation processing can be implemented using a minimum mean square error estimation method based on cross-spectral analysis or a parameterized modeling method based on subspace decomposition.

[0070] Through the above-described scheme, this embodiment can promptly identify significant echo path shifts caused by sudden changes in the physical environment, avoiding long-term performance degradation of traditional adaptive filters due to insufficient convergence speed. Compared with existing technologies, this embodiment can maintain effective echo suppression capabilities even when encountering scenarios such as equipment movement, changes in user location, or sudden changes in room acoustic structure, significantly improving the robustness and recovery speed of the system under strong disturbance conditions.

[0071] In one specific implementation, the method further includes: dynamically adjusting a preset threshold based on the energy of the residual error signal generated by the second adaptive filter. Specifically, the system continuously monitors the residual error signal output by the second adaptive filter. And calculate its average energy or moving average energy within a preset time window. Then, this energy value is input into a nonlinear mapping function to generate a corresponding dynamic threshold. This nonlinear mapping function can be a piecewise linear function or a Sigmoid function, such that when... At a low level, Maintain at a baseline sensitivity level; while when An increase indicates that environmental noise or interference is increasing. The path changes then rise smoothly, thereby reducing the trigger sensitivity of the path change detection module and preventing normal fluctuations in the filter coefficients under high noise conditions from being misjudged as substantial changes in the acoustic path.

[0072] The energy calculation unit performs windowing processing on the continuous residual error signal and calculates its sum of squares or root mean square value to obtain a representative energy index. Next, this energy index is processed through a low-pass filter to form a stable energy estimate, which is used for subsequent threshold generation. Then, based on this energy estimate, the current threshold level to be set is determined in a mapping lookup table or analytical function and passed to the path change detection logic unit as a benchmark for judging whether the stability index exceeds the limit.

[0073] In some other alternative implementations, the dynamic adjustment mechanism may also incorporate near-end speech activity detection (VAD) results as an auxiliary criterion. When valid speech input is detected near the end, the preset threshold is further increased to prevent false detections caused by normal disturbances in the echo path model in a two-talk scenario. Alternatively, the time window length for energy calculation may be adaptively adjusted based on the historical stability of the communication link. After long-term stable operation, the window may be extended to improve estimation accuracy, while in frequently changing environments, the window may be shortened to enhance response speed.

[0074] Through the above scheme, this embodiment can achieve intelligent and context-aware adjustment of the path change detection threshold, enabling the system to maintain reasonable detection sensitivity under different noise intensities and call conditions. Compared with the prior art, this embodiment effectively reduces the false alarm rate of path changes caused by factors such as background noise and dual-talk interference, improves the overall robustness and stability of the system, and ensures sufficient response capability when the actual physical path does change.

[0075] In one specific implementation, the method further includes: separating and suppressing the nonlinear distortion residual components of the output audio signal after echo cancellation.

[0076] A high-precision linear echo prediction signal is generated using an established linear echo path model. This prediction signal is based on the far-end audio signal and processed by a well-converged adaptive filter. Specifically, this prediction signal is further subtracted from the echo suppression output of the current stage, thereby achieving deep stripping of residual linear components and effectively separating the residual signal components dominated by nonlinear distortion. Then, the energy distribution of the residual signal and the current output signal is compared and analyzed in the transform domain to construct a time-frequency related gain adjustment mechanism. Next, a dynamic suppression coefficient matrix is ​​generated based on this mechanism and applied to the time-frequency representation of the current output signal to selectively attenuate time-frequency units with a high proportion of nonlinear distortion while preserving the integrity of near-end speech.

[0077] In other alternative implementations, transform domain processing can employ time-frequency analysis methods such as short-time Fourier transform, wavelet transform, or Cohen class distribution; gain adjustment mechanisms can be implemented through threshold comparison functions, piecewise linear mapping, or nonlinear activation functions, and their parameters can be adaptively adjusted according to the system noise level or signal statistical characteristics.

[0078] Through the above-described scheme, this embodiment can effectively identify and suppress nonlinear echo components introduced by loudspeaker nonlinearity, structural resonance, and complex acoustic environments, significantly reducing residual echo energy in traditional linear echo cancellation architectures and improving the clarity and naturalness of voice communication. Compared with existing technologies, this embodiment achieves refined modeling and spatially selective suppression of nonlinear distortion through post-processing without relying on prior nonlinear models, enhancing the robustness of the echo cancellation system in real physical devices and complex acoustic scenarios.

[0079] In one specific embodiment, the step of separating and suppressing the residual components of nonlinear distortion includes: The original far-end audio signal is input to a second adaptive filter, which has converged to the linear portion of the current echo path through a slow update mechanism, thereby generating a high-fidelity second predicted linear echo signal. Specifically, this second predicted linear echo signal reflects the time-domain waveform that the far-end signal should exhibit after propagation through the physical acoustic channel without the influence of nonlinear distortion.

[0080] The second predicted linear echo signal is subtracted from the intermediate output signal after linear echo cancellation processing to further remove residual linear components. This interpolation operation significantly enhances the components related to nonlinear distortion in the remaining signal, thereby achieving effective separation of residual nonlinear distortion components.

[0081] In some other alternative implementations, the generation of the second predicted linear echo signal can be based on the output of a long filter model that has been trained in the same system, or it can be provided by a dedicated linear path estimation module deployed independently; the subtraction operation can be performed directly in the time domain, or it can be implemented in a block or overlapping manner in the transform domain (such as the frequency domain or subband domain), as long as it is mathematically equivalent to time domain subtraction.

[0082] Through the above-described scheme, this embodiment can actively construct and eliminate residual linearly correlated components using precisely modeled linear echo path information, thereby improving the identifiability and processability of nonlinear distortion components and creating favorable conditions for subsequent targeted suppression. Compared with existing technologies, this embodiment achieves efficient separation of nonlinear distortion components through signal reconstruction and cancellation mechanisms without relying on additional sensors or prior knowledge, enhancing the robustness and suppression capability of the echo suppression system in complex acoustic environments with equipment nonlinearity and spatial resonance.

[0083] In one specific embodiment, the step of separating and suppressing the residual components of nonlinear distortion further includes: The output audio signal obtained after linear echo cancellation and the nonlinear distortion residual signal separated by the "prediction-cancellation" operation are synchronously converted to the time-frequency domain. Specifically, the two signals are subjected to frame segmentation, windowing, and discrete Fourier transform processing using Short-Time Fourier Transform (STFT) to obtain their respective complex spectral representations, denoted as . and ,in Indicates the frequency bin index, Indicates the time frame index.

[0084] In each time-frequency unit The ratio of the energy of the residual nonlinear distortion signal to the energy of the linearly suppressed signal at the corresponding position is calculated, i.e., the nonlinear distortion ratio (NDR), which is expressed as:

[0085] This ratio is used to quantify the relative intensity of the nonlinear component at each time-frequency point.

[0086] Next, based on the NDR value, the corresponding gain adjustment coefficient is generated through a monotonically decreasing nonlinear mapping function. Its value range is In a typical implementation, the mapping function takes the form of a parameter-tuned Sigmoid function:

[0087] in Control the steepness of the curve, These are threshold offsets, and both can be preset or adaptively adjusted online based on device type and application scenario. When the NDR value is low, A value close to 1 indicates that the time-frequency unit mainly contains speech or noise and does not require suppression; when the NDR value is high, A value approaching 0 indicates significant nonlinear distortion, requiring intensity attenuation.

[0088] Subsequently, the calculated gain adjustment coefficient matrix is ​​multiplied point by point by the time spectrum of the linearly suppressed signal:

[0089] Complete the frequency domain masking process for nonlinear distortion.

[0090] Finally, through the analysis of Perform the inverse short-time Fourier transform (ISTFT), then overlap and add the results to reconstruct the time-domain signal, which is the final output clean audio.

[0091] In other alternative implementations, the time-frequency transformation method is not limited to STFT; improved discrete cosine transform (MDCT) or multi-resolution wavelet transform can also be used to adapt to different bandwidths or non-stationary signal characteristics. The gain adjustment coefficient can also be determined based on a lookup table method, that is, the preset mapping relationship from NDR to gain is stored in a lookup table, and the gain value is obtained in real time by interpolation. In addition, a smoothing constraint can be introduced in the gain application stage to impose an upper limit on the gain change between adjacent time-frequency units to avoid artificial noise caused by drastic amplitude modulation.

[0092] Through the above scheme, this embodiment can achieve refined and adaptive frequency domain suppression of residual nonlinear distortion. While preserving the integrity of near-end speech, it effectively reduces the nonlinear echo components caused by loudspeaker harmonic distortion, structural resonance and complex acoustic environment, thereby improving the output sound quality and communication clarity of the overall echo suppression system.

[0093] In one specific implementation, the method further includes using the output audio signal obtained after echo cancellation processing as a new error input signal to update the coefficient vectors of the first and second adaptive filters. Specifically, firstly, the output audio signal obtained by subtracting the predicted echo signal from the near-end microphone signal is acquired; this signal already contains the results of suppressing linear and nonlinear echoes. Then, using this signal as the error signal at the current moment, it is substituted into the coefficient update formula of the Normalized Least Mean Square (NLMS) algorithm to iteratively adjust the tap coefficients of the first and second adaptive filters, respectively. The coefficient update process follows standard NLMS rules:

[0094]

[0095] in, and The two filters are respectively in the first... The coefficient vector at time step, This is the reference signal vector at the current moment (i.e., the delayed frame of the far-end audio signal). This is the error signal used in this test, which is the final output audio signal. , For their respective step size parameters, To prevent small constants with zero denominators, a purer error signal is used for feedback updates. This makes the convergence direction of the filter coefficients more accurate, avoiding model drift or convergence stalling caused by residual echo interference in traditional methods. In other optional implementations, the error signal can be further processed by a nonlinear suppression module before entering the filter update stage to enhance the suppression of residual nonlinear components; alternatively, different error signal paths can be used for different filters. For example, only the signal can be used to update the slow model to ensure stability, while the fast model still uses the original residual error to maintain response speed.

[0096] Through the above scheme, this embodiment can achieve closed-loop optimization of the adaptive filter coefficient update process, so that the error signal on which the filter is based not only reflects the residual situation of linear echo, but also includes the overall headroom effect after nonlinear suppression, thereby improving the filter's approximation accuracy and convergence stability to the real echo path. Compared with the prior art, this embodiment effectively reduces the modeling bias caused by the accumulation of historical residual errors, accelerates the overall convergence speed of the system, and maintains higher consistency in echo suppression performance in dynamically changing environments.

[0097] In one specific implementation, the method further includes processing the far-end audio signal using a pre-trained speech separation model before inputting the far-end audio signal to the first adaptive filter and the second adaptive filter. Specifically, the speech separation model is configured to analyze the input mixed audio signal and decompose it into multiple semantically independent signal streams, including components highly correlated with human speech. The human voice stream signal is then extracted from the separation results as a reference signal for subsequent echo path modeling.

[0098] Next, the parallel filtering module receives the human voice stream signal and drives the first and second adaptive filters to perform their respective echo path estimation operations based on the signal. Since non-human voice components (such as background music and environmental sound effects) have been effectively stripped at the front end, the reference signal input to the adaptive filter has lower correlation interference and clearer speech structure features, thereby improving the stability and accuracy of the filter coefficient update process.

[0099] In other alternative implementations, the speech separation model may employ a blind source separation method based on signal subspace decomposition, or a spectral decomposition technique based on a statistical model; alternatively, a machine learning model with time-frequency mask generation capability may be used to distinguish and extract different types of audio components.

[0100] Through the above scheme, this embodiment can significantly reduce the interference of non-speech components in the far-end signal on the adaptive filtering process. Especially in scenarios where there is continuous background music or periodic noise at the far end, it effectively avoids the problem of filter misconvergence or divergence caused by high autocorrelation signals, thereby improving the overall robustness and steady-state performance of the echo suppression system.

[0101] In one specific implementation, the speech separation model employs a deep learning-based neural network model to separate the human voice stream signal from the non-human voice stream signal in a far-end mixed audio signal. Specifically, the neural network model is configured to receive a far-end audio signal containing human voice, background music, and noise components, and output the corresponding time-aligned human voice stream signal and non-human voice stream signal. The network architecture of this model includes an encoder-decoder structure, where the encoder extracts the time-frequency domain features of the input signal through multiple layers of convolutional kernels to form a high-dimensional feature representation; the intermediate layers introduce a time-series modeling module, such as a bidirectional gated recurrent unit (Bi-GRU) or a self-attention mechanism, to capture the long-term time-dependent characteristics of the speech signal; the decoder then restores the feature mapping to a time-domain waveform or time-spectrum graph through transposed convolution or upsampling operations to generate the separated human voice stream signal.

[0102] During the training phase, this neural network model uses a large audio dataset composed of real human voices mixed with music of different styles and environmental noise in random proportions for supervised learning. The loss function employs signal-level distortion metrics (such as SI-SNR, Scale-Invariant Signal-to-Noise Ratio) to optimize separation quality. During the inference phase, the model processes the received far-end audio frame sequence in real time and outputs a clean human voice stream signal as a reference signal for subsequent echo path modeling.

[0103] In other alternative implementations, the deep learning-based neural network model can employ different network topologies, such as: a time-frequency domain mask prediction network based on the U-Net architecture, which generates a soft mask for human voice components by combining downsampling and upsampling paths on the input short-time Fourier transform spectrum; or a time-domain end-to-end separation network based on the Conv-TasNet architecture, which performs convolution operations directly in the time domain to complete signal decomposition, avoiding phase problems caused by time-frequency transformation; or lightweight models such as DeepFilterNet can be used to adapt to the deployment requirements of resource-constrained embedded communication devices.

[0104] Through the above approach, this embodiment can effectively improve the purity of the reference signal and reduce the interference of non-human voice components on the convergence process of the adaptive filter, thereby enhancing the robustness and stability of the echo suppression system in complex audio content scenarios.

[0105] In one specific implementation, the method further includes: firstly, estimating the small frequency deviation between the sampling clocks of the far-end audio signal and the near-end microphone signal by calculating the rate of change of the phase difference at a specific frequency point over time, thus obtaining the clock drift factor. Specifically, the system selects one or more stable frequency points with high signal-to-noise ratios as monitoring targets. These frequency points can be the frequencies of preset pilot signals or the frequencies of high-energy speech formants in the spectrum. Next, short-time Fourier transforms (STFTs) are performed on the far-end audio signal and the near-end microphone signal respectively to extract the complex frequency domain coefficients at the corresponding frequency points and calculate their phase values. Then, by analyzing the first-order difference sequence of the phase difference at this frequency point between adjacent time frames, a linear slope of the phase change is fitted. This slope is proportional to the sampling rate deviation, thereby deriving the clock drift factor.

[0106] Based on the estimated clock drift factor, a fractional-order delay filter is used to dynamically resample the far-end audio signal. Specifically, this fractional-order delay filter is implemented using a Farrow architecture, supporting continuous, non-integer multiples of the sampling rate. The resampled far-end audio signal is precisely stretched or compressed on the time axis to synchronize its sampling clock reference with the actual sampling clock of the near-end microphone hardware system. Subsequently, a first and second adaptive filter, operating in parallel, receive the resampled far-end audio signal as reference input to continue echo path modeling and prediction operations.

[0107] In other alternative implementations, the clock drift factor can be estimated based on the offset of the peak position of the cross-correlation function. This involves detecting the drift trend of the delay position corresponding to the maximum correlation over time by performing cross-correlation calculations on the far-end and near-end signals within a sliding window. Alternatively, a phase-locked loop (PLL) mechanism can be used to track the phase error online and generate a compensation control signal. Furthermore, the fractional-order delay filter can be replaced by a multiphase interpolation filter bank or a window-function-based Sinc interpolation method to meet the computational complexity and accuracy requirements of different hardware platforms.

[0108] Through the above solution, this embodiment can effectively eliminate the long-term cumulative mismatch problem caused by asynchronous sampling clock, and avoid the model divergence or slow convergence caused by time drift between the reference signal and the real echo path of the adaptive filter, thereby improving the stability and robustness of the echo suppression system in long-term operation scenarios.

[0109] In one specific implementation, the method is applied to an audio device including a loudspeaker. The method further includes: before the far-end audio signal is input to the first and second adaptive filters, the signal is processed by a loudspeaker nonlinear predistortion module. Specifically, the predistortion module internally stores or learns online an inverse model characterizing the nonlinear characteristics of the target loudspeaker. This inverse model is obtained by offline measurement of the loudspeaker's harmonic distortion, frequency response compression, and phase distortion characteristics at different input levels, or by real-time updating through an online adaptive algorithm. After processing by the inverse model, the far-end audio signal generates a pre-compensation signal, which applies a pre-distortion to the original signal in the time domain that is opposite to the nonlinear characteristics of the loudspeaker. This pre-compensation signal is then input as a reference signal to the first and second adaptive filters and simultaneously sent to the loudspeaker for playback. Since the output acoustic signal of the pre-compensation signal, after passing through the physical nonlinearity of the loudspeaker, ideally recovers to a near-linear form close to the original far-end audio signal, the nonlinear source at the front end of the echo path is pre-eliminated or significantly weakened.

[0110] In other alternative implementations, the inverse model used in the loudspeaker nonlinear predistortion module can be implemented based on different modeling paradigms, including but not limited to: using static nonlinear functions (such as sigmoid or polynomial mappings) for amplitude precorrection; using dynamic system models such as the Hammerstein structure, which includes a cascade of pre-nonlinear units and subsequent linear filters; or using neural network-based deep learning models, such as feedforward fully connected networks or recurrent neural networks, to capture the complex time-varying nonlinear behavior of the loudspeaker under different operating conditions. Furthermore, the parameters of this predistortion module can be calibrated offline by periodically injecting test signals and acquiring loudspeaker output feedback, or by performing closed-loop online optimization during normal device operation using the distortion components in the residual echo signal acquired by the near-end microphone.

[0111] Through the above solution, this embodiment can effectively reduce the echo component introduced by the nonlinear distortion of the speaker itself, making the echo path faced by the subsequent adaptive filter closer to a linear system, thereby improving the accuracy and convergence speed of linear modeling. Especially in high-volume playback scenarios, this technique significantly reduces the residual echo problem caused by the inability of traditional echo suppressors to track harmonic distortion, thus improving the overall echo suppression performance and voice communication clarity.

[0112] In one specific implementation, the step size parameter of the first adaptive filter is dynamically adjusted according to changes in network state parameters. Specifically, the system acquires network state parameters in real time, representing the network communication latency jitter value, which is obtained by the receiver through RTCP receiver reports or packet timestamp analysis. First, this network state parameter is input to a mapping module, which generates a corresponding step size adjustment factor based on a preset monotonically increasing function relationship; for example, a linear mapping or piecewise linear function is used, such that the output adjustment factor increases as the jitter value increases. Then, this adjustment factor is applied to the original step size base value of the first adaptive filter to obtain the dynamic step size parameter at the current moment, thereby achieving adaptive control of the filter update rate.

[0113] To further enhance the dynamic tracking capability of the first adaptive filter (fast model), its step size parameter... The system can be dynamically adjusted based on changes in network state parameters. Specifically, after the network state acquisition module obtains network state parameters (e.g., jitter value) that characterize network communication delay jitter, its internal mapping logic generates a corresponding step size adjustment factor based on a preset monotonically increasing function. This function can be implemented using a linear mapping or a piecewise linear function, ensuring that the output adjustment factor increases as the jitter value increases. This step size adjustment factor is then provided to the parallel filtering module, which applies it to the original step size base value of the first adaptive filter to obtain the dynamic step size parameter at the current moment, thereby achieving adaptive control of the filter update rate.

[0114] Under normal network conditions, the jitter value is low, and the mapping module outputs a small adjustment factor, allowing the first adaptive filter to maintain a relatively moderate step size, balancing convergence speed and steady-state error. When the network experiences severe jitter and the jitter value rises significantly, the mapping module outputs an adjustment factor close to the upper limit, driving the step size parameter to approach the maximum allowable safe value, thereby maximizing the filter's ability to track rapidly changing echo paths. This mapping relationship can be statically configured in a lookup table or fine-tuned by an online learning mechanism based on historical performance indicators (such as the error signal energy change rate) to adapt to different devices and deployment environments.

[0115] In some other alternative implementations, the dynamic adjustment of the step size parameter can also introduce a hysteresis mechanism, that is, set two different thresholds to trigger step size increase and recovery respectively, so as to avoid frequent step size switching when the network status fluctuates frequently; or, in addition to jitter, the network status parameters can also include packet loss rate, round-trip delay change rate, etc., and generate a comprehensive network quality score by multi-parameter weighted fusion, and use this score as the basis for step size adjustment.

[0116] Through the above scheme, this embodiment enables the first adaptive filter to have a stronger dynamic response capability when facing drastic changes in network latency, rapidly improve the convergence speed during path abrupt changes, and automatically reduce the step size after the network stabilizes to reduce steady-state noise amplification, thereby significantly enhancing the system's adaptability to time-varying channels without sacrificing the overall modeling accuracy.

[0117] In one specific implementation, the step of determining the fusion weights includes: when network state parameters indicate an increase in network communication delay jitter, the system first obtains the network delay jitter value provided in real time by the network state acquisition module, which is calculated based on the statistical rate of change of the timestamps of received data packets. Specifically, the weighted fusion module dynamically adjusts the allocation strategy of the fusion weights according to the changing trend of the network delay jitter value: when a significant increase in the current jitter value relative to a preset benchmark or historical average is detected, the fusion weight α(n) allocated to the output signal of the first adaptive filter is gradually increased, while the fusion weight α(n) allocated to the output signal of the second adaptive filter is correspondingly decreased. For example, when the jitter value exceeds a certain set threshold range, the fusion weight α(n) is linearly or piecewise linearly increased to above 0.8 to allow the fast-updating, short-memory first model to dominate the predicted echo signal. This adjustment process can be further combined with a hysteresis mechanism to avoid frequent oscillatory switching near the jitter value critical point.

[0118] In some other alternative implementations, the adjustment curve for the fusion weights can be implemented using a lookup table (LUT) that stores a mapping between different jitter levels and their corresponding optimal weights; or, the adjustment process can be controlled by a lightweight state machine that divides multiple response levels based on the duration and magnitude of the jitter changes, triggering different degrees of weight shifts respectively.

[0119] Through the above scheme, this embodiment can intelligently adjust the contribution ratio of fast and slow paths in the dual model according to the real-time fluctuations in network communication quality. During periods of network instability, it prioritizes the tracking ability of time-varying echo paths, thereby effectively reducing the risk of echo residue and filter divergence caused by sudden delay changes, and improving the dynamic adaptability and stability of the echo suppression system in complex network environments.

[0120] This invention provides an adaptive echo path modeling system for echo suppression, applicable to audio signal processing scenarios in network communication devices. For example... Figure 2 As shown, the system includes: The signal receiving module M100 is configured to receive far-end audio signals and acquire near-end microphone signals containing echo components. The far-end audio signal originates from voice stream data transmitted over the network, while the near-end microphone signal is a locally picked-up sound signal, which includes the echo generated after the far-end user's voice is played through the speaker, possible near-end user voice, and environmental noise.

[0121] The parallel filtering module M200 integrates a first adaptive filter and a second adaptive filter, configured to process far-end audio signals in parallel. The first adaptive filter uses a larger step size and a shorter filter length, resulting in faster response and higher dynamic tracking capability; the second adaptive filter uses a smaller step size and a longer filter length, resulting in higher modeling accuracy and better steady-state performance. The two filters operate independently in the time domain, each generating its own echo prediction output based on the same reference signal.

[0122] The network status acquisition module M300 is configured to acquire network status parameters reflecting latency variations from the network communication link in real time. These parameters are obtained by parsing the receiver reports of the Real-time Transmission Control Protocol (RTCP) or analyzing the timestamp sequence of media data packets, and are used to quantify the degree of fluctuation in the arrival time of data packets in the current network.

[0123] The weighted fusion module M400, connected to the parallel filtering module M200 and the network state acquisition module M300, is configured to dynamically determine the fusion weights between the output signals of the first and second adaptive filters based on the network state parameters provided by the network state acquisition module M300, and perform a weighted summation operation to generate the final predicted echo signal. The weight allocation strategy prioritizes the output of the first filter with a fast response when the network state is unstable, while emphasizing the use of the more accurately modeled output of the second filter when the network state is stable.

[0124] The echo cancellation module M500 is configured to subtract the predicted echo signal from the original near-end microphone signal, thereby generating an intermediate output signal after preliminary linear echo cancellation. This module may further include an error feedback path to provide residual information to the parallel filtering module M200 to support continuous updates of the filter coefficients.

[0125] In one embodiment, both the first and second adaptive filters are implemented based on the Normalized Least Mean Square (NLMS) algorithm. The first adaptive filter has a filter length of 128 taps and a step size parameter set between 0.8 and 1.0; the second adaptive filter has a filter length of 512 taps and a step size parameter set between 0.1 and 0.3. The fusion weight α(n) in the weighted fusion module M400 is a scalar value that varies with time, ranging from [0,1]. Its specific value is calculated from the network state parameters using a nonlinear mapping function. For example, when increased network jitter is detected, α(n) approaches 1, indicating that it mainly relies on the output of the first adaptive filter; otherwise, it approaches 0. In some other optional embodiments, the adaptive filter may also employ the affine projection algorithm (APA) or the recursive least squares (RLS) method, and the determination of the fusion weight may also incorporate additional information sources such as historical path stability indices or frequency domain coherence analysis.

[0126] The aforementioned modules are deployed collaboratively on the same hardware platform via a digital signal processor (DSP) or application-specific integrated circuit (ASIC), and the signal is processed in a pipelined manner frame by frame. The signal receiving module M100 synchronously distributes the received data to the parallel filtering module M200 and the network status acquisition module M300; the parameters output by the network status acquisition module M300 are used as context information input to the weighted fusion module M400; the parallel filtering module M200 outputs two prediction signals for use by the weighted fusion module M400; finally, the echo cancellation module M500 completes the signal reconstruction.

[0127] The collaborative working relationships among these components include: the dynamic context information provided by the network state acquisition module M300 guides the weighted fusion module M400 to rationally allocate the outputs of two filters with different characteristics, enabling the system to maintain rapid response in highly dynamic network environments while maintaining high-precision modeling capabilities during stable periods. The fast-slow dual-model architecture and weighted fusion mechanism in the parallel filtering module M200 form a closed-loop cooperation, resolving the technical contradiction that traditional single adaptive filters struggle to balance convergence speed and steady-state error. The echo cancellation module M500 utilizes the high-quality prediction signal generated by fusion to perform cancellation operations, significantly reducing residual echo energy. The overall structure achieves efficient tracking and modeling of time-varying echo paths, making it particularly suitable for real-world communication scenarios with severe network jitter.

[0128] Compared with existing technologies, this embodiment constructs a dual-filter parallel architecture driven by network state and a dynamic weighted fusion mechanism. The system can effectively cope with rapid fluctuations in network latency without sacrificing steady-state suppression accuracy, improve the robustness and adaptability of the echo suppression algorithm under complex network conditions, avoid filter divergence caused by path abrupt changes, and thus ensure the quality and stability of voice communication.

[0129] In one specific implementation, the weighted fusion module M400 is configured to dynamically adjust the fusion weights by incorporating the convergence error of the first adaptive filter during the process of determining the fusion weights based on network state parameters. Specifically, the weighted fusion module M400 receives the first predicted echo signal output by the first adaptive filter and the corresponding convergence error signal from the parallel filtering module M200. This convergence error signal is obtained by the difference between the near-end microphone signal and the predicted echo signal generated after weighted fusion at the current time. The weighted fusion module M400 calculates the energy or average power of the convergence error signal in real time and compares it with the energy of the corresponding convergence error of the second adaptive filter. When the convergence error energy of the first adaptive filter is significantly lower than that of the second adaptive filter, it indicates that the fast model fits the echo path more accurately in the current time-varying environment. Accordingly, the weighted fusion module M400 applies a positive bias to the fusion weights initially determined by the network state parameters, appropriately increasing the weight ratio allocated to the output signal of the first adaptive filter. Conversely, if the error energy is high, the weights are negatively corrected to limit the dominance of the fast model and prevent system instability due to over-response.

[0130] In other alternative implementations, the convergence error can be evaluated using normalized error energy, higher-order statistical characteristics of the error signal, or error trend analysis within a sliding time window; the adjustment strategy for the fusion weights can be based on lookup tables, piecewise linear functions, or lightweight nonlinear mapping functions (such as Sigmoid or ReLU variants) to improve decision accuracy while ensuring computational efficiency.

[0131] Through the above scheme, this embodiment can dynamically optimize the fusion strategy of the two models according to the actual modeling effect. It not only relies on external network indicators, but also introduces an internal error feedback mechanism to improve the accuracy of weight allocation and system adaptability, thereby maintaining more stable echo suppression performance under complex and ever-changing communication conditions.

[0132] In one specific implementation, the system further includes a nonlinear processing module, which comprises a linear echo prediction generation unit, a time-frequency conversion and analysis unit, a gain adjustment coefficient calculation unit, and a time-frequency mask application unit. The linear echo prediction generation unit is configured to receive the original far-end audio signal and input it to a second adaptive filter to generate a second predicted linear echo signal. This second predicted linear echo signal is provided to the intermediate signal output by the echo cancellation module M500, i.e., the residual signal obtained by subtracting the fused predicted echo signal from the near-end microphone signal, for further cancellation operations, thereby separating the signal component mainly containing nonlinear distortion residual components.

[0133] The time-frequency conversion and analysis unit is configured to perform short-time Fourier transforms on the residual signal and the nonlinear distortion residual signal respectively to obtain their complex spectral representations in the time-frequency domain. In each frequency-time unit, the ratio of the energy of the nonlinear distortion residual signal to the energy of the residual signal is calculated as the nonlinear distortion ratio (NDR). Based on this NDR value, the gain adjustment coefficient calculation unit generates the corresponding gain adjustment coefficient through a monotonically decreasing nonlinear function mapping, so that when the NDR is low, the coefficient is close to 1, allowing the signal to pass, while when the NDR is high, the coefficient approaches 0, achieving strong suppression of the time-frequency unit. The time-frequency mask application unit then multiplies the gain adjustment coefficient matrix point by point by the time-frequency representation of the residual signal to complete the directional attenuation of nonlinear distortion.

[0134] In other alternative implementations, the gain adjustment coefficient can be implemented using various nonlinear functions, including but not limited to sigmoid functions, piecewise linear functions, or lookup tables; the time-frequency transformation method is not limited to short-time Fourier transform, but can also employ wavelet transform, filter banks, or multi-resolution analysis tools; the calculation of the nonlinear distortion ratio can also incorporate a smoothing mechanism, such as exponentially weighted averaging, to improve its stability in noisy environments; the mask type is not limited to multiplicative amplitude masks, but can also employ complex spectral masks, while adjusting the phase information to reduce artificial noise.

[0135] Through the above solution, this embodiment can effectively identify and suppress nonlinear echo components caused by speaker nonlinearity, structural resonance and complex acoustic reflection. Without relying on additional sensors or external calibration, it can achieve high-precision residual echo separation and fine processing by using the output of the existing model inside the system, which significantly improves the overall performance of the echo suppression system. Especially in scenarios with high volume playback, strong reverberation or obvious harmonic distortion of the equipment, it can still maintain good speech clarity and naturalness.

[0136] In one specific implementation, the system further includes a speech separation module, located between the signal receiving module M100 and the parallel filtering module M200, and configured to perform source separation processing on the received far-end audio signal. This speech separation module is constructed to decompose the mixed far-end audio signal containing human voice and non-human voice components into two independent signal streams: a human voice stream signal and a non-human voice stream signal. The human voice stream signal serves as the reference input for subsequent echo path modeling and is provided to the first and second adaptive filters in the parallel filtering module M200 to generate predicted echo signals. The speech separation module is implemented using a deep learning-based neural network model, specifically including a time-frequency domain convolutional encoder-decoder structure. Its network architecture is U-Net or Conv-TasNet, and it learns the time-frequency distribution pattern of human voice audio features through pre-training, thereby achieving high-precision human voice extraction in multi-source mixed scenarios. The input of this module is the time-frequency representation of the original far-end audio signal, and the output is the reconstructed human voice time-domain signal after masking processing. In other alternative implementations, the speech separation module may also employ traditional blind source separation algorithms, such as Independent Component Analysis (ICA) or Non-negative Matrix Factorization (NMF), to achieve basic separation of human voice and background music in hardware environments where deep learning deployment is not feasible.

[0137] Through the above scheme, this embodiment can effectively reduce the interference of non-human voice signal components on the convergence process of the adaptive filter. Especially when the far-end signal contains background music, environmental noise or multimedia playback content, it can significantly improve the purity of the reference signal, thereby enhancing the accuracy and stability of echo path modeling and reducing the risk of filter divergence caused by strong autocorrelation non-speech signals.

[0138] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An adaptive echo path modeling method for echo suppression, applied to sound analysis and processing of network communication devices, characterized in that, include: Receives far-end audio signals and acquires near-end microphone signals including echoes; The far-end audio signal is processed in parallel using a first adaptive filter and a second adaptive filter, wherein the step size parameter of the first adaptive filter is greater than the step size parameter of the second adaptive filter, and the filter length of the first adaptive filter is less than the filter length of the second adaptive filter. Obtain network state parameters that characterize network communication latency jitter. Based on the network state parameters, determine the fusion weights assigned to the output signals of the first adaptive filter and the second adaptive filter; The output signals of the first adaptive filter and the second adaptive filter are weighted and fused to generate a predicted echo signal. The predicted echo signal is subtracted from the near-end microphone signal to obtain the echo-cancelled output audio signal; The output audio signal after echo cancellation is subjected to separation and suppression of nonlinear distortion residual components. The steps of separating and suppressing nonlinear distortion residual components include: The far-end audio signal is input into the second adaptive filter to generate a second predicted linear echo signal; The second predicted linear echo signal is subtracted from the echo-eliminated output audio signal to separate the nonlinear distortion residual component; Before processing the far-end audio signal in parallel using the first adaptive filter and the second adaptive filter, the human voice stream signal is separated from the far-end audio signal using a speech separation model; The step of processing the far-end audio signal in parallel using the first adaptive filter and the second adaptive filter includes: processing the human voice stream signal in parallel using the first adaptive filter and the second adaptive filter.

2. The method according to claim 1, characterized in that, The network status parameters include network latency jitter values.

3. The method according to claim 1, characterized in that, The step of determining the fusion weights assigned to the output signals of the first adaptive filter and the second adaptive filter based on the network state parameters includes: The fusion weights are initially determined based on the network state parameters; The first convergence error of the first adaptive filter and the second convergence error of the second adaptive filter are obtained in parallel. Based on the comparison between the first convergence error and the second convergence error, the initially determined fusion weights are adjusted to obtain the final fusion weights.

4. The method according to claim 1, characterized in that, The method further includes: Calculate the stability index of the coefficient vector of the second adaptive filter within a preset time window; When the stability index exceeds a preset threshold, the near-end microphone signal is converted from the time domain to the frequency domain, and the converted frequency domain signal is compensated. The preset threshold is dynamically adjusted based on the energy of the residual error signal generated by the second adaptive filter.

5. The method according to claim 1, characterized in that, The step of separating and suppressing the nonlinear distortion residual components further includes: The nonlinear distortion residual components of the echo-eliminated output audio signal are all converted to the time-frequency domain. The gain adjustment coefficient is determined based on the energy ratio of the residual nonlinear distortion component to the echo-eliminating output audio signal in the time-frequency unit. The time-frequency domain representation of the echo-canceled output audio signal is processed using the gain adjustment coefficient.

6. The method according to claim 1, characterized in that, The method further includes: The echo-cancelled output audio signal is used as the input signal to update the coefficient vectors of the first adaptive filter and the second adaptive filter.

7. The method according to claim 1, characterized in that, The step of determining the fusion weight includes: When the network state parameter indicates an increase in network communication delay jitter, the fusion weight allocated to the output signal of the first adaptive filter is increased, and the fusion weight allocated to the output signal of the second adaptive filter is decreased.

8. An adaptive echo path modeling system for echo suppression, used to implement the adaptive echo path modeling method for echo suppression as described in any one of claims 1 to 7, applied to the sound analysis and processing of network communication devices, characterized in that, include: The signal receiving module is configured to receive far-end audio signal acquisition from a near-end microphone signal that includes echo; A parallel filtering module, which includes a first adaptive filter and a second adaptive filter, is configured to process the far-end audio signal in parallel using the first adaptive filter and the second adaptive filter, wherein the step size parameter of the first adaptive filter is greater than the step size parameter of the second adaptive filter, and the filter length of the first adaptive filter is less than the filter length of the second adaptive filter. The network status acquisition module is configured to acquire network status parameters that characterize the network communication delay jitter status. A weighted fusion module is connected to the parallel filtering module and the network state acquisition module, and is configured to determine the fusion weights allocated to the output signals of the first adaptive filter and the second adaptive filter based on the network state parameters acquired by the network state acquisition module, and to perform weighted fusion on the output signals of the first adaptive filter and the second adaptive filter to generate a predicted echo signal. An echo cancellation module is configured to subtract the predicted echo signal from the near-end microphone signal to obtain an echo-cancelled output audio signal.

Images