A Linux system echo cancellation method based on a DTLN-AEC model and an intelligent device

Abstract

The application discloses a Linux system echo cancellation method based on a DTLN-AEC model and intelligent equipment. The method comprises the following steps: resampling the collected audio data to a preset sampling rate, and extracting the amplitude spectrum of the near-end microphone signal and the far-end reference signal through a short-time Fourier transform and performing smooth denoising preprocessing; constructing a DTLN-AEC model; adding a timestamp to each audio frame under the Linux system, realizing the synchronization of collection and processing through a ring buffer, the ring buffer having a preset frame number threshold, and triggering model reasoning when the cache frame number reaches the threshold. The application performs full-link lightweight design on the DTLN-AEC model, and designs a frame synchronization mechanism according to the audio processing characteristics of the Linux system, so that the system performance of end-to-end processing delay < 20 ms, CPU occupancy < 30%, and memory occupancy < 4 MB is realized while the high echo cancellation quality is ensured, so as to adapt to embedded Linux devices such as intelligent access control and building intercoms.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing technology, and in particular to a Linux system echo cancellation method and intelligent device based on the DTLN-AEC model. Background Technology

[0002] In full-duplex voice communication scenarios such as smart access control and building intercoms, devices are typically installed in complex acoustic environments such as corridors and main entrances, resulting in problems such as long reverberation times (up to 1.5 seconds for the T60) and diverse background noise (such as conversations, footsteps, and wind noise). After the remote voice signal is played through the local speaker, it is picked up again by the microphone through the acoustic coupling path, forming an acoustic echo that seriously interferes with call quality.

[0003] Currently, the most widely used echo cancellation scheme on Linux embedded systems is a port of the WebRTC AEC3 algorithm. This scheme estimates the echo path based on adaptive filters (such as NLMS) and processes the backend through dual-talk detection and residual spectrum subtraction. Specifically, it uses a 16kHz sampling rate, a 20ms frame length, a 128-tap NLMS filter, and relies on a fixed power ratio threshold for dual-talk decision-making. At the Linux system level, a single-process processing mode is typically used, with audio frame acquisition and processing synchronized through a circular buffer.

[0004] However, this solution has the following significant drawbacks in actual access control and intercom scenarios: Poor environmental adaptability: The fixed 128-tap filter can only cover the impulse response of about 128ms, which cannot fully model the long reverberation environment (more than 256 taps are required when T60≥1.5s), resulting in an echo suppression gain (ERLE) of only 28~32dB under high reverberation, with obvious residual echo.

[0005] Nonlinear echo processing failure: Based on the linear system assumption, it is impossible to eliminate the nonlinear distortion generated by the high power output of the loudspeaker, and the residual nonlinear echo reaches -25~-30dB, which can easily trigger howling.

[0006] Real-time performance is not up to standard: the end-to-end processing latency is 25~35ms, which exceeds the usual requirement of less than 20ms for access control intercoms, and the CPU usage is too high, affecting other functions of the device (such as unlocking and video transmission).

[0007] Dual-talk detection has a high false alarm rate: relying solely on the power ratio as a single criterion, the accuracy is only 84%~86% when there is high background noise or network jitter, and the false alarm rate is 8%~12%, which leads to the failure of echo cancellation or the false cancellation of near-end speech.

[0008] Besides traditional signal processing methods, echo cancellation technology based on deep learning has also made progress in recent years. Among them, the DTLN (Dual-signal Transformation LSTM Network) model, by combining frequency and time domain dual feature extraction, LSTM time-series modeling, and mask decoding, has shown excellent performance in nonlinear echo suppression. However, when the original DTLN-AEC model is directly deployed on resource-constrained embedded Linux devices, it suffers from problems such as a large number of parameters, high computational load, and high memory consumption, making it difficult to meet the low latency and low power consumption requirements of real-time communication.

[0009] In addition, some lightweight echo cancellation solutions have been proposed in published patent applications. For example: CN120340517A (Goertek Inc.) employs a hybrid architecture of linear filtering and neural networks. It first performs preliminary linear echo cancellation using NLMS, and then extracts Bark spectral features to input into a small neural network for secondary processing. This solution relies on the cascading of traditional signal processing and shallow neural networks, does not adopt an end-to-end deep model, and lacks low-level adaptation for the Linux system.

[0010] CN119694330B (Guangzhou Maritime University, etc.) proposes a decoupled dual-branch network to handle the amplitude spectrum and complex spectrum separately, and introduces Dual... path Skip The Transformer and SCConv modules aim to reduce the number of parameters. However, their lightweight design is mainly limited to network architecture design and does not address model quantization, inference framework deployment, or synchronous optimization at the operating system level.

[0011] CN121056569A (Guangdong Baolun Electronics Co., Ltd.) is based on an improved DeepFilterNet framework, utilizing ERB scale gain and deep filtering networks to enhance the periodic components of speech, and designs multi-task losses such as echo perception and VAD. However, its model base differs from that of this invention, and it does not propose solutions to problems such as audio driver synchronization and real-time scheduling of the circular buffer in embedded Linux systems.

[0012] In summary, existing echo cancellation solutions either suffer from poor echo suppression performance and insufficient real-time performance in complex acoustic environments, or face challenges such as high computational load, large memory consumption, and difficult system adaptation when deployed on resource-constrained Linux embedded systems. How to effectively lightweight and deeply integrate high-performance deep learning echo cancellation models (especially DTLN-AEC) with Linux systems, while simultaneously meeting the stringent requirements of high echo suppression gain (≥45dB), low processing latency (<20ms), and low resource consumption (CPU <30%, memory <4MB), has become a pressing technical challenge in this field. Summary of the Invention

[0013] In view of this, the purpose of this invention is to propose an echo cancellation method for Linux systems based on the DTLN-AEC model. By performing a lightweight end-to-end design of the DTLN-AEC model (including input sampling rate adjustment, depthwise separable convolution replacement, model structure optimization, quantization and deployment optimization), and designing a frame synchronization mechanism (timestamp and circular buffer threshold control) specifically for the audio processing characteristics of Linux systems, this method achieves high echo cancellation quality (ERLE≥45dB, nonlinear echo suppression -38~-42dB, dual-talk detection accuracy≥97%) while maintaining system performance with end-to-end processing latency <20ms, CPU usage <30%, and memory usage <4MB, thus adapting to embedded Linux devices such as smart access control and building intercom systems.

[0014] According to one aspect of the present invention, a method for echo cancellation in a Linux system based on the DTLN-AEC model is provided, the method comprising: S1. Resample the collected audio data to a preset sampling rate, and extract the amplitude spectrum of the near-end microphone signal and the far-end reference signal through short-time Fourier transform and perform smoothing and noise reduction preprocessing. S2. Construct the DTLN-AEC model, which includes: A frequency domain feature encoder is used to perform nonlinear transformation on the amplitude spectrum to extract frequency domain features; The temporal feature encoder uses depthwise separable convolution to extract temporal features; The feature fusion module is used to stitch together and fuse the frequency domain features and time domain features of the near end and the far end; Unidirectional LSTM is used for temporal modeling of the fused features; Channel attention mechanism and temporal attention mechanism are applied sequentially to the output features of LSTM, respectively performing adaptive weighted enhancement on the channel dimension and the temporal dimension; A dual-mask decoder is used to generate frequency domain masks and time domain masks; The mask fusion module is used to fuse the frequency domain mask and the time domain mask with the original near-end amplitude spectrum to obtain the enhanced amplitude spectrum; The DTLN-AEC model is quantized and then deployed in a lightweight manner. S3. Under the Linux system, add a timestamp to each audio frame and realize the synchronization of acquisition and processing through a circular buffer. The circular buffer has a preset frame number threshold. When the number of cached frames reaches the threshold, model inference is triggered. S4. Input the synchronized near-end and far-end amplitude spectra into the deployed DTLN-AEC model. After model processing, the enhanced amplitude spectrum is output. Then, the time-domain signal is reconstructed through inverse short-time Fourier transform to obtain clean near-end speech.

[0015] In the above technical solution, a full-link processing method from audio input and model inference to system synchronization is proposed to meet the real-time echo cancellation requirements of embedded Linux scenarios such as smart access control and building intercom. Its core objective is to ensure high echo suppression performance (ERLE≥45dB, nonlinear echo suppression below -38dB) while meeting the strict real-time requirements (end-to-end latency <20ms) and resource constraints (CPU usage <30%, memory <4MB) of embedded devices.

[0016] Unlike existing technologies that focus on isolated improvements, the key feature of this solution lies in its integrated design of lightweight model, enhanced inference, and operating system adaptation. Specifically: First, at the input and model level, the scheme resamples the audio to a lower preset sampling rate (e.g., 8kHz) and uses short-time Fourier transform to extract the amplitude spectrum, reducing the feature dimensionality from the source. The frequency domain encoder employs a nonlinear compression-expansion transform, and the time domain encoder introduces depthwise separable convolution. The combination of these two significantly reduces the number of model parameters and computational cost while preserving multi-domain feature information. Based on this, a unidirectional LSTM and a serial channel-time dual attention mechanism enable the model to adaptively focus on the frequency and time domain features of effective near-end speech, and achieve dual suppression of echo and noise through the fusion of the product of the frequency and time domain masks. Compared to schemes using only a single mask or ordinary convolution, this structure improves the suppression accuracy of echo and noise under complex reverberation and nonlinear echoes with only a limited increase in the number of parameters.

[0017] Secondly, at the deployment and system synchronization level, the solution quantizes and lightweightes the model for deployment, further compressing the model size and accelerating inference. The solution explicitly designs a frame synchronization mechanism based on timestamps and a circular buffer under the Linux system: each audio frame is timestamped, and a preset frame count threshold is set for the circular buffer; inference is triggered only when the cached frame count reaches the threshold. This mechanism effectively solves the data alignment and congestion problems between the audio driver and application processing in embedded Linux, avoiding frame loss or waiting delays that are prone to occur with traditional single-process circular buffers, thus ensuring the stability of end-to-end latency.

[0018] In summary, this scheme reduces model complexity by employing a lower sampling rate, enabling the quantized and lightweight deployed model to quickly complete inference under ring buffer threshold triggering. Furthermore, the high-precision suppression capabilities of dual attention and dual masks compensate for potential performance losses due to resource compression. Overall, this scheme achieves efficient cancellation of both linear and nonlinear echoes using a single DTLN-AEC model without relying on complex architectures such as traditional cascaded adaptive filters or decoupled dual-branch systems, and achieves verifiable real-time performance metrics in a Linux embedded environment.

[0019] In some embodiments, the preset sampling rate is 8kHz to 16kHz, and the preset frame rate threshold is 2 to 5 frames.

[0020] In the above technical solution, the sampling rate range is further defined as 8kHz to 16kHz, and the trigger frame threshold for the circular buffer is 2 to 5 frames. This range selection is an optimized interval derived after comprehensively weighing the voice frequency band (300Hz to 3400Hz), reverberation time (0.2s to 2.0s), and audio driver buffer characteristics of the Linux embedded system in the access control intercom scenario.

[0021] First, the sampling rate range of 8kHz to 16kHz is based on a balance between speech fidelity and computational load. Below 8kHz, speech band loss occurs (Nyquist frequencies below 4kHz), affecting the intelligibility and naturalness of near-speech; above 16kHz, it exceeds the main range of human speech perception, but significantly increases the number of STFT points, frequency domain feature dimensions, and the computational load of subsequent LSTM and attention mechanisms. This scheme limits the sampling rate to this range, which fully preserves the core frequency band information of speech while avoiding redundant high-frequency computations, allowing the model to run on ARM Cortex. Real-time inference on A7-class devices is now possible. This range offers a flexible gradient selection to adapt to different hardware capabilities (e.g., 16kHz for slightly higher computing power platforms, 8kHz for extremely lightweight requirements) while still ensuring the core metric of ERLE≥45dB.

[0022] Secondly, setting the circular buffer threshold to 2-5 frames is an adaptation to the latency of Linux audio interrupt handling and model inference. A threshold that is too low (e.g., 1 frame) will lead to frequent inference triggers, increasing CPU scheduling overhead, and making it difficult to optimize single-frame processing using batch parallelism; a threshold that is too high (>5 frames) will introduce additional buffer latency, accumulating beyond the 20ms real-time requirement. This solution limits the threshold to 2-5 frames, allowing the system to trigger model inference after accumulating 2-5 frames (i.e., an audio window of 20ms-50ms) at the natural frame rate driven by ALSA at the acquisition end (typically 10ms per frame). This allows the model to utilize multi-frame context for more accurate echo mask estimation (especially for modeling the reverberation tail), and reduces the average overhead per frame through batch processing. Compared to the fixed single-frame triggering or the lack of threshold control in the general circular buffer of existing WebRTC AEC3, this solution's preset threshold mechanism improves the robustness and latency predictability of the audio processing chain under the Linux system.

[0023] In summary, when the sampling rate is high (e.g., 16kHz), the amount of data per frame increases. In this case, using a lower frame threshold (e.g., 2 frames) can control the input tensor size and avoid memory overload. When the sampling rate is low (e.g., 8kHz), a higher threshold (e.g., 5 frames) can be used to utilize a longer temporal context to improve reverberation suppression, while the total latency remains within 20ms. This cooperative adaptation is not disclosed in existing technologies.

[0024] In some embodiments, the frequency domain feature encoder consists of a three-layer fully connected network, each layer performing linear transformation, layer normalization, GELU activation, and Dropout operations in sequence. The first layer compresses the high-dimensional input features to a low-dimensional dimension, the second layer expands the low-dimensional features by a factor of 2, and the third layer re-compresses the expanded features to a low dimension.

[0025] In the above technical solution, a three-layer fully connected coding structure of "compression-expansion-recompression" is designed in the frequency domain processing branch of the DTLN-AEC model. This structure extracts highly discriminative frequency domain features under controllable parameter scale by compressing the dimension, expanding the features, and reprojecting the input amplitude spectrum, providing information-rich representations for subsequent time series modeling and mask generation.

[0026] The key feature of this encoder lies in its use of an asymmetric dimensionality transformation path and a specific normalization. activation Regularization combination achieves a balance between refined extraction of frequency domain features and overfitting suppression.

[0027] Specifically, the first layer compresses the original amplitude spectrum to a low dimension, forcing the network to learn the most representative low-dimensional frequency domain basis and eliminating irrelevant fluctuations between redundant frequency points; the second layer expands the low-dimensional features by a factor of two (dimensionality increase), providing the network with sufficient expressive power to capture subtle spectral features of echo and near-end speech in harmonic structure, phase differences, etc.; the third layer compresses it back to a low dimension, projecting the high-dimensional features into a compact space suitable for interfacing with LSTM and attention mechanisms. This asymmetric "compression" Extend The "compressed" path, compared to directly using single-layer mapping or fixed-dimensional transformation, can achieve better nonlinear fitting capabilities with the same number of parameters.

[0028] Furthermore, configuring linear transformation, layer normalization (LN), GELU activation, and Dropout in sequence for each layer yields the following effects: Layer normalization (LN) replaces the commonly used batch normalization (BN), is unaffected by the statistics of small batch samples, and is compatible with the smooth nonlinear characteristics of the GELU activation function, ensuring the stability of the feature distribution during training and inference. For sudden changes in near-end speech and echo in access control scenarios, LN can quickly adapt to the amplitude spectrum distribution of a single sample, avoiding feature shifts caused by sudden environmental changes.

[0029] Compared to ReLU, GELU (Gaussian Error Linear Unit) retains a small non-zero output even when the input is negative, making it more suitable for the transmission of weak signal components in the audio amplitude spectrum (such as long-distance near-end speech or low-energy echoes); at the same time, its smooth nonlinear characteristics are beneficial to gradient flow and reduce saturation problems in deep networks.

[0030] Dropout (dropout rate 0.2) introduces mild regularization after each fully connected layer to prevent the encoder from over-relying on specific frequency domain patterns, thereby improving the model's ability to generalize to spectral variations caused by different microphones and installation locations.

[0031] In summary, without increasing the dimensionality of hidden layers, this encoder enhances the richness of feature representation through "extension layers." Compared to directly using two fully connected layers of the same dimension, this encoder exhibits higher frequency domain discrimination for echoes and noise, and particularly improves the mask estimation accuracy for nonlinear distortion. The combination of LN+GELU+Dropout achieves a good balance between training convergence speed and generalization ability, converging within ≤100 epochs in actual training, and the validation set SI... SDR is significantly improved. The number of parameters in the entire encoder is controllable (mainly affected by the hidden layer dimension), and it works in conjunction with the subsequent depthwise separable convolutional temporal encoder and LSTM. The overall model runs on ARM Cortex. Memory usage on the A7 platform is less than 4MB.

[0032] In some embodiments, the temporal feature encoder adopts a structure combining depthwise separable convolution and residual connections, wherein the depthwise separable convolution includes depthwise convolution and 1×1 point convolution, and the residual connections use 1×1 convolution for dimension matching when the input and output dimensions are inconsistent.

[0033] In the above technical solution, an encoder structure combining depthwise and pointwise convolutions with adaptive residual connections is used in the time-domain processing branch of the DTLN-AEC model to extract local dynamic features (such as echo delay envelope and noise impulses) from the original time-domain waveform. This design aims to capture the fine structure of the time domain with extremely low computational overhead and complement the frequency domain features.

[0034] The essential feature of a temporal encoder is that it combines the low complexity of depthwise separable convolution with the gradient preservation capability of residual connections, and uses 1×1 convolution to achieve flexible projection in dimension matching.

[0035] The core advantage of depthwise separable convolution lies in decomposing standard convolution into two steps: depthwise convolution spatially convolves each input channel individually, capturing local temporal patterns within a single channel (e.g., short-term delays in echoes, pitch periods in speech); subsequently, 1×1 pointwise convolution linearly combines the outputs of each channel, achieving cross-channel information fusion. Compared to ordinary convolution, this decomposition reduces computation to approximately 1 / Cin (where Cin is the number of input channels). In this scheme, the temporal encoder deals with the raw PCM audio stream (typically 1 or 2 channels). Depthwise separable convolution can extract sufficiently rich temporal features with minimal multiply-accumulate operations, while avoiding the parameter explosion caused by ordinary convolution, directly contributing to the achievement of ARM Cortex. The CPU usage on the A7 is less than 30%.

[0036] The introduction of residual connections addresses the vanishing gradient problem that can occur as the encoder layers deepen. Since temporal signals are sensitive to subtle time shifts, deep networks that cannot effectively propagate gradients can lead to inaccurate echo edge localization. The residual connections in this scheme allow the input signal to bypass directly to the output, forcing the network to learn a residual function relative to the input, thereby stabilizing training and improving feature reuse. When the input and output dimensions are inconsistent (e.g., due to changes in the number of channels after depthwise convolution), the scheme uses 1×1 convolutions for dimension matching. This adaptive matching method is more accurate than identity mapping (only applicable to the same dimensions) or simple padding / clipping, ensuring the integrity of the residual path without introducing additional nonlinear transformations and avoiding information distortion.

[0037] In summary, depthwise separable convolution reduces the number of multiply-accumulate operations in the temporal encoder by an order of magnitude compared to ordinary convolution. Combined with the subsequent frequency-domain encoder and LSTM, the overall model achieves an end-to-end latency of approximately 15ms at an 8kHz sampling rate. Residual connections allow gradients to flow directly to shallow layers, enabling the deep convolution kernel to learn longer temporal dependencies (e.g., echo wakes within a 20ms frame), which is particularly beneficial for echo suppression in reverberant environments. The 1×1 convolution dimension matching mechanism allows the encoder to flexibly connect to front-ends with different channel numbers (e.g., single-microphone or dual-microphone arrays) without modifying the network backbone, improving the portability of the solution on various Linux access control hardware. Compared to existing temporal encoders using ordinary convolution or fixed-dimensional residual blocks, this solution, through the combination of depthwise separable convolution and adaptive residuals, significantly reduces the number of parameters and inference time while maintaining temporal feature representation capabilities, making it an indispensable optimization for meeting embedded real-time requirements.

[0038] In some embodiments, the channel attention mechanism generates a channel attention mask through global average pooling, multilayer perceptron, and sigmoid function; the temporal attention mechanism generates a temporal attention mask through one-dimensional convolution and Softmax function; the channel attention mechanism and the temporal attention mechanism are executed sequentially, with channel attention being executed first and then temporal attention.

[0039] In the above technical solution, in DTLN Following the LSTM temporal modeling of the AEC model, channel attention and temporal attention mechanisms are further embedded sequentially. Channel attention generates a channel-dimensional mask through global average pooling, a multilayer perceptron (MLP), and a sigmoid function, while temporal attention generates a temporal-dimensional mask through one-dimensional convolution and a softmax function. The sequential order of channel attention execution is strictly defined, with channel attention performed first, followed by temporal attention. This design aims to progressively enhance features through "channel filtering—temporal focusing" to improve the model's accuracy in extracting effective near-end speech.

[0040] The key features of this scheme in terms of attention mechanism include: channel attention using global pooling + MLP + Sigmoid, temporal attention using one-dimensional convolution + Softmax, and a fixed sequential execution order. These choices are specifically designed for the echo cancellation task.

[0041] In echo cancellation tasks, different feature channels often correspond to different frequency bands or time-frequency patterns. Some channels may primarily encode the harmonic structure of near-end speech, while others may carry periodic components of echoes or background noise. By compressing the temporal information of each channel into a scalar through global average pooling, and then using two layers of MLP (with reduction ratio) and the sigmoid function to generate a channel mask in the range [0,1], the model can automatically suppress those channels dominated by echo / noise and enhance those effective for speech. Compared with heuristic methods such as directly using channel-level standard deviation or energy thresholds, this data-driven channel attention can learn more complex channel correlations, and is particularly suitable for complex acoustic environments with large reverberation time variations and irregular nonlinear distortion in access control scenarios.

[0042] Each step in the time dimension corresponds to a short frame (e.g., 20ms). Echoes are delayed relative to near-end speech and their intensity varies across frames. Temporal attention uses one-dimensional convolutions (kernel size 3) to capture local dependencies between adjacent time steps, and then generates frame-level weights through Softmax. This allows the model to highlight frames dominated by near-end speech and downplay frames containing only echoes or background noise. Compared to using global recurrent networks or fully connected networks to generate temporal weights, one-dimensional convolutions have fewer parameters, are more computationally lightweight, maintain temporal locality, are compatible with the forward characteristics of unidirectional LSTMs, and do not introduce information from future frames, thus ensuring the causality of real-time inference.

[0043] This scheme prioritizes channel attention over temporal attention. The rationale behind this order is that channel attention globally recalibrates the multidimensional structure of features without altering the temporal step order; while temporal attention further focuses on keyframes based on the enhanced channel features. If temporal attention were performed first, the temporal mask would be applied equally to all channels, potentially causing some discriminative channel information to be prematurely averaged. The sequential order of channel attention followed by temporal attention allows for temporal selection based on the enhanced, high-quality features, creating a progressive relationship of "first filtering feature types, then locating important moments." Experiments show that this order, compared to reverse or parallel fusion, improves dual-talk detection accuracy to over 97% while maintaining model lightweightness.

[0044] In summary, the combination of channel attention and temporal attention enables the model to adaptively process frequency simultaneously. The two-dimensional interference is more suitable for non-stationary and nonlinear interference characteristics in echo cancellation than single attention mechanisms or fixed weighting methods. The lightweight implementation (global pooling + MLP, small kernel one-dimensional convolution) keeps the additional computational overhead of dual attention within 10%, without affecting the real-time metric of end-to-end latency <20ms. The explicit constraint of the serial order ensures the determinism of feature enhancement, facilitating operator fusion optimization when deployed on embedded Linux devices, further reducing inference time.

[0045] In some embodiments, the fusion method of the mask fusion module includes: element-wise mask multiplication fusion and weighted adaptive mask fusion.

[0046] In the above technical solutions, this solution generates a frequency domain mask through dual-mask decoding. With temporal mask Building upon this foundation, two further methods for fusing the mask with the original near-end amplitude spectrum are defined: one is element-wise multiplication fusion, and the other is weighted adaptive fusion. Both methods are applicable to different acoustic scenarios and aim to jointly suppress echo / noise from both the frequency and time domains to extract clean near-end speech.

[0047] The mathematical expression for element-wise multiplication and fusion is: The essential characteristic of this method lies in treating the two masks as independent suppression factors, achieving "double penalty" through a product: a time-frequency point can only be determined as "non-echo" (i.e., non-echo) if it is simultaneously determined by both the frequency domain mask and the time domain mask. and When both masks are close to 0, the energy at that point is fully preserved; if either mask indicates an echo (mask value close to 1), the point is significantly attenuated. This product fusion produces a nonlinear synergistic effect—when both masks are at moderate values ​​(e.g., 0.5), the overall suppression factor is 0.25, equivalent to applying stronger attenuation to the uncertain region, thus effectively reducing residual echoes at blurred boundaries. Compared to linear fusion such as direct addition or taking the maximum value, the product form is better at suppressing echo components that are difficult to distinguish in a single dimension of the frequency or time domain, especially suitable for reverberation tails (which have obvious time-frequency diffusion characteristics) and broadband noise generated by nonlinear distortion.

[0048] Weighted adaptive fusion introduces adjustable weight coefficients. (0< <1), the fusion formula is: The key feature of this method is its ability to dynamically adjust the contribution ratio of the frequency domain mask and the time domain mask according to the actual acoustic environment. When the scene is dominated by long reverberation (such as an empty corridor), the frequency domain mask is more effective in suppressing steady-state echoes and can increase... Value; when there is sudden noise in the scene (such as the sound of a door closing or footsteps), the time-domain mask is more sensitive to impulse interference and can reduce the impact of sudden noise. The value makes the time-domain mask dominant. These weights can be modified online via a Linux configuration file without retraining or compiling the model, greatly improving the flexibility of field deployment.

[0049] In summary, product fusion is a fixed nonlinear combination that does not add extra parameters and is naturally suitable for scenarios requiring independent estimation of dual masks, compared to existing technologies that only use single masks or simple linear superposition. Weighted adaptive fusion introduces adjustable degrees of freedom, allowing the same model to adapt to different acoustic environments, thus solving the robustness problem of fixed fusion methods in variable access control scenarios. This mechanism does not change the internal parameters of the model, only adding two multiplication operations in the post-processing stage, with negligible overhead. Both methods share the same dual mask generation network, configured through configuration files. You can toggle it on or off.

[0050] In some embodiments, the training process of the DTLN-AEC model employs a combined loss function; an automatic stopping mechanism is used during training, which stops training when the model's score on the validation set does not improve for several consecutive rounds; or automatic stopping of training is achieved by monitoring the learning rate threshold.

[0051] In the above technical solution, in DTLN In AEC model training, a combined loss function (composed of multiple weighted sub-losses) is employed, along with an automatic stopping mechanism. This mechanism provides two termination conditions: first, stopping when the target score on the validation set fails to improve for several consecutive rounds; and second, stopping when the monitoring learning rate falls below a preset threshold. This strategy aims to ensure that the model's optimization objective considers multiple evaluation dimensions and adaptively controls training convergence, thereby improving efficiency and generalization.

[0052] The essential characteristic of a combined loss function lies in the weighted fusion of multiple sub-loss functions with different emphases, allowing the model to be guided by multiple constraints simultaneously during training. For echo cancellation tasks, a single loss function (such as only MSE or only SI) is insufficient. SDR (Sound Reduction) often struggles to balance echo suppression strength and near-end speech preservation. Combination loss coordinates the sub-losses through pre-defined weights, ensuring that optimizing one metric (e.g., numerical fitting of the amplitude spectrum) doesn't excessively sacrifice another (e.g., speech perception quality). Compared to a fixed single loss, combination loss allows for dynamic adjustment of the contribution weights of sub-losses during training, thus more flexibly adapting to the changing environmental conditions in access control scenarios. This strategy directly improves the model's overall score on the validation set (e.g., dual-talk detection accuracy, ERLE, etc.), enabling it to achieve higher echo cancellation quality during the inference phase.

[0053] The essential characteristic of the automatic stopping mechanism lies in providing two selectable termination conditions, forming redundant judgment logic. The first is based on the target score on the validation set (e.g., the SI score, which is strongly correlated with echo cancellation performance). The first method uses SDR or PESQ as the monitoring metric, stopping when the model fails to exceed its historical best value for a certain number of consecutive rounds (e.g., 10 rounds). This condition measures the model's generalization performance on the echo cancellation task, avoiding the "spurious convergence" trap where validation scores don't improve due to a continuous decrease in training loss. The second method uses the learning rate as the monitoring metric, stopping when the learning rate decays below a preset threshold using a scheduling strategy (e.g., ReduceLROnPlateau). An extremely low learning rate indicates that model parameter updates have stagnated, making further training unlikely to yield significant gains and prone to numerical instability.

[0054] In summary, the combined loss mechanism enables the model to maintain excellent performance on multiple key metrics simultaneously in the later stages of training, with accurate output enhancement amplitude spectra. The limited training data size in access control scenarios means that stopping too early can lead to underfitting, while stopping too late can result in the model memorizing specific echo patterns. A stopping condition based on validation set scores accurately locates the optimal generalization point using metrics strongly correlated with the task; a stopping condition based on the learning rate ensures the model exits promptly when it reaches its convergence limit, preventing overfitting. In actual training, both can be triggered selectively or sequentially, keeping the number of training rounds under 100, saving approximately 30%–50% of time compared to a fixed number of rounds. This mechanism operates fully automatically, and the "number of consecutive rounds" and "learning rate threshold" can be adjusted via configuration files to adapt to different dataset sizes and hardware training environments, significantly lowering the barrier to model tuning.

[0055] According to another aspect of the present invention, a Linux system echo cancellation system based on the DTLN-AEC model is provided. Based on the above method, the system includes: The audio acquisition and preprocessing module is used to resample the acquired audio data to a preset sampling rate, and extract the amplitude spectrum of the near-end microphone signal and the far-end reference signal through short-time Fourier transform and perform smoothing and noise reduction preprocessing. A lightweight DTLN-AEC model inference engine, deployed in Linux user space, includes the following models: A frequency domain feature encoder is used to perform nonlinear transformation on the amplitude spectrum to extract frequency domain features; The temporal feature encoder uses depthwise separable convolution to extract temporal features; The feature fusion module is used to stitch together and fuse the frequency domain features and time domain features of the near end and the far end; Unidirectional LSTM is used for temporal modeling of the fused features; Channel attention mechanism and temporal attention mechanism are applied sequentially to the output features of LSTM, respectively performing adaptive weighted enhancement on the channel dimension and the temporal dimension; A dual-mask decoder is used to generate frequency domain masks and time domain masks; The mask fusion module is used to fuse the frequency domain mask and the time domain mask with the original near-end amplitude spectrum to obtain the enhanced amplitude spectrum; The model in question is a lightweight model that has undergone quantization and lightweight deployment processing. The frame synchronization module is used to add a timestamp to each audio frame under the Linux system. It achieves synchronization between acquisition and processing through a circular buffer. The circular buffer has a preset frame number threshold. When the number of cached frames reaches the threshold, model inference is triggered. The echo cancellation inference module is used to input the synchronized near-end and far-end amplitude spectra into the deployed DTLN-AEC model. After model processing, the enhanced amplitude spectrum is output, and the time-domain signal is reconstructed through inverse short-time Fourier transform to obtain clean near-end speech.

[0056] In order to better utilize the above methods, this application proposes a Linux system echo cancellation system based on the DTLN-AEC model. Each module corresponds to a step of the above method, and its specific principle has been described above and will not be repeated here.

[0057] According to another aspect of the present invention, an embedded voice processing intelligent device is provided, comprising: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0058] In the above technical solution, to better operate and process the method, the method is stored in memory, and the processor executes the stored method. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here.

[0059] According to another aspect of the present invention, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method.

[0060] In the above technical solution, to better operate and use the method, the method is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart illustrating an embodiment of an echo cancellation method for Linux systems based on the DTLN-AEC model according to the present invention. Figure 2 This is a DTLN-AEC model structure diagram of an embodiment of the Linux system echo cancellation method based on the DTLN-AEC model of the present invention; Figure 3 This is a frequency domain feature extractor structure diagram of an embodiment of the Linux system echo cancellation method based on the DTLN-AEC model of the present invention; Figure 4 This is a temporal feature extractor structure diagram of an embodiment of the Linux system echo cancellation method based on the DTLN-AEC model of the present invention; Figure 5 This is a timing modeling structure diagram of an embodiment of the Linux system echo cancellation method based on the DTLN-AEC model of the present invention; Figure 6 This is a structural diagram of the channel attention mechanism of an embodiment of the Linux system echo cancellation method based on the DTLN-AEC model of the present invention; Figure 7 This is a structural diagram of the time attention mechanism of an embodiment of the Linux system echo cancellation method based on the DTLN-AEC model of the present invention; Figure 8 This is a mask decoder structure diagram of an embodiment of the Linux system echo cancellation method based on the DTLN-AEC model of the present invention; Figure 9 This is a flowchart of the DTLN-AEC model training process, which is an embodiment of the Linux system echo cancellation method based on the DTLN-AEC model according to the present invention. Figure 10 This is a schematic diagram of an embodiment of an echo cancellation device for a Linux system based on the DTLN-AEC model according to the present invention. Detailed Implementation

[0063] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Example 1 Please see Figure 1 A Linux system echo cancellation method based on the DTLN-AEC model, the method comprising: S1. Resample the collected audio data to a preset sampling rate, and extract the amplitude spectrum of the near-end microphone signal and the far-end reference signal through short-time Fourier transform and perform smoothing and noise reduction preprocessing. In this embodiment, the preset sampling rate is 8kHz to 16kHz.

[0065] For example, in an access control intercom scenario, the sampling rate of the audio data is modified by resampling the 16kHz audio data to 8kHz. Since the main frequency range of human speech is 300Hz-3400Hz, the 8kHz sampling rate can fully cover this range (according to the Nyquist theorem, 8kHz can reproduce all sounds below 4kHz). Calculations show that using audio data with an 8kHz sampling rate reduces the number of parameters in the frequency domain encoder, time domain encoder, and mask decoder by 20480, 1440, and 20480, respectively. Furthermore, the window length is increased from 32ms to 40ms, and the window shift is increased from 8ms to 20ms. These two steps further reduce the computational complexity and model size. After data acquisition, further smoothing and denoising preprocessing is performed. S2. Construct the DTLN-AEC model, which includes: A frequency domain feature encoder is used to perform nonlinear transformation on the amplitude spectrum to extract frequency domain features; The temporal feature encoder uses depthwise separable convolution to extract temporal features; The feature fusion module is used to stitch together and fuse the frequency domain features and time domain features of the near end and the far end; Unidirectional LSTM is used for temporal modeling of the fused features; Channel attention mechanism and temporal attention mechanism are applied sequentially to the output features of LSTM, respectively performing adaptive weighted enhancement on the channel dimension and the temporal dimension; A dual-mask decoder is used to generate frequency domain masks and time domain masks; The mask fusion module is used to fuse the frequency domain mask and the time domain mask with the original near-end amplitude spectrum to obtain the enhanced amplitude spectrum; The DTLN-AEC model is quantized and then deployed in a lightweight manner. In this embodiment, the frequency domain feature encoder consists of a three-layer fully connected network. Each layer sequentially performs linear transformation, layer normalization, GELU activation, and Dropout operations. The first layer compresses the high-dimensional input features to a low dimension, the second layer expands the low-dimensional features by a factor of 2, and the third layer re-compresses the expanded features to a low dimension.

[0066] In this embodiment, the temporal feature encoder adopts a structure that combines depthwise separable convolution with residual connections. The depthwise separable convolution includes depthwise convolution and 1×1 point convolution. The residual connections use 1×1 convolution for dimension matching when the input and output dimensions are inconsistent.

[0067] In this embodiment, the channel attention mechanism generates a channel attention mask through global average pooling, multilayer perceptron, and sigmoid function; the temporal attention mechanism generates a temporal attention mask through one-dimensional convolution and Softmax function; the channel attention mechanism and the temporal attention mechanism are executed sequentially, with channel attention being executed first and then temporal attention.

[0068] In this embodiment, the fusion methods of the mask fusion module include: element-wise mask multiplication fusion and weighted adaptive mask fusion.

[0069] In this embodiment, the training process of the DTLN-AEC model adopts a combined loss function; an automatic stopping mechanism is used during training, which stops training when the model's score on the validation set does not improve for several consecutive rounds; or automatic stopping of training is achieved by monitoring the learning rate threshold.

[0070] For example, the present invention makes the following three improvements to the DTLN-AEC model, as detailed below: 1. DTLN-AEC Model Inference Enhancement: A "reinforcement-generation-processing" inference architecture is constructed. First, the audio amplitude spectrum is extracted through STFT transform, and the feature representation is enhanced by combining time-domain / frequency-domain dual convolutional block encoding and channel / time dual attention mechanisms. Then, a dual-mask decoder generates frequency-domain / time-domain masks, and the suppression strength is dynamically adjusted through an adaptive fusion strategy. Finally, an inference script is designed for the 8kHz PCM format, supporting automatic matching and processing of single files / batch files, improving inference accuracy and adaptability. Details are as follows: The batch file automatic matching processing scheme is as follows: 1. Load the model and model weights; 2. Load the near-end audio (microphone audio, including echo) and far-end reference signal, and perform one-to-one correspondence verification between the near-end and far-end audio; 3. Sequentially read and preprocess each near-end audio and its corresponding far-end audio. Preprocessing includes verifying the audio sampling rate and resampling to 8kHz, verifying the length of the near-end audio and the length of the far-end audio, truncating the longer audio based on the shorter audio if the lengths are inconsistent, performing STFT transformation and separating the amplitude spectrum and phase spectrum, and normalizing and smoothing the amplitude spectrum and phase spectrum respectively; 4. Input the amplitude spectrum features into the DTLN-AEC model and perform model inference, outputting the enhanced amplitude spectrum; 5. After inverse normalization of the amplitude spectrum, recombine the amplitude spectrum and phase spectrum; 6. Perform inverse STFT processing and output the processed audio file.

[0071] The core processing flow of the DTLN-AEC model is as follows: dual-domain feature extraction, feature fusion, temporal modeling, dual-attention mechanism enhancement, dual-mask decoding, dual-mask fusion, and clean near-end speech extraction. The model takes the amplitude spectrum of the near-end / far-end audio signals as input, and through multi-dimensional feature extraction and mask optimization, finally outputs an enhanced near-end speech amplitude spectrum that eliminates echoes and noise. The original audio phase information is preserved throughout the process to ensure the naturalness of the speech reconstruction. The model's enhanced inference structure architecture is shown in the diagram below. Figure 2 As shown.

[0072] (1) Frequency domain feature encoder Please see Figure 3 Frequency domain feature extraction is achieved through a frequency domain encoder composed of a three-layer fully connected network. This encoder performs a nonlinear "compression-expansion-compression" transformation on the amplitude spectrum of the input near-end / far-end audio signals to extract high-dimensional frequency domain features. This effectively preserves the distinguishing features between echoes, noise, and valid speech in the spectrum, while controlling model complexity. A set of identical frequency domain encoders is configured for each of the near-end / far-end signals. The core transformation formula is expanded hierarchically: First layer (feature compression) The high-dimensional frequency domain input features are compressed into a low-dimensional hidden layer to complete the initial feature extraction and standardization. The formula is as follows:

[0073] Second layer (feature expansion) Extending low-dimensional features to higher dimensions enhances the model's frequency domain feature representation ability and captures more refined spectral differences. The formula is as follows:

[0074] Third layer (feature recompression) The high-dimensional extended features are compressed back to the target hidden layer dimension to reduce subsequent computational complexity, while retaining the core frequency domain features. The formula is as follows:

[0075] in, For the input frequency domain amplitude spectrum features (near end) / Remote ), dimension ; Here are the weight matrices for each layer of the fully connected network. For the bias term, where The output dimension is 2×hiddensize (the dimension of the hidden layer), which enables feature expansion; This is a layer normalization operation that eliminates amplitude differences in spectral features and improves the stability of model training and inference. It is a Gaussian error linear activation function to achieve nonlinear feature transformation and adapt to the non-uniform distribution characteristics of the audio spectrum; This is a regularization operation (dropout rate of 0.2) to prevent overfitting and improve generalization ability; The final output is a high-dimensional frequency domain feature with dimension . .

[0076] (2) Temporal feature encoder Temporal feature extraction is achieved through a temporal encoder composed of depthwise separable convolutions and residual connections. This encoder extracts local features from the input near-end and far-end audio temporal signals, capturing the temporal dynamics of the audio (such as echo delay characteristics and noise temporal impulse characteristics). A set of identical temporal encoders is configured for each of the near-end and far-end signals, as shown in the following diagram. Figure 4 As shown. The core transformation formula is:

[0077] Residual connection dimension matching rules: If the input and output feature dimensions are the same, the identity mapping can be used directly. If the dimensions are inconsistent, dimension matching is achieved through 1×1 convolution: .

[0078] in, The input is the original time-domain audio signal with dimension . ; For depthwise separable convolution, each channel is convolved independently to capture local temporal features of a single channel. The kernel size is 3×1 and the stride is 1. It uses 1×1 point convolution to achieve information fusion between channels and integrate multi-channel temporal features; , This represents the bias term for depthwise convolution and pointwise convolution; Residual connections are used to alleviate the gradient vanishing problem in deep networks and ensure deep extraction of temporal features; The final output is the temporal local feature, with dimension . .

[0079] (3) Feature fusion module Extracted high-dimensional frequency domain features and temporal local features The first and last parts are spliced ​​together and fused, integrating features from both the near and far ends. This maximizes the preservation of original feature information from different domains and signals, avoids information loss during feature fusion, and provides rich feature input for subsequent time-series modeling. The fusion formula is as follows:

[0080] in, Frequency domain characteristics of far-end and near-end signals; The time-domain characteristics of far-end and near-end signals; This is a concatenation operation along the feature dimension; the fused feature dimension is... ; This provides complete feature input for dual-domain feature fusion and enables temporal modeling. Furthermore, fusion methods can also include weighted summation fusion, gated fusion, and multilayer perceptron fusion.

[0081] (4) Temporal modeling (unidirectional LSTM) A unidirectional LSTM with layer normalization is used to fuse features. Temporal modeling is performed to effectively capture the temporal dependencies of audio signals (such as the time delay characteristics of echoes and the continuous temporal features of speech). Meanwhile, the unidirectional LSTM design ensures the model's real-time inference capability and avoids the latency issues of bidirectional LSTMs. The corresponding structure is as follows: Figure 5 As shown. The core transformation formula is:

[0082]

[0083] in, It is a unidirectional long short-term memory network with a hidden layer dimension of 2×hiddensize, retaining only forward temporal features; The hidden state and cell state of LSTM provide temporal context for subsequent attention mechanisms; The temporal modeling features output by the LSTM have a dimension of ; This prepares the standardized temporal features for enhanced attention mechanisms.

[0084] (5) Channel attention mechanism and time attention mechanism Temporal modeling features using channel attention and temporal attention mechanisms Adaptive weighted enhancement is performed, allowing the model to automatically focus on feature channels and time steps valuable for echo / noise suppression, suppressing ineffective features, and improving the model's ability to extract effective near-end speech. Two attention mechanisms are executed sequentially: channel enhancement first, followed by temporal enhancement. The structure diagrams of the two attention mechanisms are shown below. Figure 6 , Figure 7 As shown. The core formulas are as follows: ① Channel attention mechanism Adaptive weighting is applied to the feature channel dimensions to highlight feature channels containing effective speech and suppress invalid channels dominated by noise / echo, as shown in the formula: (Dimensional compression to) )

[0085] (Generate channel attention mask, dimension) )

[0086] in, To achieve adaptive global average pooling, the temporal features of each channel are averaged to extract the global features of the channels. , The weight matrix of the multilayer perceptron (MLP) is used to realize the dimensionality reduction-up transformation of the channel features, with a reduction ratio of 16. This is the channel attention mask, with a value range of [0,1]. The closer the mask value is to 1, the more important the corresponding channel is. To achieve channel-weighted enhancement of features through element-wise multiplication; The temporal features are those resulting from channel attention enhancement.

[0087] ② Time attention mechanism Adaptive weighting is applied to the feature time step dimension to highlight time steps containing effective near-end speech and suppress invalid time steps where echoes / noise occur. The formula is as follows: (1×1 convolution captures time-step dependency, kernel size 3) (Generate temporal attention mask, dimension) )

[0088] in, This is a one-dimensional convolution that captures local dependencies between time steps; This is a temporal attention mask, with values ​​ranging from [0,1]. The closer the mask value is to 1, the more important the corresponding time step is. The core features enhanced by the channel-temporal dual attention mechanism provide the optimal feature input for mask decoding.

[0089] It is worth noting that the attention mechanism can also replace the original channel-temporal dual attention mechanism with a combination of lightweight spatial attention and channel attention: remove the 1D convolutional layer of the temporal attention mechanism and replace it with lightweight spatial attention using global average pooling and global max pooling, reducing the computational cost of the attention mechanism while retaining the core weighting logic of channel attention. Under the premise of ensuring feature enhancement effect, the inference time of the attention mechanism is reduced by 40% without affecting the robustness of echo cancellation.

[0090] (6) Dual-mask decoder Enhanced features are achieved through frequency-domain mask decoders and time-domain mask decoders with identical structures. Decode the code and generate frequency domain masks. and temporal mask The mask values ​​are all in the range [0,1], used for subsequent two-dimensional suppression of the original near-end signal. Both decoders are three-layer fully connected networks, and their structure diagrams are as follows. Figure 8 As shown. The core decoding formula is the same (taking frequency domain mask decoding as an example):

[0091]

[0092]

[0093]

[0094] in, , , , This is the weight matrix of the mask decoder. , , , For bias terms; This is a frequency domain mask with the same dimension as the frequency points of the original near-end amplitude spectrum. ; It is a time-domain mask with the same dimension as the time dimension of the original near-end amplitude spectrum. Mask value rules: The closer the value is to 1, the more likely the corresponding position is echo / noise, which needs to be suppressed; the closer the value is to 0, the more likely the corresponding position is valid near-end speech, which needs to be preserved.

[0095] (7) Mask fusion module and clean near-end speech extraction The frequency domain mask obtained from decoding and temporal mask Amplitude spectrum compared to the original near-end microphone signal Element-wise multiplication and fusion are performed to suppress echo / noise in both the frequency and time domains, ultimately extracting the enhanced, clean near-end speech amplitude spectrum. The core fusion formula is:

[0096] It is important to note that the mask fusion method can also replace the element-by-element multiplication fusion of the original frequency domain and time domain masks with a mask weighted adaptive fusion scheme: introducing a fusion weight coefficient α (0 < α < 1, which can be modified in the Linux configuration file, with a default of 0.5), the fusion formula is as follows: The mask weights can be dynamically adjusted according to the actual acoustic environment (such as increasing α in high reverberation scenarios and decreasing α in high noise scenarios), which improves the model's adaptability to different scenarios without increasing additional computing power.

[0097] Signal restoration rule: Preserve the phase information of the original near-end audio signal. The enhanced amplitude spectrum Combined with the original phase, the complex spectrum is reconstructed and then converted into a time-domain signal through inverse short-time Fourier transform (ISTFT) to obtain the final clean near-end speech:

[0098]

[0099] in, The amplitude spectrum of the original near-end microphone signal is used as the model input. This is a mask inversion process to preserve valid speech and suppress echo / noise. The amplitude spectrum of the enhanced clean near-end speech; The amplitude and phase information are fused to form the reconstructed complex spectrum; The final output is a clean, time-domain near-end speech signal, free from echoes and complex noise interference. The inverse short-time Fourier transform is used, with a frame length of 20ms, a frame shift of 10ms, and a Hanning window, similar to the preprocessing... Parameter matching. It should be noted that the time-frequency transformation method can also replace the original STFT / ISTFT short-time Fourier transform with a lightweight time-frequency transformation scheme based on 512-point FFT: simplify the windowing and frame shifting logic of STFT, adopt a lightweight configuration of fixed Hanning window + 50% frame shift, fix the number of FFT points to 512, reduce the transformation computing power overhead, and at the same time retain the core information of frequency domain features. Under the premise of accuracy loss <1%, the time-frequency transformation time is reduced by 20%, which is compatible with the low computing power architecture of ARM Cortex-A7.

[0100] 2. Lightweight and Real-Time Adaptation Based on Linux System: Focusing on a simplified model structure and real-time processing adaptation, a lightweight 128-dimensional hidden layer design is adopted, coupled with a unidirectional LayerNormLSTM for real-time sequence processing. A dual-path architecture simplifies the frequency / time domain feature extraction process, combined with batch parallel data loading. Optimizations are also performed on the 8kHz PCM format, including the aforementioned smoothing and denoising, normalization, and data augmentation. The entire process, from model structure to data processing, ensures lightweight design and low latency. Details are as follows: (1) Streamline network architecture 1) Reduce the number of hidden layers and units in temporal modeling: In the unidirectional LSTM temporal modeling module, while ensuring model stability, the model can be simplified by pruning the number of units in the hidden layers or reducing the number of hidden layers, thereby reducing the computational complexity of the model and improving the real-time performance of access control calls. It should be noted that the model pruning method can also replace the original pruning method of adjusting the hidden layer dimension (hidden_size) with a scheme of on-demand pruning of network layers + operator simplification: While retaining the core dual-domain feature extraction and dual attention mechanism of the model, the number of layers in the frequency domain encoder, mask decoder, and non-core fully connected layers of temporal modeling can be pruned (e.g., from 3 layers to 2 layers) or the number of units can be pruned. At the same time, some GELU activation functions can be replaced with the lighter ReLU activation function (used only in non-core feature layers) to achieve a step-wise reduction in the number of parameters, adapt to ARM Linux devices with different computing power, and ensure algorithm consistency.

[0101] 2) Depthwise separable convolution replaces ordinary convolution: The core convolutional layer of the temporal feature extractor adopts depthwise separable convolution (Depthwise + Pointwise), which splits the "channel fusion + spatial convolution" of ordinary convolution and reduces the computational cost to 1 / n of the original (n is the number of input channels), greatly reducing the computational cost of temporal feature extraction; combined with the lightweight dimension matching method of residual connection (identity mapping / 1×1 convolution), it avoids gradient vanishing in deep networks and does not increase the additional computational burden.

[0102] 3) Frequency domain encoder optimization: The frequency domain encoder adopts a lightweight "compression-expansion-compression" structure. Under the premise of extracting sufficient features, the core features of the frequency domain can be preserved while minimizing the number of parameters by reducing the number of hidden layers in the intermediate expansion layer.

[0103] (2) Model quantization The DTLN-AEC model supports INT8 quantization pruning. After training, the model undergoes quantization calibration: 32-bit floating-point (FP32) weights / activations are converted to 8-bit integers (INT8), directly compressing the model size to 1 / 4 of the original. At the same time, floating-point operations during inference are converted to integer operations, significantly reducing the computational burden on embedded CPUs and improving inference speed. After quantization, accuracy loss is minimized through calibration using access control scenario data, with only a slight loss in echo cancellation effect, ensuring the core metric of ERLE≥45dB. It's important to note that the model quantization method can also replace the original INT8 integer quantization with a lightweight solution combining FP16 half-precision quantization and sparsity: For Linux devices with ARM Cortex-A53 / A55 architecture and a hardware floating-point unit (FPU), the model accuracy advantage of FP16 half-precision quantization is retained, while structured sparsity (30% channel-level sparsity) is applied to the model weights, skipping the multiplication and addition operations of zero-value weights. This achieves a 50% reduction in model size and a 30% increase in inference speed, ultimately achieving resource consumption metrics consistent with INT8 quantization (CPU < 30%, memory < 4MB). Furthermore, (3) Deployment process optimization (adapting the inference framework to embedded devices) The DTLN-AEC model supports exporting in TFLite (TensorFlow Lite) format, offering lightweight optimizations for deployment on embedded devices. TFLite is a lightweight inference framework designed by Google for embedded / mobile devices, specifically optimized for models. Combined with model quantization and lightweight architecture, it allows models to be directly deployed to various embedded speech processing smart devices without requiring additional framework adaptation development. Specific optimizations include: 1) Perform lightweight fusion of model operators (e.g., merge "Conv+BN+Activation" into a single operator) to reduce the number of operator calls during inference and improve execution efficiency; 2) Employ a memory reuse mechanism to optimize the memory usage of model inference and avoid inference lag caused by insufficient memory in embedded devices; It is important to note that the deployment framework can also replace the original TFLite Micro inference framework with the ONNX Runtime for ARM Embedded Simplified Version: it retains the core features of operator fusion and memory reuse, optimizes for the ARM Cortex-A7 / A53 / A55 architecture with the NEON instruction set, supports efficient inference of INT8 / FP16 quantized models, and is compatible with more deep learning operators. It can achieve the same inference latency and resource consumption as TFLite Micro without modifying the core structure of the model.

[0104] 3. DTLN-AEC Model Training Process: First, batch parallel loading and standardization preprocessing of three types of audio data are achieved through custom Dataset and DataLoader. Then, the Adam optimizer, adaptive learning rate scheduling strategy, and automatic stopping mechanism are configured, and a combined loss function is used to balance training effect and convergence speed. During training, SI-SDR and MSE core metrics are monitored in real time, and the optimal model is automatically saved based on the validation loss. It is important to note that the training data preprocessing part can replace the original batch parallel loading + standardization preprocessing with an online data augmentation + lightweight preprocessing scheme: lightweight data augmentation (such as random noise addition and random reverberation) is performed in real time during training, replacing offline large-scale dataset augmentation. At the same time, the standardization preprocessing is simplified to mean normalization, reducing the computational cost of data preprocessing, adapting to scenarios with limited training hardware resources, and the model's generalization ability is not affected.

[0105] A dedicated adaptation and optimization algorithm was designed specifically for the characteristics of the Linux system to solve problems such as data synchronization, resource scheduling, and driver compatibility. The specific model training flowchart is as follows. Figure 9 As shown. Specifically: 1) Loss Function Design: The adopted combined loss is a weighted combined loss function carefully designed for speech echo cancellation tasks. It integrates SI-SDR loss (60%) and MSE loss (40%), aiming to simultaneously optimize the subjective perceived quality of speech and the numerical fitting accuracy. SI-SDR loss focuses on evaluating the perceived clarity and naturalness of speech, is insensitive to signal scaling changes, and better reflects the characteristics of human hearing; MSE loss calculates the mean squared error between the prediction and the target value, ensuring accurate numerical fitting of the model output. With weights of 0.6 and 0.4, this combined loss prioritizes enhancing the auditory quality of speech while avoiding excessive numerical bias, ultimately guiding the model to learn echo cancellation results that are both natural and intelligible and accurately match the target signal. It's important to note that the loss function can also be changed from the original SI-SDR+MSE combined loss function to a PESQ+L1 loss function combination: using PESQ loss (70%) to ensure subjective speech quality and L1 loss (30%) to ensure numerical fitting accuracy. Alternatively, the combined loss function (SI-SRD+MES) can be replaced with other loss combinations, such as (SI-SRD+STOI, speech quality + speech intelligibility), or the combination method can be changed. Using speech evaluation standards adapted to access control intercom scenarios, the trained model achieves better speech reproduction accuracy, and the convergence speed is basically consistent with the original loss function.

[0106] 2) Training and Validation Process: Before model training, the DTLN-AEC model is initialized (configuring a frequency domain encoder, temporal convolutional blocks, a layered normalized LSTM, and a dual attention mechanism). The Adam optimizer, learning rate scheduler (ReduceLROnPlateau), and combined loss of SI-SDR and MSE are set. Multiple rounds of training and validation are then performed. In each training round, the model calculates the enhanced speech amplitude spectrum through forward propagation and calculates the combined loss score obtained in that round. The combined loss guides the model to continuously optimize its parameters and dynamically adjust the learning rate. In each validation round, the model's predictive performance on the validation set is evaluated, i.e., the SI-SDR score on the validation set is calculated. When the SI-SDR value in a round is greater than the historical maximum, the maximum SI-SDR score and the corresponding optimal model are updated. To prevent overfitting, an automatic stopping mechanism is added during the multiple training and validation rounds. If the model's SI-SDR score on the validation set does not improve for ten consecutive rounds, model training is stopped, and the optimal model is output. The entire process achieves efficient model training and performance monitoring. It's important to note that the optimizer and learning rate scheduling can be improved by replacing the original Adam optimizer + ReduceLROnPlateau learning rate scheduling with the AdamW optimizer + cosine annealing learning rate scheduling. AdamW's weight decay achieves more efficient regularization, preventing overfitting; cosine annealing learning rate scheduling smoothly decreases the learning rate, improving generalization ability. After training, the model's ERLE (Earning Experience Scale) remains ≥45dB in complex acoustic environments. Furthermore, an automatic stopping mechanism can be implemented by monitoring a learning rate threshold. During training, the learning rate value obtained in each round is compared with a set threshold; training terminates when the learning rate falls below the threshold. Similarly, an automatic stopping mechanism can be implemented by monitoring a validation set loss threshold; training terminates when the loss on the validation set falls below the set threshold.

[0107] S3. Under the Linux system, add a timestamp to each audio frame and realize the synchronization of acquisition and processing through a circular buffer. The circular buffer has a preset frame number threshold. When the number of cached frames reaches the threshold, model inference is triggered. In this embodiment, the preset frame rate threshold is 2 to 5 frames.

[0108] For example, the audio data frame synchronization algorithm adopts a dual synchronization mechanism of timestamp and buffer threshold. A millisecond-level timestamp of the Linux system is added to each audio frame. Synchronization of acquisition and processing is achieved through a circular buffer. The buffer threshold is set to 3 frames. When the amount of data in the buffer is ≥ 3 frames, model inference is triggered to avoid frame loss or duplicate processing. S4. Input the synchronized near-end and far-end amplitude spectra into the deployed DTLN-AEC model. After model processing, the enhanced amplitude spectrum is output. Then, the time-domain signal is reconstructed through inverse short-time Fourier transform to obtain clean near-end speech.

[0109] The example here refers to the relevant description of "(7) Mask fusion module and clean near-end speech extraction" in step S2 above, and will not be repeated here.

[0110] In this embodiment, the algorithm execution flow is as follows: System initialization: Loading lightweight model weights, real-time data sequence processing, and adapting for Linux deployment; Audio acquisition: Synchronous near / far audio acquisition, timestamp addition, and data validity verification; Data preprocessing: audio frame segmentation and windowing, STFT time-frequency transformation, amplitude spectrum phase separation; Dual-domain feature extraction: frequency domain encoder compression expansion, and temporal-domain depthwise separable convolution extraction; Feature fusion modeling: far / near features are concatenated, and unidirectional LayerNormLSTM is used to complete temporal feature modeling; Attention enhancement: Channel / temporal attention weighting, adaptive enhancement of audio features; Dual-mask decoding: Generate frequency domain / time domain masks and perform 0-1 normalization to accurately match echo features; Mask fusion enhancement: Double mask weighted operation, enhanced near-end amplitude spectrum generation; Audio post-processing: complex spectrum reconstruction, inverse ISTFT transform, frame splicing + gain adjustment to restore speech; Output control: Outputs clean 8kHz PCM near-end audio data; In this embodiment, the core parameters of the present invention are divided into DTLN-AEC model parameters and Linux system configuration parameters. All parameters can be modified in the configuration file of the Linux system to adapt to different hardware platforms and application scenarios. The specific key parameter values ​​are as follows:

[0111] In this embodiment, addressing the aforementioned shortcomings of existing WebRTC AEC3 algorithm ports in Linux system echo cancellation applications, this invention solves the problem of poor environmental adaptability by deeply adapting and optimizing the DTLN-AEC deep learning model to the Linux system, achieving model lightweighting, and thereby improving the quality, efficiency, and real-time performance of echo cancellation. Specific quantitative technical objectives are as follows: (1) Model lightweighting: Adjust the model's input parameters (sampling rate, window length, frame shift, number of feature extraction layers) and perform lightweighting processes such as INT8 quantization, parameter pruning, and replacing ordinary convolution with depthwise separable convolution to reduce the computational complexity of the model while meeting the requirements, thereby achieving model lightweighting and reducing memory usage. The model's deployment package size under the ARM architecture is <15MB. The lightweight model can be adapted to Linux embedded hardware resources without additional pruning. (2) Improve the quality of echo cancellation: The accuracy of dual-talk detection is improved to over 97%, the robustness to reverberation and non-steady-state noise is significantly enhanced, it is adapted to a reverberation time of 0.2-2.0s (T60), the echo suppression gain (ERLE) is ≥45dB, and the nonlinear echo suppression capability reaches -38 to -42dB; the SI-SDR index is improved by ≥15dB during training.

[0112] (3) System real-time performance: The end-to-end processing latency under the Linux system in the lightweight mode is <20ms (average 15ms), CPU usage on the ARM Cortex-A7 platform is <30% (average 25%), and memory usage is <4MB; (4) Model training efficiency: An 8k sampling rate suitable for access control scenarios is used to reduce the computational complexity of the model. At the same time, an automatic stopping mechanism is added during the training process. This allows the model to perceive the training effect in advance and end the training, preventing overfitting and thus improving training efficiency. In actual training, the number of training convergence rounds is ≤100.

[0113] The echo cancellation method for Linux systems based on the DTLN-AEC model of this invention achieves significant improvements in echo cancellation effect, real-time performance, system adaptability, and hardware compatibility compared to the existing WebRTCAEC3 algorithm port. It combines qualitative advantages with quantitative effects, as detailed below: (1) Comparison of quantitative effects:

[0114] (2) Qualitative advantages 1) Data augmentation to improve model generalization: To increase the diversity of audio data and improve the model's generalization ability, data augmentation processing is performed on the input audio data. The augmented data covers scenarios such as background conversations, footsteps, door closing sounds, and key presses. This enables the model to adapt to echo cancellation in various access control intercom systems, solving the problem of poor model adaptability to different environments.

[0115] 2) Lightweight Model with Strong Adaptability and Linux Environment Deployment Compatibility: For Linux environments, the model uses 8kHz sampling rate audio data and incorporates lightweight processing techniques such as window size adjustment, frame shifting, reduction of feature extraction layers, INT8 quantization, parameter pruning, and replacing ordinary convolutions with depthwise separable convolutions. This minimizes the model's computational complexity while meeting requirements, achieving lightweight design and reduced memory usage. The lightweight model is fully compatible with resource-constrained Linux systems. Furthermore, the DTLN-AEC model supports the TFLite format, making it easy to deploy on Linux hardware systems, and exhibits excellent environmental adaptability and operational compatibility.

[0116] 3) Model improvements and optimizations to enhance echo cancellation performance: The improved combined loss (SI-SDR+MES) not only focuses on evaluating the perceived clarity and naturalness of speech and is insensitive to signal scaling changes, but also better matches the characteristics of human hearing. Furthermore, by calculating the mean squared error between the prediction and the target value, it ensures accurate numerical fitting of the model output, making it more suitable for guiding the training process of the model in access control intercom scenarios. At the same time, it integrates frequency domain STFT transform + encoder and temporal depth-separable convolution dual-path feature extraction, coupled with channel / temporal dual attention mechanism to adaptively enhance features, making the capture and suppression of echo features more accurate and the speech restoration higher, thus improving the echo cancellation rate of the model to over 95% during intercom. 4) Excellent real-time performance, suitable for voice communication scenarios: The unidirectional LSTM architecture is adapted for real-time sequence processing, and the entire link is specially optimized for 8kHz PCM audio. Batch parallel data processing further reduces inference latency and meets the real-time interactive needs of voice communication under Linux system. 5) Closed-loop processing flow with high engineering feasibility: From feature extraction and temporal modeling to post-processing reconstruction (ISTFT inverse transform, frame splicing, etc.), a complete inference chain is formed, with supporting log recording and status reporting functions, which meets the practical requirements of Linux system engineering deployment. 6) Efficient training process and guaranteed model performance: Based on the efficient training process of combined loss function + Adam optimizer + adaptive learning rate, the SI-SDR core indicators are monitored in real time during the training process, the pre-trained model has strong generalization ability and no need for complex secondary tuning after deployment.

[0117] Based on the above embodiments, the present invention has the following advantages: 1. Perform data augmentation on the input audio data to improve the model's generalization ability, thereby adapting to different access control intercom scenarios; 2. By reducing the audio data sampling rate input to the DTLN-AEC training model, the number of parameters in the frequency domain encoder, temporal domain encoder, and mask decoder was reduced by 20480, 1440, and 20480 respectively, thereby reducing the model's computational complexity and size (total model parameters were reduced by 10%-15%). Furthermore, by adjusting the number of hidden layers and the input window length and window shift parameters of the DTLN-AEC model, and by implementing lightweight processing such as INT8 quantization, parameter pruning, and replacing ordinary convolution with depthwise separable convolution, the model size was further reduced, making the model suitable for the hardware resources of Linux embedded systems. 3. The DTLN-AEC model achieves efficient suppression of nonlinear echoes and complex noise by using time-domain + frequency-domain dual feature extraction, dual-mask fusion, and channel / time attention mechanisms, and extracts clean near-end speech from mixed audio signals.

[0118] 4. Use SI-SDR loss, Adam optimizer, adaptive learning rate, and automatic stopping mechanism to monitor the training process of DTLN-AEC model and achieve real-time model tuning and training.

[0119] Example 2 Please see Figure 10 An echo cancellation system for a Linux system based on the DTLN-AEC model, based on the method described in one of the embodiments, the system comprising: The audio acquisition and preprocessing module is used to resample the acquired audio data to a preset sampling rate and extract the amplitude spectrum smoothing and noise reduction preprocessing of the near-end microphone signal and the far-end reference signal through short-time Fourier transform. A lightweight DTLN-AEC model inference engine, deployed in Linux user space, includes the following models: A frequency domain feature encoder is used to perform nonlinear transformation on the amplitude spectrum to extract frequency domain features; The temporal feature encoder uses depthwise separable convolution to extract temporal features; The feature fusion module is used to stitch together and fuse the frequency domain features and time domain features of the near end and the far end; Unidirectional LSTM is used for temporal modeling of the fused features; Channel attention mechanism and temporal attention mechanism are applied sequentially to the output features of LSTM, respectively performing adaptive weighted enhancement on the channel dimension and the temporal dimension; A dual-mask decoder is used to generate frequency domain masks and time domain masks; The mask fusion module is used to fuse the frequency domain mask and the time domain mask with the original near-end amplitude spectrum to obtain the enhanced amplitude spectrum; The model in question is a lightweight model that has undergone quantization and lightweight deployment processing. The frame synchronization module is used to add a timestamp to each audio frame under the Linux system. It achieves synchronization between acquisition and processing through a circular buffer. The circular buffer has a preset frame number threshold. When the number of cached frames reaches the threshold, model inference is triggered. The echo cancellation inference module is used to input the synchronized near-end and far-end amplitude spectra into the deployed DTLN-AEC model. After model processing, the enhanced amplitude spectrum is output, and the time-domain signal is reconstructed through inverse short-time Fourier transform to obtain clean near-end speech.

[0120] In this embodiment, in order to better utilize the method described in one of the embodiments, this application proposes a Linux system echo cancellation system based on the DTLN-AEC model. Each module corresponds to each step of the above method, and its specific principle has been described above and will not be repeated here.

[0121] Example 3 An embedded voice processing smart access control device includes: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described in one of the embodiments.

[0122] The memory stores a lightweight DTLN-AEC inference program, which executes the echo cancellation method described in one of the embodiments. The inference program incorporates a frequency domain / time domain dual-feature encoder, a unidirectional LSTM timing module, a serial dual-channel and time attention module, and a dual-mask decoder. The device is equipped with a Linux audio frame synchronization unit, which uses a circular buffer with a frame number threshold to trigger inference for audio frames.

[0123] The embedded voice processing intelligent device is suitable for full-duplex voice communication scenarios such as building access control and indoor intercom.

[0124] The processor can be an ARM Cortex-A7 / A53 / A55 architecture; the preset frame threshold for the circular buffer can be 2 to 5 frames; the inference model can be deployed using INT8 quantization and TFLiteMicro lightweight deployment.

[0125] In this embodiment, to better run and process the method described in one of the embodiments, the above method is stored in a memory, and the stored method is executed using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0126] Example 4 A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in one of the embodiments.

[0127] In this embodiment, to better operate and use the method described in one of the embodiments, the above method is stored in a computer-readable storage medium, and the above method is implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0128] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A Linux system echo cancellation method based on the DTLN-AEC model, characterized in that, The method includes: S1. Resample the collected audio data to a preset sampling rate, and extract the amplitude spectrum of the near-end microphone signal and the far-end reference signal through short-time Fourier transform and perform smoothing and noise reduction preprocessing. S2. Construct the DTLN-AEC model, which includes: A frequency domain feature encoder is used to perform nonlinear transformation on the amplitude spectrum to extract frequency domain features; The temporal feature encoder uses depthwise separable convolution to extract temporal features; The feature fusion module is used to stitch together and fuse the frequency domain features and time domain features of the near end and the far end; Unidirectional LSTM is used for temporal modeling of the fused features; Channel attention mechanism and temporal attention mechanism are applied sequentially to the output features of LSTM, respectively performing adaptive weighted enhancement on the channel dimension and the temporal dimension; A dual-mask decoder is used to generate frequency domain masks and time domain masks; The mask fusion module is used to fuse the frequency domain mask and the time domain mask with the original near-end amplitude spectrum to obtain the enhanced amplitude spectrum; The DTLN-AEC model is quantized and then deployed in a lightweight manner. S3. Under the Linux system, add a timestamp to each audio frame and realize the synchronization of acquisition and processing through a circular buffer. The circular buffer has a preset frame number threshold. When the number of cached frames reaches the threshold, model inference is triggered. S4. Input the synchronized near-end and far-end amplitude spectra into the deployed DTLN-AEC model. After model processing, the enhanced amplitude spectrum is output. Then, the time-domain signal is reconstructed through inverse short-time Fourier transform to obtain clean near-end speech.

2. The Linux system echo cancellation method based on the DTLN-AEC model as described in claim 1, characterized in that, The preset sampling rate is 8kHz to 16kHz, and the preset frame rate threshold is 2 to 5 frames.

3. The Linux system echo cancellation method based on the DTLN-AEC model as described in claim 1, characterized in that, The frequency domain feature encoder consists of a three-layer fully connected network. Each layer performs linear transformation, layer normalization, GELU activation and dropout operations in sequence. The first layer compresses the high-dimensional input features to a low dimension, the second layer expands the low-dimensional features by 2 times, and the third layer re-compresses the expanded features to a low dimension.

4. The Linux system echo cancellation method based on the DTLN-AEC model as described in claim 1, characterized in that, The temporal feature encoder adopts a structure that combines depthwise separable convolution with residual connections. The depthwise separable convolution includes depthwise convolution and 1×1 point convolution. The residual connections use 1×1 convolution to match dimensions when the input and output dimensions are inconsistent.

5. The Linux system echo cancellation method based on the DTLN-AEC model as described in claim 1, characterized in that, The channel attention mechanism generates a channel attention mask through global average pooling, multilayer perceptron, and sigmoid function; the temporal attention mechanism generates a temporal attention mask through one-dimensional convolution and Softmax function; the channel attention mechanism and the temporal attention mechanism are executed sequentially, with channel attention being executed first and then temporal attention.

6. The Linux system echo cancellation method based on the DTLN-AEC model as described in claim 1, characterized in that, The fusion methods of the mask fusion module include: element-wise mask multiplication fusion and weighted adaptive mask fusion.

7. The Linux system echo cancellation method based on the DTLN-AEC model as described in claim 1, characterized in that, The training process of the DTLN-AEC model adopts a combined loss function; an automatic stopping mechanism is used during training, which stops training when the model's score on the validation set does not improve for several consecutive rounds; or automatic stopping of training is achieved by monitoring the learning rate threshold.

8. An echo cancellation system for Linux systems based on the DTLN-AEC model, characterized in that, Based on the method according to any one of claims 1-7, the system comprises: The audio acquisition and preprocessing module is used to resample the acquired audio data to a preset sampling rate, and extract the amplitude spectrum of the near-end microphone signal and the far-end reference signal through short-time Fourier transform and perform smoothing and noise reduction preprocessing. A lightweight DTLN-AEC model inference engine, deployed in Linux user space, includes the following models: A frequency domain feature encoder is used to perform nonlinear transformation on the amplitude spectrum to extract frequency domain features; The temporal feature encoder uses depthwise separable convolution to extract temporal features; The feature fusion module is used to stitch together and fuse the frequency domain features and time domain features of the near end and the far end; Unidirectional LSTM is used for temporal modeling of the fused features; Channel attention mechanism and temporal attention mechanism are applied sequentially to the output features of LSTM, respectively performing adaptive weighted enhancement on the channel dimension and the temporal dimension; A dual-mask decoder is used to generate frequency domain masks and time domain masks; The mask fusion module is used to fuse the frequency domain mask and the time domain mask with the original near-end amplitude spectrum to obtain the enhanced amplitude spectrum; The model in question is a lightweight model that has undergone quantization and lightweight deployment processing. The frame synchronization module is used to add a timestamp to each audio frame under the Linux system. It achieves synchronization between acquisition and processing through a circular buffer. The circular buffer has a preset frame number threshold. When the number of cached frames reaches the threshold, model inference is triggered. The echo cancellation inference module is used to input the synchronized near-end and far-end amplitude spectra into the deployed DTLN-AEC model. After model processing, the enhanced amplitude spectrum is output, and the time-domain signal is reconstructed through inverse short-time Fourier transform to obtain clean near-end speech.

9. An embedded voice processing intelligent device, characterized in that, include: At least one processor and a memory, an audio acquisition device, and an audio player communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images