An intelligent audio noise reduction system based on an attention mechanism

Abstract

The application relates to the technical field of artificial intelligence, in particular to an intelligent audio noise reduction system based on an attention mechanism, which comprises an audio input end, a feature extraction module, a noise reduction module and an audio output end; the audio input end is used for acquiring a noisy audio signal and outputting the noisy audio signal; the feature extraction module is used for performing multi-scale time-frequency feature extraction and feature fusion on the noisy audio signal; the noise reduction module comprises a time sequence modeling unit, an attention unit and a noise estimation and suppression unit. The application can effectively distinguish speech and noise, avoid speech distortion, intelligently adjust attention weights, pertinently retain speech dominant area features and suppress noise dominant areas, iteratively converge to improve noise reduction effect and output high-quality noise reduction audio.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intelligent audio noise reduction system based on an attention mechanism. Background Technology

[0002] In scenarios such as far-field voice interaction, smart speakers, and voice wake-up, sound signals are constantly interfered with by ambient noise during propagation. Although traditional systems can suppress steady-state noise to some extent, existing statistical models cannot accurately estimate noise components for non-steady-state noise sources. This results in excessive reduction of the spectrum of the target speech or severe noise residue, which seriously affects the robustness of the speech recognition system and the naturalness of the speech synthesis system.

[0003] On the other hand, due to limitations in computing resources, edge devices often deploy lightweight noise reduction models. These models have limited parameters and computational complexity, making it impossible for them to effectively model the long-term evolution of noise. This results in a significant decrease in noise reduction performance when the noise type changes abruptly. Furthermore, in complex acoustic environments, it is impossible to distinguish the time-frequency representation of the target speech from the interference noise, causing the time-frequency structure of the target speech to be destroyed, key semantic features such as vowel formants to be damaged, and pronunciation clarity to decrease. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent audio noise reduction system based on an attention mechanism to solve the problems mentioned in the background art.

[0005] This invention provides an intelligent audio noise reduction system based on an attention mechanism, comprising: An audio input terminal is used to acquire a noisy frequency signal and output the noisy frequency signal. A feature extraction module is located at the output of the audio input terminal. The feature extraction module includes multiple parallel branches and a feature fusion layer. Each parallel branch is equipped with a time-frequency analysis unit of different scales. The time-frequency analysis unit is used to perform feature extraction on the noisy frequency signal and obtain the corresponding time-frequency features. The feature fusion layer is used to perform fusion processing on the time-frequency features and obtain feature representations. A noise reduction module is located at the output of the feature extraction module. The noise reduction module includes a temporal modeling unit, an attention unit, and a noise estimation and suppression unit. The temporal modeling unit is used to perform temporal dependency modeling on the feature representation and obtain temporal modeling features. The attention unit is used to calculate the attention weights of the temporal modeling features and adjust the temporal modeling features according to the attention weights to determine the speech-dominant region and the noise-dominant region, and generate attention-weighted features. The noise estimation and suppression unit is used to estimate noise features based on the noise-dominant region and generate a noise reference signal, and obtain an intermediate noise-reduced signal. The audio output terminal is used to receive the signal output by the noise reduction module and feed the intermediate noise reduction signal back to the feature extraction module to perform iterative noise reduction processing. After N iterations and convergence, the final noise-reduced audio signal is output.

[0006] In some embodiments, the plurality of parallel branches includes a first branch, a second branch, and a third branch; wherein, The first branch is used to extract high-frequency detail features from the noisy frequency signal; The second branch is used to extract low-frequency fundamental frequency features from the noisy frequency signal; The third branch is used to extract multi-resolution time-frequency features from the noisy frequency signal.

[0007] In some embodiments, the feature fusion layer includes a learnable projection matrix for mapping the time-frequency features of each of the parallel branches to a unified feature space and performing feature merging.

[0008] In some embodiments, the temporal modeling unit employs a bidirectional long short-term memory network structure to perform forward and backward temporal feature extraction on the feature representation.

[0009] In some embodiments, the attention unit includes a computation subunit for calculating adaptive attention weights of the temporal modeling features and generating the attention-weighted features.

[0010] In some embodiments, the noise estimation and suppression unit includes a spectral subtraction subunit and a Wiener filtering subunit; wherein... The spectral subtraction unit is used to perform power spectrum estimation on the intermediate noise-reduced signal, calculate the noise power spectrum, and perform subtraction operation on the spectrum of the noisy frequency signal according to the noise power spectrum to obtain the preliminary noise-reduced spectrum; The Wiener filtering subunit is used to calculate the speech power spectrum and noise power spectrum according to the speech-dominant region and the noise-dominant region respectively, construct Wiener filter coefficients, and perform Wiener filtering processing on the preliminary noise reduction spectrum.

[0011] In some embodiments, the attention weights calculated by the attention unit include temporal attention weights and frequency attention weights. The temporal attention weights are used to label the speech dominance of each time frame in the audio signal, and the frequency attention weights are used to label the speech dominance of each frequency bin. The noise estimation and suppression unit performs noise region determination and noise suppression based on the joint distribution of the temporal attention weights and the frequency attention weights.

[0012] In some embodiments, a detection module is further included, which is disposed at the output end of the audio input terminal and is used to perform voice activity detection on the noisy frequency signal and generate a voice activity flag bit; the noise reduction module performs a complete noise reduction process in the voice active region according to the voice activity flag bit, and performs only a lightweight noise reduction process in the non-voice region.

[0013] In some embodiments, a post-processing enhancement module is further included, which is disposed between the noise reduction module and the audio output terminal, and is used to perform post-processing on the noise-reduced audio signal after iterative convergence.

[0014] In some embodiments, the post-processing enhancement module includes a spectral smoothing unit and a smoothing convolution unit; the spectral smoothing unit is used to perform smoothing processing on adjacent frequency points of the spectrum of the noise-reduced frequency signal; the smoothing convolution unit is used to perform adaptive smoothing convolution on the time-domain waveform of the noise-reduced frequency signal.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves effective differentiation between speech and noise in complex noise environments by coordinating the work of the feature extraction module and the noise reduction module, thus avoiding the speech distortion problem in traditional noise reduction methods; 2. This invention, through the attention unit in the noise reduction module, can dynamically adjust the attention weight according to different audio content, making the noise reduction process more intelligent. It can selectively preserve the characteristics of the speech-dominant region while effectively suppressing the noise-dominant region, ensuring the naturalness of the speech after noise reduction. 3. This invention further improves the noise reduction effect through an iterative convergence mechanism at the audio output end, making the output audio quality significantly better than the result of a single noise reduction process, thus meeting the demand for high-quality audio in scenarios such as real-time voice communication, smart speakers, and voice assistants. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a diagram of the intelligent audio noise reduction system based on the attention mechanism of the present invention. Figure 2 This is a diagram of the noise reduction module architecture of the present invention. Detailed Implementation

[0018] The following will refer to the appendices in the embodiments of the present invention. Figure 1-2 The technical solutions in the embodiments of the present invention will be clearly and completely described together. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] The overall system architecture of this invention is as follows: Figure 1 As shown, the system includes an audio input terminal, a feature extraction module, a noise reduction module, and an audio output terminal. The feature extraction module performs multi-scale feature extraction on the noisy frequency signal through multiple parallel branches to obtain time-frequency features at different resolutions, and performs fusion processing through a feature fusion layer to obtain rich feature representations. The noise reduction module models temporal dependencies through a temporal modeling unit and calculates attention weights through an attention unit to intelligently distinguish between speech-dominated and noise-dominated regions, achieving adaptive precipitation decisions. The noise estimation and suppression unit accurately estimates noise based on attention-weighted features and generates a noise reference signal. The noise reference signal is subtracted from the noisy frequency signal to complete the noise reduction process. The audio output terminal feeds back the intermediate noise-reduced signal to the feature extraction module for iterative convergence, ultimately outputting a high-quality noise-reduced frequency signal.

[0020] first, The audio input terminal receives a noisy frequency signal from an external audio acquisition device, performs digital sampling of the noisy frequency signal at a preset sampling rate (e.g., 16kHz), and transmits the sampled digital audio signal to the feature extraction module for subsequent processing.

[0021] In some embodiments, the noise-containing frequency signal may include ambient noise, background noise, or other interfering sounds.

[0022] Secondly The feature extraction module performs multi-scale feature extraction on noisy frequency signals through multiple parallel branches. Each branch is equipped with time-frequency analysis units of different scales, which can capture feature information of noisy frequency signals from different resolutions and time scales. Then, the feature extraction module fuses the multi-scale time-frequency features extracted by multiple parallel branches through a feature fusion layer to obtain the fused multi-scale feature representation, and transmits the multi-scale feature representation to the noise reduction module.

[0023] In detail, the multiple parallel branches include a first branch, a second branch, and a third branch. The first branch sets up a short-time Fourier transform unit to perform a 1024-point short-time Fourier transform on the noisy frequency signal. The window function is a Hanning window, and the frame shift is set to 256 sampling points. This can extract high-frequency detail features of the noisy frequency signal with high time resolution, such as the consonant part of speech and transient noise. The second branch sets up a long-time Fourier transform unit to perform a 4096-point long-time Fourier transform on the noisy frequency signal. The window function is a Hanning window, and the frame shift is set to 1024 sampling points. This can extract low-frequency fundamental frequency features of the noisy frequency signal with high frequency resolution, such as the vowel part of speech and periodic noise. The third branch sets up a wavelet transform unit to perform three-level wavelet decomposition on the noisy frequency signal. The Daubechies wavelet (db4) is used as the mother wavelet. This can extract multi-resolution time-frequency features in both the time and frequency domains simultaneously and capture non-stationary components in the noisy frequency signal. The output features of the first, second, and third branches have different dimensions and resolutions, and are subsequently processed uniformly through a feature fusion layer.

[0024] In detail, the feature fusion layer works as follows: First, the feature fusion layer maps the features of the first, second, and third branches to a unified feature space using a learnable projection matrix. The dimension of the projection matrix can be set according to actual needs, for example, mapping the features of the first, second, and third branches to a unified 256-dimensional feature space. The learnable projection matrix is ​​implemented through linear transformation, and its weight parameters are automatically learned during training, which can adaptively adjust the fusion ratio of features from different branches. Then, the feature fusion layer uses an additive fusion strategy to merge the mapped features, adding the features of the first, second, and third branches along the feature dimension to obtain a fused multi-scale feature representation. The additive fusion strategy can preserve the complementary information of each branch, and at the same time, it achieves adaptive weighted fusion of features through the learnable projection matrix, improving the integrity and discriminativeness of the feature representation.

[0025] then, The noise reduction module receives the fused multi-scale feature representation from the feature fusion layer. The detailed processing flow is as follows: The temporal modeling unit performs temporal dependency modeling on the feature representation. The temporal modeling unit adopts a bidirectional long short-term memory network structure, including 512 hidden units, which can simultaneously perform forward and backward temporal feature extraction, capture long-term dependencies and contextual information in noisy frequency signals, and obtain temporal modeling features.

[0026] In detail, the temporal modeling unit employs a bidirectional long short-term memory (Bi-LSTM) network structure. This network consists of two LSTMs, one for forward and one for backward temporal feature extraction. Each LSTM unit includes three gating mechanisms: an input gate, a forget gate, and an output gate. These mechanisms selectively retain or discard historical information, effectively mitigating the gradient vanishing problem. The fused multi-scale feature representations are sequentially input into the Bi-LSTM network according to the time sequence. The Bi-LSTM network performs forward and backward feature extraction on the input at each time step, outputting the corresponding temporal modeling features. These temporal modeling features can characterize the dynamic changes of noisy frequency signals in the time dimension, providing rich temporal contextual information for subsequent adaptive attention computation.

[0027] Temporal modeling features are input into the attention unit, which calculates adaptive attention weights for the temporal modeling features. These attention weights can label the degree of speech dominance in each time frame and frequency bin of the audio signal. The attention unit adjusts the temporal modeling features according to the attention weights, intelligently distinguishing between speech-dominant and noise-dominant regions, and generating attention-weighted features. In detail, the attention unit includes computational subunits, which further include query matrix computation subunits, key matrix computation subunits, and value matrix computation subunits. The query matrix computation subunit calculates the query matrix Q, the key matrix computation subunit calculates the key matrix K, and the value matrix computation subunit calculates the value matrix V. Here, the query matrix Q represents the query vector at the current time step, the key matrix K represents the key vectors at all time steps, and the value matrix V represents the value vectors at all time steps. The attention score matrix is ​​obtained by calculating the matrix product of the query matrix Q and the transpose of the key matrix K. This matrix is ​​then scaled using a scaling factor and normalized using the softmax function to obtain the attention weight matrix. The attention weight matrix and the value matrix V are multiplied to obtain the attention weighted features. The attention weighted features can adaptively adjust the weights of different time frames and frequency bins according to the content, giving higher attention weights to speech-dominant regions and lower attention weights to noise-dominant regions.

[0028] In detail, the attention unit further calculates the temporal attention weights and frequency attention weights separately through computational subunits. The temporal and frequency attention weights are then combined through an outer product operation to obtain a complete attention weight matrix. This matrix more accurately locates speech-dominant and noise-dominant regions, providing a more precise basis for subsequent noise estimation and suppression. Specifically, the temporal attention weights are used to label the speech dominance of each time frame in the audio signal. The calculation process is as follows: First, the temporal modeling features are globally pooled along the frequency dimension to obtain a temporal feature representation. Then, a temporal attention weight vector is calculated using two fully connected layers and a ReLU activation function. This vector has the same number of time frames, and each element represents the speech dominance of the corresponding time frame. Similarly, the frequency attention weights are used to label the speech dominance of each frequency bin. The calculation process is as follows: First, the temporal modeling features are globally pooled along the time dimension to obtain a frequency feature representation. Then, a frequency attention weight vector is calculated using two fully connected layers and a ReLU activation function. This vector has the same number of frequency bins, and each element represents the speech dominance of the corresponding frequency bin.

[0029] Attention-weighted features are input to the noise estimation and suppression unit. Based on the noise-dominant region in the attention-weighted features, the noise estimation and suppression unit accurately estimates the noise features and generates a noise reference signal. The noise reference signal is then subtracted from the noisy frequency signal to obtain the intermediate denoised signal. Specifically, the noise estimation and suppression unit includes a spectral subtraction subunit and a Wiener filtering subunit. First, the spectral subtraction subunit performs power spectrum estimation on the intermediate denoised signal, calculates the noise power spectrum, and subtracts the spectrum of the noisy frequency signal based on the noise power spectrum to obtain the preliminary denoised spectrum. Next, the Wiener filtering subunit calculates the speech power spectrum and noise power spectrum based on the speech-dominant region and noise-dominant region in the attention-weighted features, constructs Wiener filter coefficients, and performs Wiener filtering on the preliminary denoised spectrum to further suppress residual noise, obtaining the intermediate denoised signal.

[0030] In detail, the noise estimation and suppression unit performs more accurate noise region identification and noise suppression based on the joint distribution of temporal and frequency attention weights output by the attention unit. Specifically, the noise estimation and suppression unit first determines the noise-dominant region based on the attention weight matrix and sets an attention threshold. When the attention weight of a certain time frame-frequency bin combination is lower than this threshold, it is identified as a noise-dominant region. Then, the noise estimation and suppression unit only performs noise feature estimation and noise suppression on the noise-dominant region, while performing lightweight processing or leaving the speech-dominant region unchanged. By using a region discrimination strategy based on attention weights, the speech distortion problem caused by the one-size-fits-all noise reduction in traditional noise reduction methods can be avoided, effectively suppressing noise while preserving the integrity and naturalness of the speech to the maximum extent.

[0031] In detail, the noise estimation and suppression unit is a two-stage noise reduction process: the first stage is a spectral subtraction subunit, and the second stage is a Wiener filtering subunit. The spectral subtraction subunit employs a traditional single-channel speech enhancement method, subtracting the estimated noise power spectrum from the power spectrum of the noisy speech to obtain the enhanced speech power spectrum. Specifically: first, the spectral subtraction subunit performs a short-time Fourier transform on the noisy frequency signal to obtain the spectrum of the noisy speech; then, based on the noise-dominant region in the attention-weighted features, it estimates the noise power spectral density; finally, it subtracts the noise power spectrum from the power spectrum of the noisy speech to obtain the preliminary denoised spectrum. The output of the spectral subtraction subunit may suffer from spectral jaggedness and musical noise, thus requiring further processing by the Wiener filtering subunit. Specifically: The spectral subtraction unit performs power spectrum estimation on the intermediate denoised signal, calculates the noise power spectrum, and performs subtraction on the spectrum of the noisy frequency signal based on the noise power spectrum to obtain the preliminary denoised spectrum. The specific process is as follows: First, the noisy frequency signal is processed by frame-by-frame windowing using a Hanning window, with a frame length of 25ms and a frame shift of 10ms. Then, a short-time Fourier transform is performed on each frame to obtain the spectrum of the noisy speech. Next, based on the noise-dominant region in the attention-weighted features, the minimum statistical tracking algorithm is used to estimate the noise power spectrum. Finally, the power spectrum of the noisy speech is calculated and spectral subtraction is performed. The square root of the subtracted power spectrum is taken to obtain the amplitude spectrum of the preliminary denoised spectrum. Combined with the phase spectrum of the original noisy frequency signal, an inverse short-time Fourier transform is performed to obtain the preliminary denoised signal, i.e., the intermediate denoised signal.

[0032] The Wiener filtering subunit calculates the speech power spectrum and noise power spectrum based on the speech-dominant and noise-dominant regions in the attention-weighted features, constructs Wiener filter coefficients, and performs Wiener filtering on the initial denoised spectrum to further suppress residual noise. It achieves optimal filter coefficients under the minimum mean square error criterion, effectively suppressing residual noise and musical noise left by the spectral subtraction subunit, and improving the perceived quality of the denoised speech. The specific process is as follows: First, based on the speech-dominant and noise-dominant regions in the attention-weighted features, the speech power spectrum and noise power spectrum are calculated. For the speech-dominant region, the Wiener filtering subunit treats the power spectrum of that region as primarily the speech power spectrum; for the noise-dominant region, it treats the power spectrum of that region as primarily the noise power spectrum. Then, Wiener filter coefficients are constructed based on the speech power spectrum and noise power spectrum. Finally, the Wiener filter coefficients are multiplied point-by-point with the initial denoised spectrum to obtain the Wiener-filtered spectrum, which is then subjected to a short-time Fourier inverse transform to obtain the further denoised signal.

[0033] at last, The audio output terminal receives the signal output by the noise reduction module and feeds back the intermediate noise reduction signal to the feature extraction module to perform iterative noise reduction processing. After N iterations and convergence, the final noise-reduced audio signal is output.

[0034] In some preferred embodiments, the audio output terminal feeds back the intermediate denoised signal output by the noise estimation and suppression unit to the input terminal of the feature extraction module to perform the next round of denoising processing. After N iterations and convergence, the final denoised audio signal is output. During iterative denoising processing, each iteration uses the intermediate denoised signal generated in the previous round as input to re-perform multi-scale feature extraction, temporal modeling, adaptive attention calculation, and noise estimation and suppression, gradually approximating the clean speech signal. After N iterations and convergence (where N is a preset number of iterations, such as 5), the audio output terminal outputs the final denoised audio signal, which meets the preset standards in terms of speech quality, clarity, and naturalness.

[0035] Furthermore, after N iterations, the audio output terminal outputs the final noise-reduced frequency signal. One optional convergence determination mechanism is to calculate the scale-invariant signal-to-noise ratio gain (SI-SNRi) between the noise-reduced signal output in the current iteration and the noise-reduced signal output in the previous iteration. When SI-SNRi is less than a preset threshold (e.g., 0.5dB), convergence is determined, and the iteration is terminated early; otherwise, it is terminated after completing the preset N iterations. Finally, the signal with the highest SI-SNRi or the signal obtained in the last iteration is output as the final noise-reduced frequency signal.

[0036] In some preferred embodiments, the system further includes a detection module, which is set at the output of the audio input terminal and is used to perform voice activity detection on the noisy frequency signal and generate a voice activity flag. The noise reduction module performs a complete noise reduction process in the voice active region according to the voice activity flag, and performs only a lightweight noise reduction process in the non-voice region, thereby reducing computational complexity and avoiding artificial noise caused by excessive noise reduction.

[0037] In detail, the detection module employs a speech activity detection algorithm based on a deep neural network. This algorithm uses Mel-frequency cepstral coefficients as input features and outputs the speech activity probability through a multi-layer fully connected network. The speech activity detection module performs frame-by-frame analysis on the noisy frequency signal. When the speech activity probability of a frame exceeds a preset threshold (e.g., 0.5), the frame is marked as a speech active frame, and a speech activity flag is generated with a value of 1; otherwise, it is marked as a non-speech frame, and the speech activity flag is set to 0. The speech activity flag is transmitted to the noise reduction module to guide the noise reduction process.

[0038] Furthermore, the noise reduction module executes an adaptive noise reduction strategy based on the speech activity flag. For active speech regions (i.e., regions where the speech activity flag is 1), the noise reduction module performs a complete noise reduction process, including multi-scale feature extraction, temporal modeling, adaptive attention calculation, noise estimation, and suppression, thereby removing noise and preserving speech to the greatest extent possible. For non-speech regions (i.e., regions where the speech activity flag is 0), the noise reduction module performs only lightweight noise reduction, such as simple spectral subtraction, without performing complex multi-scale feature extraction and adaptive attention calculation. This adaptive noise reduction strategy significantly reduces the computational complexity of the system, enabling it to meet real-time processing requirements; it also avoids excessive noise reduction in non-speech regions, reduces the generation of artificial noise, and improves the naturalness of the denoised audio.

[0039] In some preferred embodiments, the system further includes a post-processing enhancement module, which is located between the noise reduction module and the audio output. This module performs post-processing on the noise-reduced audio signal after iterative convergence to further improve the voice quality. The post-processing enhancement module includes a spectral smoothing unit and a smoothing convolution unit; wherein: The spectrum smoothing unit performs a short-time Fourier transform on the denoised frequency signal to obtain its spectral representation. Then, it performs weighted average smoothing on adjacent frequency points to eliminate the jagged effect in the spectrum. The specific implementation process of the spectrum smoothing unit is as follows: for each frequency bin in the spectrum, it calculates the weighted average value with its adjacent frequency bins. The weights are determined according to a preset smoothing coefficient, for example, using a Gaussian smoothing kernel with a smoothing coefficient set to 0.5. The spectrum smoothing unit can make the spectral curve smoother and reduce the loss of spectral detail caused by noise reduction processing.

[0040] The smoothing convolution unit performs adaptive smoothing convolution on the time-domain waveform of the noise-reduced frequency signal, employing a low-pass filter of adjustable length. The filter parameters are adaptively adjusted based on the signal's local statistical characteristics. The specific implementation process of the smoothing convolution unit involves calculating the variance of the noise-reduced frequency signal within a sliding window. When the variance is large, a shorter filtering window is used; when the variance is small, a longer filtering window is used. Through adaptive smoothing convolution processing, glitches and noise in the time-domain waveform can be eliminated while preserving speech intelligibility, thus improving the naturalness and intelligibility of the speech.

[0041] In some preferred embodiments, the system further includes a voiceprint registration module for registering the voiceprint features of the target speaker, enabling personalized noise reduction based on voiceprints. Specifically, the voiceprint registration module employs speaker recognition technology based on x-vector or ECAPA-TDNN, capable of extracting unique voiceprint feature vectors from the target speaker's speech samples and storing them in a voiceprint feature library. The voiceprint registration module operates as follows: First, it receives the registered speech sample from the target speaker, which can be an audio file containing the target speaker's speech or a real-time acquired speech stream; then, it performs preprocessing on the registered speech sample, including framing, windowing, and extraction of acoustic features (such as MFCC, FBANK, etc.); next, it inputs the preprocessed features into a pre-trained recognition model to extract the speaker's voiceprint feature vector; finally, it stores the voiceprint feature vector in the voiceprint feature library and assigns a unique speaker ID.

[0042] Furthermore, during the noise reduction process, the noise reduction module enhances the target speech in the audio signal based on the voiceprint characteristics of the target speaker, and suppresses non-target speech or other interfering speech to a greater extent, achieving personalized noise reduction based on voiceprint. The specific process is as follows: First, the noise reduction module uses a recognition model to perform speaker recognition on the speech in the current noisy frequency signal, extracting the voiceprint feature vector of the current speech. Then, it performs a similarity calculation (e.g., cosine similarity) between the current speech's voiceprint feature vector and the target speaker's voiceprint feature vector stored in the voiceprint registration module. If the similarity is higher than a preset threshold (e.g., 0.8), the current speech is determined to be the target speaker's speech; otherwise, it is determined to be non-target speech or interfering speech. During the noise reduction process, for the target speaker's speech, the noise reduction module uses a lighter noise reduction intensity to preserve the integrity of the speech to the greatest extent; for non-target speech or interfering speech, the noise reduction module uses a stronger noise reduction intensity to suppress noise to a greater extent.

[0043] In some preferred embodiments, the system is implemented using a deep learning model. One training method for this model is as follows: using a training dataset comprising clean speech signals and noise signals, the clean speech and noise are mixed at different signal-to-noise ratios to generate noisy frequency signals. During training, the generated noisy frequency signals are used as model input, and the corresponding clean speech signals are used as training targets. Scale-invariant signal-to-noise ratio loss (SI-SNR Loss) is used as the loss function to optimize the parameters of the entire model. Through the backpropagation algorithm, the projection matrix in the model automatically learns how to map features of different scales to a unified feature space that is beneficial to the noise reduction task; the attention network also automatically learns how to calculate attention weights that can effectively distinguish between speech and noise based on the current input features, without the need for manual rule formulation. In summary, the intelligent audio noise reduction system based on the attention mechanism provided in this invention comprehensively captures speech and noise information in noisy frequency signals by extracting time-frequency features at different scales in parallel through the feature extraction module; it models temporal dependencies through the temporal modeling unit in the noise reduction module, and the attention unit intelligently distinguishes between speech-dominant and noise-dominant regions to achieve adaptive noise reduction decisions; it accurately estimates and suppresses noise through the noise estimation and suppression unit to complete the noise reduction process; it further improves the noise reduction effect through an iterative convergence mechanism; it performs spectral smoothing and temporal smoothing through the post-processing enhancement module to improve the naturalness and intelligibility of the speech; it implements an adaptive noise reduction strategy through the detection module to reduce computational complexity; and it achieves personalized noise reduction based on voiceprint through the voiceprint registration module. This application can effectively overcome the technical defects of existing audio noise reduction technologies, such as speech distortion and poor noise reduction performance in complex noisy environments.

[0044] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0045] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment includes only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An intelligent audio noise reduction system based on an attention mechanism, characterized in that, include: An audio input terminal is used to acquire a noisy frequency signal and output the noisy frequency signal. A feature extraction module is located at the output of the audio input terminal. The feature extraction module includes multiple parallel branches and a feature fusion layer. Each parallel branch is equipped with a time-frequency analysis unit of different scales. The time-frequency analysis unit is used to perform feature extraction on the noisy frequency signal and obtain the corresponding time-frequency features. The feature fusion layer is used to perform fusion processing on the time-frequency features and obtain feature representations. A noise reduction module is located at the output of the feature extraction module. The noise reduction module includes a temporal modeling unit, an attention unit, and a noise estimation and suppression unit. The temporal modeling unit is used to perform temporal dependency modeling on the feature representation and obtain temporal modeling features. The attention unit is used to calculate the attention weights of the temporal modeling features and adjust the temporal modeling features according to the attention weights, determine the speech-dominant region and the noise-dominant region, and generate attention-weighted features. The noise estimation and suppression unit is used to estimate noise characteristics based on the noise-dominant region, generate a noise reference signal, and obtain an intermediate noise-reduced signal. The audio output terminal is used to receive the signal output by the noise reduction module and feed the intermediate noise reduction signal back to the feature extraction module to perform iterative noise reduction processing. After N iterations and convergence, the final noise-reduced audio signal is output.

2. The system according to claim 1, characterized in that, The multiple parallel branches include a first branch, a second branch, and a third branch; wherein, The first branch is used to extract high-frequency detail features from the noisy frequency signal; The second branch is used to extract low-frequency fundamental frequency features from the noisy frequency signal; The third branch is used to extract multi-resolution time-frequency features from the noisy frequency signal.

3. The system according to claim 1, characterized in that, The feature fusion layer includes a learnable projection matrix, which is used to map the time-frequency features of each of the parallel branches to a unified feature space and perform feature merging.

4. The system according to claim 1, characterized in that, The temporal modeling unit employs a bidirectional long short-term memory network structure to perform forward and backward temporal feature extraction on the feature representation.

5. The system according to claim 4, characterized in that, The attention unit includes a computation subunit for calculating the adaptive attention weights of the temporal modeling features and generating the attention-weighted features.

6. The system according to claim 1, characterized in that, The noise estimation and suppression unit includes a spectral subtraction subunit and a Wiener filtering subunit; wherein... The spectral subtraction unit is used to perform power spectrum estimation on the intermediate noise-reduced signal, calculate the noise power spectrum, and perform subtraction operation on the spectrum of the noisy frequency signal according to the noise power spectrum to obtain the preliminary noise-reduced spectrum; The Wiener filtering subunit is used to calculate the speech power spectrum and noise power spectrum according to the speech-dominant region and the noise-dominant region respectively, construct Wiener filter coefficients, and perform Wiener filtering processing on the preliminary noise reduction spectrum.

7. The system according to claim 1, characterized in that, The attention weights calculated by the attention unit include temporal attention weights and frequency attention weights. The temporal attention weights are used to label the degree of speech dominance in each time frame of the audio signal, and the frequency attention weights are used to label the degree of speech dominance in each frequency bin. The noise estimation and suppression unit performs noise region determination and noise suppression based on the joint distribution of the time attention weight and the frequency attention weight.

8. The system according to claim 1, characterized in that, It also includes a detection module, which is set at the output end of the audio input terminal, for performing voice activity detection on the noisy frequency signal and generating a voice activity flag; the noise reduction module performs a complete noise reduction process in the voice active range according to the voice activity flag, and performs only a lightweight noise reduction process in the non-voice range.

9. The system according to claim 1, characterized in that, It also includes a post-processing enhancement module, which is located between the noise reduction module and the audio output terminal, and is used to perform post-processing on the noise-reduced audio signal after iterative convergence.

10. The system according to claim 9, characterized in that, The post-processing enhancement module includes a spectral smoothing unit and a smoothing convolution unit; the spectral smoothing unit is used to perform smoothing processing on adjacent frequency points of the spectrum of the noise-reduced frequency signal; the smoothing convolution unit is used to perform adaptive smoothing convolution on the time-domain waveform of the noise-reduced frequency signal.

Images