Method for reducing noise of complex noise voice based on Mama architecture

Abstract

The invention relates to a voice signal filtering and noise reduction method based on a Mama generative adversarial network, and the method comprises the following steps: designing a noise synthesis strategy for different application scenes, collecting real environment noise to construct a mixed noise library, and constructing a voice data set with noise and a voice data set without noise, and the voice data set and the voice data set do not need to be matched; constructing a Mama-based generative adversarial network, and realizing end-to-end conversion from the noisy voice to a time sequence filtering result; the construction of the model comprises the steps of designing a generator network, designing a discriminator network and optimizing a loss function. According to the invention, the robustness and generalization ability of voice signal noise reduction can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer audio signal processing, in particular to a method for reducing noise of complex noise speech based on Mamba architecture. BACKGROUND

[0002] At present, there are many kinds of methods for speech noise reduction based on deep learning. Such algorithms learn the complex characteristics of speech and noise through data-driven methods, and have good noise reduction effect on speech in non-stationary and complex noise scenes. In such algorithms, end-to-end models based on supervised learning are more common, which usually take the spectral features of noisy speech as input and output clean speech or noise mask. Typical models such as convolutional neural network (CNN) are good at extracting local features, but have deficiencies in capturing long-term sequential dependencies of speech signals; recurrent neural network (RNN / LSTM / GRU) can capture time sequence dependencies to some extent, but it is prone to gradient vanishing or explosion when processing long sequence speech, and the computational efficiency is not high; the Transformer model can model long-term dependencies through self-attention mechanism, but its computational complexity increases quadratically with the increase of sequence length, which limits its application in real-time scenarios.

[0003] The self-attention mechanism has high computational and memory requirements for long time sequences in resource-constrained environments, while the selective state mechanism of the Mamba architecture can effectively combine short-term and long-term dependencies, process global information of long time sequences while preserving signal details. By dynamically selecting the updated state to capture important information in the time sequence signal, and by selectively filtering the input signal, unnecessary computational burden is reduced. This enables it to significantly reduce computational complexity while maintaining complex time sequence feature modeling capabilities.

[0004] The traditional generative adversarial network model based on Mamba architecture strictly depends on completely matched noisy and clean speech sample pairs (i.e. noisy speech and corresponding clean speech of the same speaker under the same content and acoustic conditions) in the training process. This requirement results in extremely high cost of training data acquisition; in real-world scenarios, complex noise is random and diverse, making it difficult to record a large number of one-to-one corresponding noisy and clean speech pairs. For scenarios with many types of noise, it is necessary to specially superimpose corresponding noise on each clean speech in each scenario to generate noisy samples, which not only complicates the process, but also makes it difficult to simulate the natural mixing state of noise in real environments. Since the training data relies on artificially constructed paired samples, the model can only adapt to specific noise types and intensities in the training set, and the noise reduction performance will be significantly reduced when facing unseen noise scenarios, making it difficult to meet the needs of dynamic noise changes in actual applications. SUMMARY

[0005] The application provides a speech signal filtering and noise reduction method based on a Mamba generative adversarial network, which can improve the speech noise reduction capability of the model in a complex environment without a large number of matched pure and noise speech pairs.

[0006] A speech signal filtering and noise reduction method based on a Mamba generative adversarial network comprises the following steps:

[0007] Step S1, data preparation stage; a combination of public speech data set and real scene noise data set is adopted;

[0008] Step S2, model construction; a CycleGAN network model based on Mamba architecture is constructed, including a generator G, a generator F and discriminators D1 and D2, and the core is realized by meansing the time sequence modeling capability and cycle consistency loss of Mamba to realize non-paired data training;

[0009] Step S3, training the model constructed in step S2;

[0010] Step S4, applying the model trained in step S3 for noise reduction.

[0011] The application introduces a CycleGAN network model into the speech noise reduction task. Its core advantage is that it does not need paired noisy and pure speech samples, and only two sets of non-paired data (a set of noisy speech set, a set of clean speech set) can complete the training. CycleGAN designs two generators (generator G: noisy speech to clean speech; generator F: clean speech to noisy speech) and two discriminators (to distinguish the generated clean speech and the real clean speech, and the generated noisy speech and the real noisy speech), and introduces a cycle consistency loss constraint: after the noisy speech is generated by the generator G to generate "pseudo clean speech", it is restored to "reconstructed noisy speech" by the generator F, and the reconstructed result needs to be as consistent as possible with the original noisy speech; similarly, the clean speech is generated by the generator F to generate "pseudo noisy speech", and then it is restored to "reconstructed clean speech" by the generator G, which needs to be consistent with the original clean speech.

[0012] The speech noise reduction model constructed by the application is a CycleGAN network model based on Mamba architecture, and the model mainly comprises the following key parts:

[0013] I. CycleGAN Generator Setup: The core function of the generator G is to convert noisy speech to clean speech. Its internal structure adopts an advanced neural network architecture, combining the powerful long sequence modeling capability of the Mamba architecture. As a new type of state space model, Mamba architecture can efficiently capture the long-distance dependence of speech signals in the time dimension, which is crucial for handling the time-varying characteristics of speech signals in complex noise environments. Generator G learns the complex mapping relationship from noisy speech to clean speech through multiple layers of nonlinear transformation, gradually converting the input noisy speech spectrum features into spectrum features close to real clean speech. For example, when facing speech containing sudden noise, Mamba architecture can quickly capture the trend of speech signal changes before and after the noise appears, helping generator G more accurately restore the noise- disturbed speech part.

[0014] II. CycleGAN Discriminator Setup: Discriminators D1 and D2 are used to distinguish between generated clean speech and real clean speech, and generated noisy speech and real noisy speech, respectively. The discriminator adopts a carefully designed convolutional neural network structure, which can comprehensively analyze the spectral features, time domain features, etc. of the input speech. During training, the discriminator learns the feature distribution of real speech samples and judges the authenticity of the speech samples generated by the generator. If the feature difference between the speech samples generated by the generator and the real speech samples is large, the discriminator will output a low score, and vice versa. This adversarial training mechanism encourages the generator to continuously optimize itself to generate samples closer to real speech.

[0015] III. Mamba: Mamba architecture plays a core role in the entire model. In addition to being used in the generator to capture the long-distance dependence of speech signals, it also has important applications in other parts of the model. For example, when extracting features from speech signals, Mamba architecture can effectively suppress redundant features caused by noise interference while preserving key features of the speech. Moreover, Mamba architecture has relatively low computational complexity, which allows the model to maintain high efficiency when processing large-scale speech data, meeting the real-time requirements of high-demand application scenarios.

[0016] IV. Innovation introduces cycle consistency loss: After the noisy speech input into the generator G, the "pseudo clean speech" is generated, and then the "pseudo clean speech" is input into the generator F to restore the "reconstructed noisy speech". During the model training process, the "reconstructed noisy speech" is required to be consistent with the original noisy speech as much as possible, and the model parameters are optimized by minimizing the difference loss between the two. Similarly, after the clean speech is generated into "pseudo noisy speech" by the generator F, it is restored into "reconstructed clean speech" by the generator G. The "reconstructed clean speech" needs to be highly consistent with the original clean speech. This cycle consistency loss constraint mechanism enables the model to deeply learn the essential feature mapping relationship between speech and noise without relying on strictly paired samples.

[0017] This mechanism enables the model to learn the essential feature mapping relationship between speech and noise without relying on strictly paired samples, effectively solving the problem of the traditional GAN's dependence on paired data, greatly reducing the difficulty of data collection, and enabling direct use of public clean speech libraries and noisy speech libraries in different scenarios for training, covering a wider range of noise types; enhance the model's generalization ability to unknown noise, and the feature mapping learned through unpaired data is more universal, and still maintains stable noise reduction performance in complex noise environments. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a network model architecture schematic diagram of the present application.

[0019] Figure 2 is a flowchart of the noise reduction method of the present application. DETAILED DESCRIPTION

[0020] The present application will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present application and not to limit the scope of the present application. In addition, it should be understood that after reading the content taught by the present application, those skilled in the art can make various modifications or modifications to the present application, and these equivalent forms also fall within the scope defined by the claims attached hereto.

[0021] The present application provides a method for filtering and denoising time series speech signals based on a Mamba architecture-based generative adversarial network, designs a noise synthesis strategy for different application scenarios, collects real environment noise to construct a mixed noise library, and constructs a speech dataset with noise and without noise, which do not need to be matched. The noise-free speech comes from a public clean speech library, and the noisy speech is generated by randomly superimposing the noise-free speech and the noise in the mixed noise library according to a signal-to-noise ratio (SNR) of 0dB-20dB, and the same noise-free speech can be combined with different noises to generate multiple noisy samples.

[0022] A Mamba-based generative adversarial network is constructed to realize end-to-end conversion of noisy speech to time series filtering results, including:

[0023] The design of the first generator network is to extract and convert the features of the input noisy time series speech signal. The Mamba architecture efficiently captures the time series dependence and potential feature patterns in the speech signal, gradually maps the noisy speech features to the denoised speech feature representation, and generates the preliminary time series filtering result, providing a basis for subsequent optimization.

[0024] The design of the second discriminator network is to distinguish and evaluate the time series filtering results output by the generator and the real noise-free speech signal. By learning the difference between the real noise-free speech signal and the generated denoised speech signal, the generator provides feedback signals to guide the generator to continuously optimize the generated denoised speech signal, making it closer to the distribution of the real noise-free speech signal.

[0025] The optimization of the third loss function considers multiple loss terms, such as the adversarial loss (used to measure the degree of adversarial game between the generator and the discriminator, prompting the generator to generate more realistic denoised speech), the reconstruction loss (ensuring that the generated denoised speech is highly consistent with the original noise-free speech in content), and the like. Through optimization algorithms, the parameters of the generator and the discriminator are continuously adjusted, so that the entire network gradually converges during the training process, achieving efficient time series speech signal filtering and denoising effect.

[0026] The mixed noise library includes: various types of artificial synthesized noise, which is generated by mathematical models and algorithms according to the noise characteristics of different application scenarios, covering noise with different frequency ranges, intensity distributions and time domain characteristics; natural noise collected from various real environments, including but not limited to urban street environment noise, factory workshop mechanical noise, public place noise, etc., collected by professional audio collection equipment at different locations and different time periods to obtain rich and diverse real noise samples; mixed noise samples are obtained by mixing artificial synthesized noise and natural noise in different proportions and different ways to simulate various noise mixing situations that may be encountered in actual speech signals.

[0027] The design of the first generator network further includes the following steps: hierarchical feature extraction is performed on the noisy time-series speech signal by using core modules in the Mamba architecture, wherein the state dimension of the Mamba module is set to 512, the time step is 256, and the selective mechanism threshold is 0.3; each layer of convolution operation adopts a 7*7 convolution kernel (step 1, padding 3), and the output channel number is 64, 128 and 256 in turn. Each layer of module gradually enhances the capture ability of different scale time-series features through specific convolution operation and attention mechanism; in the feature extraction process, a residual connection mechanism is introduced to fuse the features at different levels, avoid information loss, and accelerate the gradient propagation in the network training process; in the feature conversion stage, a fully connected layer or other nonlinear transformation layer is used to further process and map the extracted features to generate the denoised speech feature representation.

[0028] The design of the second discriminator network further includes the following steps: a multi-layer convolutional neural network structure is designed to extract features from the input speech signal layer by layer, and high-level semantic features of the speech signal are gradually extracted through sliding of the convolution kernel and down-sampling of the feature map; an activation function is introduced after each layer of convolution operation to increase the non-linear expression ability of the network, so that the discriminator can better learn the complex differences between the real noise-free speech and the generated denoised speech; finally, a probability value is output through the fully connected layer and the output layer, representing the probability that the input speech signal is real noise-free speech, providing feedback signals for the generator.

[0029] The loss function is optimized by considering various loss terms, including:

[0030] Adversarial loss, which prompts the generator to generate realistic denoised speech, and enables the discriminator to accurately distinguish between real samples and generated samples;

[0031] Cycle consistency loss, which ensures that the generated denoised speech can recover the information of the original noisy speech as much as possible after reverse mapping; reconstruction loss, which calculates the difference between the generated denoised speech and the original noise-free speech to ensure that the generated denoised speech is highly consistent with the original noise-free speech in content and details.

[0032] Perceptual loss, which measures the difference between the generated denoised speech and the real noise-free speech in perceptual features from the perspective of human auditory perception, so that the generated speech is more in line with human auditory habits.

[0033] By combining these loss terms by weighting, the total loss = 0.5x adversarial loss + 0.3x cycle consistency loss + 0.1x reconstruction loss + 0.1x perceptual loss; wherein the perceptual loss is calculated based on the Mel-frequency cepstral coefficient (MFCC), the cycle consistency loss adopts the L1 norm, a comprehensive loss function is formed, and the network parameters of the generator and the discriminator are continuously adjusted by using gradient descent and other optimization algorithms, so that the value of the comprehensive loss function gradually decreases, thereby realizing the convergence of the entire network and improving the effect of the time series speech signal filtering and noise reduction.

[0034] Model training and application

[0035] The generated adversarial network based on Mamba is trained using the constructed speech data set, and Adam and other optimization algorithms are used in the training process, the initial learning rate is 0.0002, β1=0.5, β2=0.999; the batch size is set to 32, the number of training rounds (epoch) is 200, and the learning rate is attenuated to 0.5 times the current value every 20 rounds. Continuously adjust the network parameters until the model converges. After training is completed, the noisy time series speech signal is input into the trained generator, and the filtered and denoised speech signal can be obtained.

[0036] As shown in Figure 2 A speech signal filtering and noise reduction method based on a Mamba generated adversarial network, comprising the following steps:

[0037] Step S1, data preparation stage; adopt the combination of public speech data set and real scene noise data set;

[0038] Step S2, model construction; construct a CycleGAN network model based on Mamba architecture, including generator G, generator F and discriminators D1, D2, the core of which realizes unpaired data training by means of the time series modeling capability of Mamba and the cycle consistency loss;

[0039] Step S3, training the model constructed in step S2;

[0040] Step S4, applying the model trained in step S3 for noise reduction.

[0041] The noise reduction method of the present application will be described in detail as follows: Figure 1

[0042] I. Data preparation stage

[0043] In this embodiment, the experimental data adopts the combination of public speech data set and real scene noise data set, specifically including:

[0044] ​Clean speech dataset: Existing clean speech dataset, artificially recorded clean speech (44.1kHz sampling rate, 16bit quantization), and relatively clean speech segments extracted by video-to-speech tools (such as FFmpeg) to build a clean speech library with a size of 5000 (each with a duration of 3-10 seconds).

[0045] Noisy speech dataset: generated by mixing clean speech with public noise libraries (such as DEMAND), including street traffic noise, office noise, industrial noise (loom running sound), and sudden noise. At the same time, various noise environments such as airplane roaring noise, Gaussian noise, and impulse noise are synthesized through python to simulate the existing noise in reality and enhance the robustness of the model. The synthesis formula of noisy speech is:

[0046] X noisy =X clean +α*N

[0047] where X noisy is the synthesized noisy speech signal, X clean is the clean speech, N is the noise signal, and α is the noise intensity coefficient.

[0048] Preprocess the data. The specific process is as follows:

[0049] Resample all speech samples to 16kHz single channel;

[0050] Convert time-domain speech to frequency spectrum using short-time Fourier transform (STFT), where the STFT window length is 25ms, the overlap rate is 50%, the Fourier transform point number is 512, and the generated frequency spectrum dimension is [time step, 257, 1] (frequency bins = 257)

[0051] Normalize the spectral features, map the amplitude value to the [-1, 1] interval, and use it as the model input; the formula is as follows:

[0052]

[0053] where X is the original spectral amplitude, X MIN represents the minimum value in the original spectral amplitude, X MAX represents the maximum value in the original spectral amplitude, and X NORM represents the normalized spectral amplitude value, which is limited to [-1, 1], which helps the model to learn features more stably during training.

[0054] Divide the training set and validation set in the ratio of 8:2 to ensure consistent noise type distribution in the training set and validation set.

[0055] II. Model construction details

[0056] The CycleGAN network model based on the Mamba architecture is constructed, including the generator G, the generator F and the discriminators D1 and D2, and the non-paired data training is realized by means of the time sequence modeling capability of Mamba and the cycle consistency loss, and the specific details are as follows:

[0057] 1. Generator G (noisy speech -> clean speech)

[0058] Input: preprocessed noisy speech spectrogram (dimension [time step, frequency bin, 1])

[0059] Network structure:

[0060] ① The feature extraction layer contains 3 convolutional blocks, each of which is composed of a 7x7 convolutional layer (step 1, padding 3), an InstanceNorm layer and a LeakyReLU activation function, which gradually extracts local spectral features, and the output dimension is [time step, frequency bin, 64].

[0061] ② The Mamba core layer embeds 2 Mamba state machine modules (state dimension 512, expansion factor 2), which captures long time sequence dependencies through selective state mechanism:

[0062] S t =Mamba(S t-1 ,X t )

[0063] Where S t is the state vector at the t-th time step, S t-1 is the state vector at the t-1 time step, and X t is the current input feature;

[0064] ③ Residual connection: the input feature and the Mamba output feature are added element by element, which alleviates the information decay in long sequence modeling.

[0065] X res =X in +X mamba

[0066] Where X in represents the feature input to the Mamba module, X mamba represents the output feature after the Mamba module processing, and X res represents the feature after the residual connection, which is the result of adding the input feature and the Mamba output feature element by element, which helps to retain the original input information and promote the gradient propagation in the network training process.

[0067] ④Feature conversion layer: 3 layers of deconvolution blocks, symmetric to the feature extraction layer, each layer composed of 7x7 transpose convolution layer, InstanceNorm layer, ReLU activation function, gradually recovering the spectral dimension.

[0068] ⑤Output layer: 1x1 convolution layer mapping to single channel, outputting clean speech spectrum through tanh activation function (dimension consistent with input, amplitude value ∈ [-1, 1]. The formula is as follows:

[0069]

[0070] wherein represents the pseudo-clean speech generated by the generator, tanh() represents the hyperbolic tangent activation function, w represents the weight parameter of the generator output layer, X trans represents the features processed by the feature conversion layer, and b represents the bias parameter of the generator output layer.

[0071] 2. Generator F (clean speech → noisy speech)

[0072] Symmetric to the structure of the generator G, the input is the clean speech spectrum, the output is the noisy speech spectrum, and the parameter configuration remains the same to ensure the symmetry of the reverse mapping.

[0073] 3. Discriminator D1 (discriminate the authenticity of clean speech)

[0074] ① Input: real clean speech or "pseudo-clean speech" spectrum output by the generator G. The real clean speech X clean or the "pseudo-clean speech" output by the generator G

[0075] ② Network structure:

[0076] (1) Feature extraction layer: 4 layers of convolution blocks, each layer composed of 4x4 convolution layer (step 2, padding 1), InstanceNorm layer, LeakyReLU activation function (slope 0.2), gradually compressing the feature dimension.

[0077] (2) Mamba discrimination layer: 1 Mamba state machine module, capturing the authenticity difference features of the time dimension.

[0078] (3) Output layer: 1x1 convolution layer + Sigmoid activation function, outputting probability values in the interval [0, 1] (1 representing true, 0 representing generated).

[0079]

[0080] wherein, represents the features processed by the Mamba discrimination layer, W dW1 represents the weight of the output layer of the discriminator D1, b1 represents the bias parameter, and σ() represents the Sigmoid activation function, which maps the processing result to the interval [0, 1]. d D1(X) represents the output result of the discriminator D1, that is, the probability that the input speech is a real clean speech, 1 represents a judgment of being real, and 0 represents a judgment of being a generated pseudo clean speech.

[0081] 4. Discriminator D2 (discriminate whether the noisy speech is real or not)

[0082] The structure is completely consistent with that of the discriminator D1, and the input is a real noisy speech or a "pseudo noisy speech" spectrum graph output by the generator F.

[0083]

[0084] wherein, W1 represents the weight of the output layer of the discriminator D1, b1 represents the bias parameter, and σ() represents the Sigmoid activation function, which maps the processing result to the interval [0, 1]. d W2 represents the weight of the output layer of the discriminator D2, b2 represents the bias parameter, and σ() represents the Sigmoid activation function, which maps the processing result to the interval [0, 1]. d D2(X) represents the output result of the discriminator D2, that is, the probability that the input speech is a real clean speech, 1 represents a judgment of being real, and 0 represents a judgment of being a generated pseudo clean speech.

[0085] 5. Loss function design

[0086] ① Adversarial loss

[0087] ② Cycle consistency loss: forward cycle: L1 distance between reconstructed noisy speech and original noisy speech. Reverse cycle: L1 distance between reconstructed clean speech and original clean speech. Total cycle loss = forward cycle + reverse cycle

[0088] ③ Total loss: L total = L GAN (G, D1) + L GAN (G, D2) + θ * L cyc

[0089] wherein θ is a weight coefficient of the cycle consistency loss, L total represents the total loss, which is a weighted sum of the adversarial loss and the cycle consistency loss, L GAN (G, D1) represents the adversarial loss between the generator G and the discriminator D1, which is used to measure the difference between the pseudo clean speech generated by the generator G and the real clean speech, and the distinguishing ability of the discriminator D1 for the two. GAN (G, D2) represents the adversarial loss between the generator F and the discriminator D2, which is used to measure the difference between the pseudo noisy speech generated by the generator F and the real noisy speech, and the distinguishing ability of the discriminator D2 for the two.cyc denotes the cycle consistency loss.

[0090] Convergence condition: In combination with the objective evaluation index PESQ (Perceptual Evaluation of Speech Quality) of speech quality, a specific threshold is set to better meet the perception requirements of the speech noise reduction effect in actual application. Specifically, when the PESQ score of the generated denoised speech and the real clean speech on the validation set reaches a high score such as 2-3, it is determined that the model converges.

[0091] A speech signal filtering and denoising method based on Mamba generative adversarial network, the specific implementation details mainly include the following steps:

[0092] Step S1: model training, complete the construction and preprocessing of unpaired data set of noisy speech and clean speech, optimize the model parameters through the adversarial training of double generator and double discriminator, the specific steps are as follows:

[0093] Step S1.1: construct mixed noise speech data set, including:

[0094] ①Collect public clean speech library and real scene noise (street, factory, public place, etc.), synthesize Gaussian noise, impulse noise and other artificial noise through Python, form the basic noise library; mix natural noise, artificial noise and combined noise according to the proportion of 3:1:1, and superimpose clean speech to generate noisy speech samples, which do not need to be strictly paired;

[0095] ②Preprocess all samples: resample to 16kHz single channel, convert to spectrogram through STFT, normalize amplitude to [-1, 1] interval, and divide training set and validation set according to the proportion of 8:2.

[0096] Step S1.2: build generator G (noisy→clean) and generator F (clean→noisy), specifically including:

[0097] ①Generator G structure: 3 layers of 7x7 convolutional blocks extract spectral features, embed 2 Mamba state machine modules, combine residual connection to alleviate information loss, and finally recover the spectral dimension through 3 layers of deconvolutional blocks, and output clean speech spectrum through tanh;

[0098] ②Generator F and G are designed symmetrically, the input is clean speech spectrum, the output is noisy speech spectrum, and the parameter configuration remains the same to meet the cycle mapping constraint.

[0099] Step S1.3: build discriminators D1 (judge clean speech authenticity) and D2 (judge noisy speech authenticity), specifically including:

[0100] ①Discriminator D1 adopts 4-layer 4x4 convolutional block to compress features, embeds a Mamba module to capture temporal differences, and finally outputs [0, 1] probability values through 1x1 convolution and Sigmoid;

[0101] ②Discriminator D2 is exactly the same as D1 in structure, and the input is real noisy speech or the "pseudo noisy speech" output by the generator F.

[0102] Step S1.4: Define the comprehensive loss function and train the model.

[0103] Step S2: Speech denoising inference: through the trained generator G, the noisy speech is processed end-to-end, and the specific steps are as follows:

[0104] Step S2.1: Initialize the speech processing array, including the input noisy speech array input_audio[], the denoised speech array output_audio[], and the quality score array pesq_scores[].

[0105] Step S2.2: Load the trained generator G model weight, read the denoised speech data, and convert it into a spectrum graph (16kHz single channel, STFT parameters consistent with training) according to the preprocessing process.

[0106] Step S2.3: Input the preprocessed spectrum graph into the generator G, and obtain the denoised spectrum through forward propagation, which includes: Mamba module dynamically selects temporal features, strengthens effective speech components, and suppresses noise interference; the output layer uses the tanh activation function to constrain the spectrum amplitude to the [-1, 1] interval, ensuring that the output format matches the input.

[0107] Step S2.4: Post-process the denoised spectrum output by the generator: use the inverse short-time Fourier transform (ISTFT) to convert the spectrum graph back to a time-domain speech signal; perform amplitude inverse normalization on the speech signal to restore the original volume range. Step S2.5: Calculate the PESQ score of the denoised speech and store it in pesq_scores[], and store the result in output_audio[]. The PESQ score is improved from the original 1.8 to above 3.2.

[0108] Through the above design, the model can capture the long-term dependence of speech with the help of Mamba in the absence of paired samples, learn the essential mapping of noise and speech by combining the cycle consistency constraint, and achieve efficient denoising in complex noise environments.

Claims

1. A speech signal filtering and denoising method based on Mamba generative adversarial network, characterized in that: The following steps are involved: Step S1, data preparation stage: using a combination of public speech dataset and real scene noise dataset; Step S2: Model construction: Build a CycleGAN network model based on the Mamba architecture, including generators G and F and discriminators D1 and D2. The core uses Mamba's temporal modeling capabilities and cycle consistency loss to achieve unpaired data training; Step S3: training the model constructed in step S2; Step S4: Apply the model trained in step S3 to perform noise reduction.

2. The method for filtering and denoising speech signals based on a Mamba generative adversarial network according to claim 1, wherein: The specific process of the above step S1 includes: Clean speech dataset: This dataset consists of existing clean speech datasets, manually recorded clean speech, and relatively clean speech segments extracted using video-to-speech tools to build a clean speech library. Noisy speech dataset: This dataset is generated by mixing a public noise library with clean speech. Noise types include street traffic noise, daily office noise, industrial noise, and sudden noise. Various noise environments are synthesized using Python to simulate real-world noise and enhance the robustness of the model. The noisy speech synthesis formula is: X noisy =X clean +a*N Among them, X noisy is the synthesized noisy speech signal; X clean is the clean speech; N is the noise signal; α is the noise intensity coefficient.

3. The method for filtering and denoising speech signals based on a Mamba generative adversarial network according to claim 2, wherein: Preprocess the data. The steps of data preprocessing are as follows: Step S11: resample all speech samples to 16kHz mono; Step S12: Convert the time domain speech into a spectrogram using short-time Fourier transform, where the STFT window length is 25 ms, the overlap rate is 50%, the number of Fourier transform points is 512, and the generated spectrogram dimension is [time step, 257, 1], with frequency bins = 257; Step S13: Normalize the spectrum features and map the amplitude values ​​to the interval [-1, 1] as the model input; the formula is as follows: where X NORM represents the normalized spectrum amplitude value, whose range is limited to [-1,1]; X is the original spectrum amplitude; X MIN Indicates the minimum value in the original spectrum amplitude; X MAX Indicates the maximum value in the original spectrum amplitude; Step S14: Divide the training set and the validation set into a ratio of 8:2 to ensure that the noise type distribution in the training set and the validation set is consistent.

4. The method for filtering and denoising speech signals based on a Mamba generative adversarial network according to claim 1, wherein: The specific process of constructing the model in step S2 above includes: Generator G, noisy speech → clean speech; Input: Preprocessed noisy speech spectrogram, dimension is [time steps, frequency bins, 1]; Network structure: The feature extraction layer contains three layers of convolution blocks. Each layer consists of a 7×7 convolution layer, an InstanceNorm layer, and a LeakyReLU activation function. It gradually extracts local spectral features, and the output dimension is [time steps, frequency bins, 64]. The Mamba core layer embeds two Mamba state machine modules to capture long-term timing dependencies through a selective state mechanism: S t =Mamba(S t-1 ,X t ) Among them S t is the state vector of the t-th time step; S t-1 is the state vector of the t-1 time step; X t is the current input feature; Residual connection: adds input features to Mamba output features element by element to alleviate information decay in long sequence modeling; X res =X in +X mamba where X in Represents the features input to the Mamba module; X mamba Represents the output features after processing by the Mamba module; X res Represents the feature after residual connection, which is the result of element-by-element addition of input feature and Mamba output feature; Feature conversion layer: It uses a three-layer deconvolution block, symmetrical to the feature extraction layer. Each layer consists of a 7×7 transposed convolution layer, an InstanceNorm layer, and a ReLU activation function to gradually restore the spectral dimension. Output layer: A 1×1 convolutional layer maps to a single channel and outputs a clean speech spectrum through a tanh activation function. The dimension is consistent with the input and the amplitude value ∈ [-1, 1]. The formula is as follows: in is the pseudo-clean speech; w represents the weight of the output layer; b represents the bias parameter, which is used to perform linear transformation on the feature; X trans Represents the features after being processed by the feature conversion layer.

5. The method for filtering and denoising speech signals based on a Mamba generative adversarial network according to claim 4, wherein: The specific process of constructing the model in step S2 above also includes: Generator F, clean speech → noisy speech; The structure is symmetrical with the generator G. The input is a clean speech spectrogram and the output is a noisy speech spectrogram. The parameter configuration remains consistent to ensure the symmetry of the reverse mapping.

6. The method for filtering and denoising speech signals based on a Mamba generative adversarial network according to claim 5, wherein: The specific process of constructing the model in step S2 above also includes: Discriminator D1, which determines the authenticity of clean speech; Input: real clean speech or "pseudo clean speech" spectrogram output by generator G; real clean speech X clean Or the "pseudo clean speech" output by generator G Network structure: Feature extraction layer: 4 layers of convolution blocks, each layer consists of a 4×4 convolution layer, an InstanceNorm layer, and a LeakyReLU activation function, which gradually compresses the feature dimensions; Mamba discrimination layer: a Mamba state machine module that captures the authenticity difference characteristics of the time series dimension; Output layer: 1×1 convolution layer + Sigmoid activation function, outputting probability values ​​in the interval [0,1], where 1 represents true and 0 represents generated; in, Represents the features after processing by the Mamba discriminant layer; W d represents the weight of the discriminator output layer; b d Represents the bias parameter; σ() represents the Sigmoid activation function, which maps the processing result to the interval [0,1]; D1(X) represents the output result of the discriminator D1, that is, the probability that the input speech is a real clean speech; 1 indicates that it is judged to be real, and 0 indicates that it is judged to be a generated pseudo-clean speech.

7. The method for filtering and denoising speech signals based on a Mamba generative adversarial network according to claim 6, wherein: The specific process of constructing the model in step S2 above also includes: Discriminator D2, which determines the authenticity of noisy speech; The structure is exactly the same as that of the discriminator D1, and the input is the real noisy speech or the "pseudo noisy speech" spectrogram output by the generator F; in, W represents the features after the Mamba discriminant layer processes the noisy speech; d ′ represents the weight of the output layer of the discriminator D2; b d ′ represents the bias parameter; σ() represents the Sigmoid activation function, which maps the processing result to the interval [0,1]; D2(X) represents the output result of the discriminator D2, that is, the probability that the input speech is a real clean speech; 1 represents the judgment as real, and 0 represents the judgment as a generated pseudo clean speech.

8. The method for filtering and denoising speech signals based on a Mamba generative adversarial network according to claim 7, wherein: The above step S3 model training includes: Loss function design, Fighting losses, Cycle consistency loss: Forward loop: reconstruct the L1 distance between the noisy speech and the original noisy speech; reverse loop: reconstruct the L1 distance between the clean speech and the original clean speech; total cycle loss = forward loop + reverse loop; Total loss: L total =L GAN (G,D1)+L GAN (G,D2)+θ*L cyc Where θ is the weight coefficient of cycle consistency loss; L total represents the total loss, which is the weighted sum of adversarial loss and cycle consistency loss; L GAN (G, D1) represents the adversarial loss between the generator G and the discriminator D1, which is used to measure the difference between the pseudo-clean speech generated by the generator G and the real clean speech, as well as the discriminator D1’s ability to distinguish between the two; L GAN (G, D2) represents the adversarial loss between the generator F and the discriminator D2, which is used to measure the difference between the pseudo noisy speech generated by the generator F and the real noisy speech, as well as the discriminator D2’s ability to distinguish between the two; L cyc represents the cycle consistency loss.

9. The method for filtering and denoising speech signals based on a Mamba generative adversarial network according to claim 8, wherein: The above step S3 model training also includes: Convergence conditions: Specific thresholds are set based on objective evaluation indicators of speech quality, and the perceived need for speech noise reduction effects is used to determine model convergence.

10. The method for filtering and denoising speech signals based on a Mamba generative adversarial network according to claim 7, wherein: The speech noise reduction process in step S4 is as follows: the noisy speech is subjected to end-to-end noise reduction processing by the trained generator G. The specific steps are as follows: Step S41: Initialize the speech processing array, including the input noisy speech array input_audio[], the noise-reduced speech array output_audio[], and the quality score array pesq_scores[]; Step S42: Load the trained generator G model weights, read the speech data to be denoised, and convert it into a spectrogram according to the preprocessing process; Step S43: The pre-processed spectrogram is input into the generator G, and the denoised spectrum is obtained through forward propagation, which specifically includes: the Mamba module dynamically filters the temporal features, strengthens the effective components of the speech, and suppresses noise interference; The output layer constrains the spectrum amplitude to the range [-1, 1] through the tanh activation function to ensure that the output format matches the input; Step S44: Post-processing the denoised spectrum output by the generator: using inverse short-time Fourier transform to convert the spectrum back into a time-domain speech signal; performing amplitude denormalization on the speech signal to restore the original volume range; Step S45: Calculate the PESQ score of the denoised speech, store it in pesq_scores[], and store the result in output_audio[].

Images