Self-supervised speech enhancement method based on de-statistical instance normalization
By introducing destatistical instance normalization technology into the self-supervised speech enhancement model, the distribution offset problem caused by noise changes is solved, the prediction accuracy and noise removal effect of the speech enhancement model are improved, and high-quality speech enhancement is achieved.
Patent Information
- Application Number
- CN202411391984.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-08
- Publication Date
- 2025-01-10
AI Technical Summary
The performance of existing speech enhancement algorithms deteriorates when noise changes. The self-supervised learning model has a distribution offset due to statistical properties such as mean and variance over time, which affects the model prediction accuracy and noise removal effect.
Using a self-supervised speech enhancement method based on destatistical instance normalization, through destatistical instance normalization between the feature extraction module and the speech enhancement module, the non-stationary information in the sequence is removed, and the data distribution is converted into the mean-center distribution, which improves the Model prediction accuracy and noise reduction effect.
It effectively improves the noise reduction effect of the speech enhancement model, improves the speech quality and intelligibility, and achieves higher speech perception quality and intelligibility.
Smart Images
Figure CN119296563A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of speech signal processing, and in particular to a self-supervised speech enhancement method based on de-statistical instance normalization. Background Art
[0002] Speech enhancement is an important research branch in the field of speech signal processing. Its main purpose is to eliminate possible interference components in speech signals and restore clean target speech signals, thereby improving speech quality and intelligibility. Common speech enhancement algorithms can be divided into two categories: traditional speech enhancement algorithms and speech enhancement algorithms based on deep learning.
[0003] Most traditional speech enhancement algorithms are based on digital signal processing technology, and mainly use the ideas of filtering noise reduction and noise estimation to filter out noise, such as spectral subtraction, filtering, spectral estimation algorithm based on minimum mean square error, and subspace-based algorithm. They are unsupervised speech enhancement methods that are highly dependent on the prediction of noise distribution laws and can achieve good speech enhancement effects under stable noise conditions. However, in real life, most noise is non-stationary noise. When the environment changes, the performance of traditional speech enhancement algorithms decreases significantly, and good speech enhancement effects cannot be achieved.
[0004] In order to overcome the problems existing in traditional speech enhancement algorithms, deep learning is used for speech enhancement and supervised learning training is performed on speech, which has made outstanding contributions to practical applications. The main methods are: 1. Using CNN (Convolutional Neural Networks) and DNN (Deep Neural Network) for speech enhancement; 2. Using RNN (Recycle Neural Networks) to predict and judge the current frame using the information of the previous frame; 3. Using LSTM (Long Short Term Memory) to replace the hidden neurons of RNN, and using LSTM-RNN to predict the log spectrum and IRM of clean speech respectively to achieve speech enhancement; 4. Speech enhancement methods based on GAN (Generative Adversarial Network), or introducing time-frequency masks in the GAN model, using time-frequency masks to represent the error between clean speech and noisy speech, and using regularization functions to calculate the error between the two; or using CNN to build a generator to make the speech generated by the generator more realistic; or based on the Transformer GAN model, the generator can capture long-distance dependencies and local features in the time and frequency domains. However, these supervised speech enhancement methods rely on a large amount of labeled data, and collecting this data is time-consuming and unrealistic.
[0005] Self-supervised learning methods aim to extract meaningful latent representations from a large amount of unlabeled data and use these latent representations as inputs for downstream tasks. The improvement of downstream task performance by self-supervised learning models has been verified in various fields. At present, common speech self-supervised learning models include CPC (Contrastive Predictive Coding), wav2vec, HuBERT, and WavLM. When applying the characteristics of self-supervised learning models to speech enhancement, the model usually uses linear layers for feature mapping to predict sequence data to meet dimensionality requirements. However, statistical properties such as mean and variance in the sequence will change over time, that is, the sequence data will encounter distribution shift problems, resulting in inaccurate model predictions and incomplete denoising of some enhanced speech. Summary of the invention
[0006] The problem to be solved by the present invention is to provide a self-supervised speech enhancement method based on de-statistical instance normalization, which is used to solve the distribution shift problem caused by the change of statistical properties such as mean and variance over time, thereby improving the accuracy of model prediction, enhancing the noise reduction effect of the model, and making the enhanced speech have higher perceptual quality and intelligibility.
[0007] The present invention adopts the following technical solution: a self-supervised speech enhancement method based on de-statistical instance normalization, including a training phase and a testing phase.
[0008] The training phase consists of the following steps:
[0009] Step S1, obtaining training corpus, the training corpus including original noisy speech and clean speech;
[0010] Step S2, preprocessing the original noisy speech and clean speech in the training corpus to obtain the noisy speech x and clean speech s with fixed sampling rate and length;
[0011] Step S3: construct a speech enhancement model, input the noisy speech x and the clean speech s obtained after preprocessing into the speech enhancement model for training, and obtain the enhanced speech Will enhance the voice Together with clean speech s, it is used to assess the loss;
[0012] Step S4: during the training of the speech enhancement model, the hyperparameters of the enhancement model are set so as to minimize the target loss function until the set number of iterations is reached, and a trained self-supervised speech enhancement model based on de-statistical instance normalization is obtained;
[0013] The testing phase includes the following steps:
[0014] Step S5: Select original noisy speech from different noise sources as the corpus to be enhanced, pre-process the original noisy speech in the corpus to be enhanced, and obtain a noisy speech x of fixed length. ′ ;
[0015] Step S6: convert the noisy speech x ′ Input the self-supervised speech enhancement model trained in step S4 for testing to obtain enhanced speech
[0016] Preferably, in step S2, the training corpus includes corpora of multiple speakers, and the original noisy speech and the original clean speech in the training corpus are preprocessed. The specific steps are: the sampling rate of the training corpus is set to 16kHz, each segment of the corpus is randomly sampled into fixed data with a length of 20480 (128 frames × 160 hop lengths), and the training corpus is randomly divided into a training set and a validation set according to a proportion.
[0017] Preferably, in step S3, the speech enhancement model includes a feature extraction module and a speech enhancement module; the feature extraction module is used to extract self-supervised learning features of speech, and the speech enhancement module processes the extracted acoustic features to obtain enhanced predictions; further, statistical instance normalization is proposed between the feature extraction module and the speech enhancement module to solve the distribution shift problem caused by the change of statistical properties such as mean and variance in sequence data over time, thereby improving the accuracy of model prediction.
[0018] Among them, the feature extraction module uses WavLM as the speech self-supervised learning model. WavLM includes a 7-layer convolutional neural network encoder and a 12-layer Transformer encoder. The feature extraction method is as follows: The convolutional neural network encoder first extracts the local features x of the speech from the noisy speech x l , then the local feature x l The output of the Transformer encoder is sent to extract the deeper global features of speech, thereby obtaining the self-supervised learning feature x output by the feature extraction module. ssl .
[0019] The speech enhancement module consists of 2 linear layers, 2 BLSTM models and a Sigmoid activation guidance function. The linear layer is used to transform the dimension of the input features to meet the needs of the model. The BLSTM model, as the main framework of the speech enhancement module, captures the long-term dependencies in the input sequence to provide a more comprehensive representation and has good performance.
[0020] The Sigmoid activation guidance function is used to map the output of the BLSTM model to the probability range of [0,1] to generate enhanced predictions. The Sigmoid activation guidance function is expressed as:
[0021]
[0022] Among them, a represents the input of the activation guidance layer, represents a hyperparameter, which is set to 0.1, and exp represents an exponential function with the natural constant e as the base.
[0023] Among them, statistical instance normalization first normalizes the input sequence and removes non-stationary information in the sequence, such as mean, variance and learnable affine parameters, to solve the distribution shift problem caused by the change of the above statistical properties over time; then the original data distribution is converted into a mean-centered distribution, so that the distribution difference between different instances is reduced, so as to more accurately predict the local dynamics within the sequence, improve the accuracy of model prediction, and thus enhance the denoising effect of the model.
[0024] Preferably, in step S4, the training process is specifically as follows:
[0025] Step S4.1: The feature extraction module extracts features from the noisy speech x to obtain the extracted self-supervised learning features x ssl ; At the same time, the noisy speech x is short-time Fourier transformed to obtain the spectrum feature, and then log1p normalized to obtain the acoustic feature x log1p ;
[0026] Step S4.2: Extract the self-supervised learning feature x ssl With acoustic characteristics x log1p Perform splicing to obtain the mixed acoustic feature Ω(x);
[0027] Step S4.3, performing de-statistical instance normalization on the mixed acoustic feature Ω(x) to obtain the acoustic feature Ω′(x);
[0028] Step S4.4: Input the acoustic feature Ω′(x) into the speech enhancement module to obtain the enhanced prediction
[0029] Step S4.5: Enhance the prediction With acoustic characteristics x log1p Multiply to perform feature fusion and obtain enhanced acoustic features
[0030] Step S4.6: Acoustic features Perform inverse short-time Fourier transform to restore the original waveform and obtain enhanced speech
[0031] Step S4.7: Enhance the speech Used together with the clean speech s to evaluate the loss, where the loss function mainly calculates the L1 loss of the spectrum;
[0032] Step S4.8, repeat the process from step S4.1 to step S4.7 to train the speech enhancement model until the set number of iterations is reached, and a trained self-supervised speech enhancement model based on de-statistical instance normalization is obtained.
[0033] Preferably, in step S4, the loss function is expressed as:
[0034]
[0035] Among them, loss spec Represents the L1 loss between enhanced speech and clean speech, which calculates the average distance between the enhanced speech spectrum and the clean speech spectrum; represents the real part of the enhanced speech spectrum, S re represents the real part of the pure speech spectrum, represents the imaginary part of the enhanced speech spectrum, S im Represents the imaginary part of the pure speech spectrum, and n represents the number of samples in the batch.
[0036] Preferably, in step S6, the testing process is specifically as follows:
[0037] Step S6.1, preprocessing the noisy speech in the to-be-enhanced corpus to obtain a noisy speech x′ with a sampling rate of 16 kHz and a fixed length;
[0038] Step S6.2: Input the noisy speech x′ into the trained self-supervised speech enhancement model based on de-statistical instance normalization, and use the feature extraction module to obtain the self-supervised learning feature x′ ssl At the same time, the noisy speech x′ is short-time Fourier transformed to obtain the spectrum feature, and then log1p normalized to obtain the acoustic feature x′ log1p ;
[0039] The self-supervised learning feature x′ ssl and the acoustic feature x′ log1p Concatenate to obtain a mixed acoustic feature Ω(x′); perform statistical instance normalization on the mixed acoustic feature Ω(x′) to obtain an acoustic feature Ω′(x′);
[0040] Input the acoustic feature Ω′(x′) into the speech enhancement module to obtain the enhanced prediction Will enhance prediction and the acoustic feature x′ log1p Multiply to get enhanced acoustic features
[0041] Step S6.3: Enhanced acoustic features Perform inverse short-time Fourier transform to restore the original waveform and obtain the final enhanced speech
[0042] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:
[0043] 1. The method of the present invention proposes statistical instance normalization between the feature extraction module and the speech enhancement module, which removes non-stationary information in the sequence while performing instance normalization on the input sequence, and solves the distribution shift problem caused by the change of statistical properties such as mean and variance over time; in addition, the original data distribution is converted into a mean-centered distribution, so that the distribution difference between different instances is reduced, so as to more accurately predict the local dynamics in the sequence and improve the accuracy of model prediction.
[0044] 2. The self-supervised speech enhancement method of the present invention can effectively improve the noise reduction effect of the self-supervised speech enhancement model based on de-statistical instance normalization, improve the phenomenon of unsaturated speech quality, thereby effectively improving the performance of speech enhancement, obtaining higher speech perception quality and intelligibility, and achieving high-quality speech enhancement. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a flowchart of the training phase of the self-supervised speech enhancement method based on de-statistical instance normalization of the present invention;
[0046] Figure 2 It is a network structure diagram of a feature extraction module in a speech enhancement model of the present invention;
[0047] Figure 3 It is a network structure diagram of de-statistical instance normalization in the speech enhancement model of the present invention;
[0048] Figure 4 It is a network structure diagram of the speech enhancement module in the speech enhancement model of the present invention. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely explain the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. At the same time, for the step numbers in the embodiments of the present invention, they are only set for the convenience of explanation, and the order between the steps is not limited. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.
[0050] In one embodiment of the present invention, a self-supervised speech enhancement method based on de-statistical instance normalization includes a training phase and a testing phase. The training phase is used to obtain the parameters and loss function required for the speech enhancement model, and the testing phase is used to achieve noisy speech enhancement.
[0051] In the training phase, the original noisy speech and clean speech are first preprocessed to obtain noisy speech and clean speech with a sampling rate of 16kHz and a fixed length. Then the noisy speech and clean speech are sent to the speech enhancement model to learn the mapping relationship between the noisy speech and the clean speech. After the training phase, the trained model is saved. In the testing phase, the noisy speech is first preprocessed and then sent to the trained model for speech enhancement, and finally the enhanced speech is output.
[0052] like Figure 1 As shown, the training phase includes the following steps:
[0053] Step S1: Obtain a large amount of training corpus.
[0054] In this embodiment, the training corpus comes from speech data in the VoiceBank-DEMAND corpus, which contains pure speech and its corresponding pre-mixed noisy speech.
[0055] During the training phase, 11,572 speech samples from 28 speakers (14 males and 14 females) were selected as clean speech with a sampling frequency of 48 kHz. During the testing phase, 824 speech samples from 2 speakers (1 male and 1 female) were selected as clean speech with a sampling frequency of 48 kHz. During the training phase, 11,572 speech samples from 28 speakers (14 males and 14 females) were selected as noisy speech samples, and 824 speech samples from 2 speakers (1 male and 1 female) were selected as noisy speech samples.
[0056] The noisy speech in the training phase was a mixture of clean speech and 10 types of noise at signal-to-noise ratios of 0dB, 5dB, 10dB, and 15dB, 8 of which were from the DEMAND dataset and the other 2 were self-made noises. The noisy speech in the test phase was a mixture of clean speech and 5 other noises from the DEMAND dataset at signal-to-noise ratios of 2.5dB, 7.5dB, 12.5dB, and 17.5dB.
[0057] Step S2: The noisy speech dataset and the clean speech dataset are resampled to 16kHz. The training corpus contains 11,572 pairs of clean speech and noisy speech, and the test corpus contains 824 pairs of clean speech and noisy speech. Each corpus is randomly sampled into fixed data with a length of 20,480 (128 frames × 160 hop length).
[0058] In this embodiment, the training corpus is randomly divided into a training set and a validation set at a ratio of 92% and 8%.
[0059] Step S3: Input the noisy speech x and the clean speech s obtained in step S2 into the speech enhancement model for training.
[0060] First, the feature extraction module extracts features from the noisy speech x and obtains the self-supervised learning feature x ssl ; At the same time, perform short-time Fourier transform on the noisy speech x and normalize it again by log1p to obtain the acoustic feature x log1p ; Extract the self-supervised learning features x ssl With acoustic characteristics x log1p Perform splicing to obtain the mixed acoustic feature Ω(x);
[0061] Next, the mixed acoustic feature Ω(x) is subjected to de-statistical instance normalization, which removes the non-stationary information in the sequence while performing instance normalization on the input sequence, and then converts the original data distribution into a mean-centered distribution to obtain the acoustic feature Ω′(x);
[0062] Then, the acoustic feature Ω′(x) is input into the speech enhancement module to obtain the enhanced prediction Will enhance prediction With acoustic characteristics x log1p Multiply to perform feature fusion and obtain enhanced acoustic features
[0063] Finally, the acoustic features Perform inverse short-time Fourier transform to restore the original waveform and obtain enhanced speech
[0064] The speech enhancement model mainly includes a feature extraction module and a speech enhancement module. De-statistical instance normalization is proposed between the feature extraction module and the speech enhancement module:
[0065] (1) In the feature extraction module, the noisy speech x first passes through the convolutional neural network encoder to obtain the local features x of the speech l , then the local feature x l The output of the Transformer encoder is sent to extract the deeper global features of speech, thereby obtaining the self-supervised learning feature x output by the feature extraction module. ssl , whose structure is as follows Figure 2 shown.
[0066] In this embodiment, the feature extraction module selects WavLM as the speech self-supervised learning model. WavLM is mainly composed of a 7-layer convolutional neural network encoder and a 12-layer Transformer encoder.
[0067] (2) In the de-statistical instance normalization, the input sequence is first instance normalized, and the non-stationary information in the sequence, such as the mean, variance, and learnable affine parameters, is removed, thereby solving the distribution shift problem caused by the change of the above statistical properties over time; then the original data distribution is converted into a mean-centered distribution, so that the distribution difference between different instances is reduced, so as to more accurately predict the local dynamics in the sequence, improve the accuracy of the model prediction, and thus improve the denoising effect of the model. Its structure is as follows: Figure 3 shown.
[0068] (3) In the speech enhancement module, the acoustic feature Ω′(x) is first input into the linear layer in the speech enhancement module to transform the dimension of the acoustic feature, and then input into the BLSTM model to obtain a more comprehensive acoustic feature representation, and then input into the linear layer again to obtain the output of the specified dimension, and finally the enhanced prediction is obtained through the Sigmoid activation guidance function. Its structure is as Figure 4 shown.
[0069] Preferably, the speech enhancement module of this embodiment is composed of 2 linear layers, 2 layers of BLSTM models and Sigmoid activation guidance function, wherein the linear layer is used to transform the dimension of the input feature to meet the needs of the model; the BLSTM model is the main framework of the speech enhancement module, the purpose is to capture the long-term dependencies in the input sequence to obtain a more comprehensive acoustic feature representation, and has good performance; the Sigmoid activation guidance function is used to map the output of the BLSTM model to a probability range of [0,1], thereby generating an enhanced prediction;
[0070] In this embodiment, the Sigmoid activation guidance function is expressed as:
[0071]
[0072] Among them, a represents the input of the activation guidance layer, represents a hyperparameter, which is set to 0.1, and exp represents an exponential function with the natural constant e as the base.
[0073] Step S4: The enhanced speech obtained in step S3 It is used together with the clean speech s to evaluate the loss, where the loss function mainly calculates the L1 loss of the spectrum;
[0074] The loss function is expressed as:
[0075]
[0076] Among them, loss specRepresents the L1 loss between enhanced speech and clean speech, which calculates the average distance between the enhanced speech spectrum and the clean speech spectrum; represents the real part of the enhanced speech spectrum, S re represents the real part of the pure speech spectrum, represents the imaginary part of the enhanced speech spectrum, S im Represents the imaginary part of the pure speech spectrum, and n represents the number of samples in the batch.
[0077] Step S5: Repeat steps S3-S4 to enhance the speech through continuous iterative training. The difference between the trained speech s and the pure speech s becomes smaller and smaller, that is, the loss function of the speech enhancement model is as small as possible, until the set number of iterations is reached, thereby obtaining a trained speech enhancement model.
[0078] Due to different specific settings of the neural network and different performance of the experimental equipment, the number of iteration rounds set is also correspondingly different. In this embodiment, the number of iteration rounds is set to 100 rounds.
[0079] The testing phase includes the following steps:
[0080] Step S6: Select original noisy speech with different noises as the corpus to be enhanced, pre-process the original noisy speech in the corpus to be enhanced, and obtain a noisy speech x with a sampling rate of 16kHz and a fixed length. ′ ;
[0081] Step S7: The noisy speech x obtained after the above preprocessing ′ Input the speech enhancement model trained in step S5 to obtain enhanced speech
[0082] In summary, the present invention is based on a self-supervised speech enhancement method based on statistical instance normalization. The feature extraction module uses the self-supervised learning model WavLM, and the speech enhancement module uses BLSTM as the main framework. A statistical instance normalization is proposed between the feature extraction module and the speech enhancement module to solve the distribution shift problem caused by the change of statistical attributes such as mean and variance in sequence data over time, improve the accuracy of model prediction, and then enhance the noise reduction effect of the model, so that the enhanced speech has higher speech perception quality and intelligibility, and achieve high-quality speech enhancement.
[0083] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A self-supervised speech enhancement method based on de-statistical instance normalization, characterized in that: It includes training phase and testing phase; The training phase includes the following steps: Step S1, obtaining training corpus, the training corpus including original noisy speech and clean speech; Step S2, preprocessing the original noisy speech and clean speech in the training corpus to obtain the noisy speech x and clean speech s with fixed sampling rate and length; Step S3: construct a speech enhancement model, input the noisy speech x and the clean speech s obtained after preprocessing into the speech enhancement model for training, and obtain the enhanced speech Will enhance the voice Together with clean speech s, it is used to assess the loss; Step S4, training the speech enhancement model, setting the hyperparameters of the enhancement model, minimizing the target loss function, until the set number of iterations is reached, and obtaining a trained self-supervised speech enhancement model based on de-statistical instance normalization; The testing phase includes the following steps: Step S5: Select original noisy speech from different noise sources as the corpus to be enhanced, pre-process the original noisy speech in the corpus to be enhanced, and obtain a noisy speech x of fixed length. ′ ; Step S6: convert the noisy speech x ′ Input the self-supervised speech enhancement model trained in step S4 for testing to obtain enhanced speech 2. The self-supervised speech enhancement method based on de-statistical instance normalization according to claim 1, characterized in that: The training corpus includes corpora of multiple speakers; In step S2, the original noisy speech and clean speech in the training corpus are preprocessed. The specific method is: the sampling rate of the training corpus is set to 16kHz, each segment of the corpus is randomly sampled into a length of 20480, represented as fixed data of 128 frames × 160 jumps, and the training corpus is randomly divided according to the proportion as a training set and a validation set.
3. The self-supervised speech enhancement method based on de-statistical instance normalization according to claim 1, characterized in that: In step S3, the speech enhancement model includes: a feature extraction module and a speech enhancement module; The feature extraction module is used to extract self-supervised learning features of speech, and the speech enhancement module processes the extracted acoustic features to obtain enhanced predictions; De-statistical instance normalization is performed between the feature extraction module and the speech enhancement module to solve the distribution shift problem caused by the change of mean and variance statistical properties in sequence data over time and improve the accuracy of model prediction.
4. The method for self-supervised speech enhancement based on de-statistical instance normalization according to claim 3, characterized in that: The feature extraction module uses WavLM as a speech self-supervised learning model, including a 7-layer convolutional neural network encoder and a 12-layer Transformer encoder; The feature extraction module extracts the self-supervised learning features of speech, and extracts the local features x of speech from the noisy speech x through the convolutional neural network encoder. l , the local feature x l The Transformer encoder extracts deeper global speech features to obtain the self-supervised learning feature x output by the feature extraction module. ssl .
5. The method for self-supervised speech enhancement based on de-statistical instance normalization according to claim 3, characterized in that: The speech enhancement module includes: 2 linear layers, 2 layers of BLSTM models and a Sigmoid activation guidance function; The linear layer is used to transform the dimension of the input features, the BLSTM model is used to capture the long-term dependencies in the input sequence, and the Sigmoid activation guidance function maps the output of the BLSTM model to a probability range of [0,1] to generate an enhanced prediction; Sigmoid activation guidance function, expressed as: Among them, a represents the input of the activation guidance layer, represents a hyperparameter, and exp represents an exponential function with the natural constant e as the base.
6. The method for self-supervised speech enhancement based on de-statistical instance normalization according to claim 3, characterized in that: The method of de-statisticing instance normalization is as follows: Step S3.1, perform instance normalization on the input sequence and remove non-stationary information in the sequence, including mean, variance and learnable affine parameter information, to solve the distribution shift problem caused by the change of statistical attributes over time; Step S3.2: Convert the original data distribution into a mean-centered distribution so that the distribution differences between different instances are reduced and the local dynamics within the sequence are accurately predicted.
7. The method for self-supervised speech enhancement based on de-statistical instance normalization according to claim 3, characterized in that: In step S4, the speech enhancement model is trained as follows: Step S4.1: Extract features of noisy speech x through the feature extraction module to obtain the extracted self-supervised learning feature x ssl ; At the same time, the noisy speech x is short-time Fourier transformed to obtain the spectral features, and the spectral features are log1p normalized again to obtain the acoustic features x log1p ; Step S4.2: Extract the self-supervised learning feature x ssl With acoustic characteristics x log1p Perform splicing to obtain the mixed acoustic feature Ω(x); Step S4.3, performing de-statistical instance normalization on the mixed acoustic feature Ω(x) to obtain the acoustic feature Ω′(x); Step S4.4: Input the acoustic feature Ω′(x) into the speech enhancement module to obtain the enhanced prediction Step S4.5: Enhance the prediction With acoustic characteristics x log1p Multiply and fuse the features to get the enhanced acoustic features Step S4.6: Acoustic features Perform inverse short-time Fourier transform to restore the original waveform and obtain enhanced speech Step S4.7: Enhance the speech Together with the clean speech s, the loss is evaluated through the loss function; Step S4.8, repeat the process from step S4.1 to step S4.7 to train the speech enhancement model until the set number of iterations is reached, and a trained self-supervised speech enhancement model based on de-statistical instance normalization is obtained.
8. The method for self-supervised speech enhancement based on de-statistical instance normalization according to claim 7, characterized in that: In step S4.7, the loss function is used to calculate the L1 loss of the spectrum, which is expressed as: Among them, loss spec represents the L1 loss between enhanced speech and clean speech, and calculates the average distance between the enhanced speech spectrum and the clean speech spectrum; represents the real part of the enhanced speech spectrum, S re represents the real part of the pure speech spectrum, represents the imaginary part of the enhanced speech spectrum, S im Represents the imaginary part of the pure speech spectrum, and n represents the number of samples in the batch.
9. The method for self-supervised speech enhancement based on de-statistical instance normalization according to claim 7, characterized in that: In step S6, the noisy speech x ′ Input the trained self-supervised speech enhancement model for testing as follows: Step S6.1, preprocessing the noisy speech in the to-be-enhanced corpus to obtain a noisy speech x′ with a sampling rate of 16 kHz and a fixed length; Step S6.2: Input the noisy speech x′ into the trained self-supervised speech enhancement model based on de-statistical instance normalization, and use the feature extraction module to obtain the self-supervised learning feature x′ ssl At the same time, the noisy speech x′ is short-time Fourier transformed to obtain the spectrum feature, and then the spectrum feature is log1p normalized to obtain the acoustic feature x′ log1p ; The self-supervised learning feature x′ ssl and the acoustic feature x′ log1p Concatenate and obtain the mixed acoustic feature Ω(x′); The mixed acoustic feature Ω(x′) is subjected to statistical instance normalization to obtain the acoustic feature Ω′(x′), which is then input into the speech enhancement module to obtain the enhanced prediction Will enhance prediction and the acoustic feature x′ log1p Multiply them together to get the enhanced acoustic features Step S6.3: Enhanced acoustic features Perform inverse short-time Fourier transform to restore the original waveform and obtain the final enhanced speech