A speech denoising method and device based on deep compressed sensing

Abstract

This invention relates to a speech denoising method and device based on deep compressed sensing, comprising the following steps: adding Gaussian white noise to a clean speech signal to obtain a noisy speech signal; performing time normalization and frame segmentation on both the clean and noisy speech signals; extracting perceptual features from both the clean and noisy speech signals to obtain perceptual features of the clean speech and the noisy speech; training the GAN using the perceptual features of the noisy speech as input to a generative adversarial network (GAN) and the perceptual features of the clean speech as the objective function to obtain perceptual features for generating clean speech; and reconstructing the generated clean speech signal using the OMP reconstruction algorithm based on compressed sensing. This invention, through deep compressed sensing, is suitable for nonlinear noise analysis and processing, improving the denoising effect of noisy speech signals.

Description

Technical Field

[0001] This invention belongs to the field of speech denoising technology, and specifically relates to a speech denoising method based on deep compressed sensing. Background Technology

[0002] Speech is a uniquely human function, a vital tool for acquiring information, and an indispensable means of communication. Speech denoising, also known as speech enhancement, primarily targets audio containing human voices, aiming to remove background noise and enhance the intelligibility of the voice. Traditional speech denoising methods mainly include spectral subtraction and Wiener filtering, which can remove some simple noise but are ineffective at handling nonlinear noise. Therefore, deep learning has been introduced for speech denoising, as it can effectively address nonlinear problems. Summary of the Invention

[0003] The purpose of this invention is to provide a speech denoising method and device based on deep compressed sensing, which can remove nonlinear noise by using compressed sensing and deep learning, and can reduce the dimensionality of the signal by compressed sensing to speed up the denoising speed.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] A speech denoising method based on deep compressed sensing includes the following steps:

[0006] Signal preprocessing: Gaussian white noise is added to the clean speech signal X to obtain a noisy speech signal. The clean speech signal X and the noisy speech signal are subjected to duration normalization and frame segmentation. Then, endpoint detection processing is performed on the clean speech signal X.

[0007] Signal compression: The compressed sensing method is used to extract perceptual features from the preprocessed clean speech signal X and the noisy speech signal to obtain the perceptual features Y1 of the clean speech and the perceptual features Y2 of the noisy speech.

[0008] Model training: A generative adversarial network (GAN) based on a deep learning model is used as the denoising model. The perceptual features Y2 of the noisy speech are used as the input of the GAN, and the perceptual features Y1 of the clean speech are used as the objective function. The GAN is trained by iterating and training repeatedly to obtain the perceptual features Y′ of the generated clean speech.

[0009] Signal reconstruction: The OMP reconstruction algorithm of compressed sensing is used to reconstruct the perceptual features Y′ of the generated clean speech, so as to obtain the generated clean speech signal X′.

[0010] In the signal reconstruction process, the perceptual features Y′ that generate clean speech are first processed by frame segmentation, and the signal is reconstructed frame by frame. Then, the reconstructed frame signals are spliced ​​together to obtain the generated clean speech signal X′.

[0011] Preferably, the voice duration normalization includes the following steps:

[0012] Set the lengths of the clean speech signal X and the noisy speech signal to integer multiples of the signal frame length.

[0013] Preferably, the endpoint detection includes the following steps:

[0014] A two-level discrimination method with dual thresholds based on short-time energy and zero-crossing rate is used to perform endpoint detection on clean speech signal X, and to distinguish between speech segments and non-speech segments in clean speech signal X.

[0015] Preferably, the perceptual feature extraction includes the following steps:

[0016] A clean speech signal X of N×1 dimensions is perceptually measured on an M×N dimensional random Gaussian matrix Φ, such that Y1 = ΦX, to obtain the perceptual features Y1 of the clean speech of M×1 dimensions, where the dimension M of the perceptual features Y1 of the clean speech is less than the dimension N of the clean speech signal X.

[0017] The Gaussian random matrix used for perceptual feature extraction of noisy speech signal is the same as the Gaussian matrix Φ used for perceptual feature extraction of clean speech signal X, and the dimension of noisy speech signal is the same as the dimension N of clean speech signal X. The perceptual feature Y2 of noisy speech signal is obtained through perceptual feature extraction.

[0018] Preferably, the perceptual feature extraction further includes the following steps:

[0019] First, perceptual features are extracted from each frame of the clean speech signal X and the noisy speech signal. Then, the perceptual features of each frame are spliced ​​together to obtain the perceptual features Y1 of the clean speech signal and the perceptual features Y2 of the noisy speech signal.

[0020] Preferably, the generative adversarial network includes a generator network G and a discriminator network D. The generator network G is used to denoise the perceptual features Y2 of the noisy speech to obtain the perceptual features Y′ of the generated clean speech. The discriminator network D is used to discriminate whether the generated clean speech signal X′ is consistent with the clean speech signal X.

[0021] Preferably, training the generative adversarial network includes the following steps:

[0022] The perceptual features Y2 of noisy speech are input into the generator network G, and the perceptual features Y1 of clean speech are used as the objective function. The output is the perceptual features Y′ of the generated clean speech. The perceptual features Y′ of the generated clean speech are reconstructed using the OMP algorithm of compressed sensing to obtain the generated clean speech signal X′. The speechless segments of the clean speech signal X and the generated clean speech X′ are extracted. The speechless segments of the clean speech X are used as the real data of the discrimination network D. The discrimination network D identifies the speechless segments of the generated clean speech signal X′. After repeated iterations and continuous training, the discrimination network D reaches Nash equilibrium. The perceptual features Y′ of the clean speech are generated through the generator network G. The mapping relationship between the perceptual features Y1 of noisy speech and the perceptual features Y2 of clean speech is established.

[0023] Preferably, the signal reconstruction includes the following steps:

[0024] The DCT matrix Ψ is selected as the sparse representation matrix to obtain the sensing matrix A=ΦΨ. According to Y′=A×theta and Y′=Ψ×theta, the least squares method is used to solve for theta, and then the pure speech signal X′ is reconstructed.

[0025] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, performs steps in a speech denoising method based on deep compressed sensing.

[0026] According to another aspect of the present invention, a speech denoising device based on deep compression sensing is also provided, comprising:

[0027] Memory, used to store software applications.

[0028] A processor for executing the software application, wherein each program of the software application correspondingly performs steps in a deep compressed sensing-based speech denoising method.

[0029] The beneficial effects of this invention are:

[0030] 1. The deep compressed sensing method used in this invention is more suitable for nonlinear noise analysis and processing. Through this method, the denoising effect of generating a clean speech signal X′ is improved.

[0031] 2. The endpoint detection method used in this invention distinguishes between silent segments and spoken segments. Combined with the deep learning process for denoising silent segments, it is applied to spoken segments to achieve denoising of spoken segments. Furthermore, in the generative adversarial network, the discriminator only needs to discriminate the denoising effect of silent segments.

[0032] 3. This invention uses compressed sensing, which reduces the data dimensionality by compressing the signal and then denoising it. That is, N×1 dimensional data is compressed into M×1 dimensional data, where N>>M, thereby improving the efficiency of the algorithm. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the process structure of the present invention. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0035] like Figure 1 As shown, the present invention provides a speech denoising method based on deep compressed sensing, comprising the following steps:

[0036] S1. Signal preprocessing: Gaussian white noise is added to the clean speech signal X to obtain a noisy speech signal. The clean speech signal X and the noisy speech signal are subjected to duration normalization and frame segmentation. Then, endpoint detection processing is performed on the clean speech signal X.

[0037] Specifically, the clean speech signal X and the noisy speech signal are divided into frames, with the frame length set to 128. At the same time, the duration of the speech signal is regularized to an integer multiple of the frame length of 128.

[0038] S2. Signal Compression: Using compressed sensing, perceptual features are extracted from the preprocessed clean speech signal X and the noisy speech signal to obtain perceptual features Y1 of the clean speech and perceptual features Y2 of the noisy speech.

[0039] S3. Model Training: A generative adversarial network (GAN) based on deep learning is used as the denoising model. The perceptual features Y2 of the noisy speech are used as the input to the GAN, and the perceptual features Y1 of the clean speech are used as the objective function. The GAN is trained through repeated iterations and multiple training sessions to obtain the perceptual features Y′ of the generated clean speech.

[0040] S4. Signal Reconstruction: Using the OMP reconstruction algorithm of compressed sensing, the perceptual features Y′ of the generated clean speech are reconstructed to obtain the generated clean speech signal X′.

[0041] In the signal reconstruction process, the perceptual features Y′ that generate pure speech are first processed by frame segmentation, and the signal is reconstructed frame by frame. Then, the reconstructed frame signals are spliced ​​together to obtain the generated pure speech signal X′.

[0042] Specifically, in step S1, the endpoint detection includes the following steps:

[0043] S11. Use a two - threshold two - level discrimination method based on short - time energy and zero - crossing rate to perform endpoint detection on the clean speech signal X, and distinguish the speech segments and non - speech segments of the clean speech signal X.

[0044] Among them, the short - time energy discrimination method is to set a relatively high average energy threshold T1 and a slightly lower threshold T2, where T2 = α1E N , E N is the average energy of the speech signal, α1 is an empirical parameter, T2 is used to determine the end point of speech in the first level, and complete the first - level decision. The zero - crossing rate discrimination method is based on the zero - crossing rate Z N of the speech signal. Set a threshold T3 to judge the starting point of speech, obtain the starting and ending point positions of the speech segment. By deleting the signal between the starting point and the ending point of the speech segment, the non - speech segment is obtained, and the second - level decision is completed.

[0045] Specifically, in step S2, the perceptual feature extraction includes the following steps:

[0046] S21. Perform perceptual measurement on the N×1 - dimensional clean speech signal X on the M×N - dimensional random Gaussian matrix Φ, so that Y1 = ΦX, and obtain the M×1 - dimensional perceptual feature Y1 of the clean speech. And the dimension M of the perceptual feature Y1 of the clean speech is less than the dimension N of the clean speech signal X;

[0047] For the perceptual feature extraction of the noisy speech signal, the Gaussian random matrix used is the same as the Gaussian matrix Φ used for the perceptual feature extraction of the clean speech signal X, and the dimension of the noisy speech signal is the same as the dimension of the clean speech signal X. Through perceptual feature extraction of the noisy speech signal, the perceptual feature Y2 of the noisy speech is obtained.

[0048] Among them, the perceptual feature extraction is based on the compressive sampling principle of compressive sensing theory;

[0049] The dimension M of the perceptual feature Y1 of the clean speech is much less than the dimension N of the clean speech signal X, that is, M << N. The compression ratios of the perceptual feature Y1 of the clean speech and the perceptual feature Y2 of the noisy speech are both M / N;

[0050] Among them, the signal frame length is set to 128, and perceptual feature extraction is performed on the 128×1 - dimensional signal on the M×128 - dimensional Gaussian random matrix to obtain the M×1 - dimensional perceptual feature. The 128×1 - dimensional signal is the clean speech signal X or the noisy speech signal. Since M << 128, the 128×1 - dimensional signal is compressed into the M×1 - dimensional perceptual feature.

[0051] Specifically, in step S2, the perceptual feature extraction further includes the following steps:

[0052] S22. First, perform perceptual feature extraction on each frame of the clean speech signal X and the noisy speech signal. Then, concatenate the perceptual features of each frame to obtain the perceptual features Y1 of the clean speech signal and the perceptual features Y2 of the noisy speech signal.

[0053] The extraction of perceptual features for clean speech signal X and noisy speech signal involves observing each frame of the clean speech signal X and noisy speech signal after they are framed, splicing together the obtained perceptual features to finally obtain the perceptual features Y1 of the clean speech signal and the perceptual features Y2 of the noisy speech signal.

[0054] Specifically, in step S3, the generative adversarial network includes a generator network G and a discriminator network D. The generator network G is used to denoise the perceptual features Y2 of the noisy speech to obtain the perceptual features Y′ of the generated clean speech. The discriminator network D is used to determine whether the generated clean speech signal X′ is consistent with the clean speech signal X.

[0055] Specifically, the training process of a generative adversarial network involves two networks in the network being trained alternately, with their capabilities improving synchronously, until the data generated by the generative network is indistinguishable from the real data, and the ability of the discriminator network D reaches Nash equilibrium.

[0056] Specifically, in step S3, training the generative adversarial network includes the following steps:

[0057] S31. Input the perceptual features Y2 of the noisy speech into the generator network G, and use the perceptual features Y1 of the clean speech as the objective function. The output is the perceptual features Y′ of the generated clean speech. Reconstruct the perceptual features Y′ of the generated clean speech using the OMP algorithm of compressed sensing in step S4 to obtain the generated clean speech signal X′. At the same time, use the endpoint detection algorithm to extract the speech-free segments of the clean speech signal X and the generated clean speech X′. Use the speech-free segments of the clean speech X as the real data of the discrimination network D. The discrimination network D discriminates the speech-free segments of the generated clean speech signal X′. The purpose of this is to ensure that the speech-free segments of the generated clean speech signal X′ are free of noise. It is necessary to determine whether the speechless segments of the generated clean speech signal X′ are consistent with the speechless segments of the clean speech X. Through repeated iterations and continuous training, the discrimination network D reaches Nash equilibrium. Then, the perceptual features Y′ of the clean speech are generated through the generator network G. A mapping relationship is established between the perceptual features Y1 of the noisy speech and the perceptual features Y2 of the clean speech, where the Nash equilibrium, representing the discrimination result of the discrimination network D, is 0.5. After establishing this mapping relationship, the trained generative adversarial network model is called, and the perceptual features Y1 of the noisy speech are input to obtain the perceptual features Y2 of the clean speech.

[0058] By changing the Gaussian random matrix used in perceptual feature extraction, multiple generative adversarial network models under different Gaussian matrices can be obtained. That is, each time a different Gaussian random matrix is ​​used for signal compression, a corresponding generative adversarial network model will be obtained. Using different generative adversarial network models makes it easier to obtain different denoising effects. The generative adversarial network model with the best denoising effect can be obtained by selection.

[0059] The OMP reconstruction algorithm selects columns of the measurement matrix Φ in a greedy iterative manner, so that the selected columns in each iteration are as correlated as possible with the current redundant vector. The correlated part is subtracted from the measurement vector and the iteration is repeated until the number of iterations reaches the sparsity K, at which point the iteration is forced to stop.

[0060] Specifically, in step S4, the signal reconstruction includes the following steps:

[0061] S41. Select the DCT matrix Ψ as the sparse representation matrix to obtain the sensing matrix A=ΦΨ. According to Y′=A×theta and Y′=Ψ×theta, use the least squares method to solve for theta, and then reconstruct the pure speech signal X′.

[0062] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, constitutes a step in the method of the present invention.

[0063] In another aspect, the present invention provides a speech denoising device based on deep compressed sensing, comprising: a memory for storing a software application, and a processor for executing the software application, wherein each program of the software application correspondingly executes a step in a speech denoising method based on deep compressed sensing of the present invention.

Claims

1. A speech denoising method based on deep compressed sensing, characterized in that, Includes the following steps: Signal preprocessing: Gaussian white noise is added to the clean speech signal X to obtain a noisy speech signal. The clean speech signal X and the noisy speech signal are subjected to duration normalization and frame segmentation. Then, endpoint detection processing is performed on the clean speech signal X. Signal compression: The compressed sensing method is used to extract perceptual features from the preprocessed clean speech signal X and the noisy speech signal to obtain the perceptual features Y1 of the clean speech and the perceptual features Y2 of the noisy speech. Model Training: A generative adversarial network (GAN) based on deep learning is used as the denoising model. The perceptual features Y2 of the noisy speech are used as the input to the GAN, and the perceptual features Y1 of the clean speech are used as the objective function. The GAN is trained through repeated iterations and multiple training sessions to obtain the perceptual features for generating clean speech. ; Signal reconstruction: The OMP reconstruction algorithm based on compressed sensing is used to analyze the perceptual features of the generated clean speech. Signal reconstruction is performed to obtain a clean speech signal. ; In the signal reconstruction process, the perceptual features for generating clean speech are first... The signal is reconstructed frame by frame, and then the reconstructed frames are spliced ​​together to obtain a clean speech signal. ; The generative adversarial network includes a generator network G and a discriminator network D. The generator network G is used to denoise the perceptual features Y2 of the noisy speech to obtain perceptual features for generating clean speech. The discrimination network D is used to discriminate the generated clean speech signal. Does it match the pure speech signal X? The training of the generative adversarial network includes the following steps: The perceptual features Y2 of the noisy speech are input into the generator network G, and the perceptual features Y1 of the clean speech are used as the objective function. The output is the perceptual features for generating clean speech. Perceptual features for generating clean speech The clean speech signal is reconstructed using the OMP algorithm of compressed sensing. Extracting clean speech signal X and generating clean speech The speechless segments of clean speech X are used as the real data for the discrimination network D, which then generates clean speech signals. The speech-free segments are repeatedly iterated and trained until the discrimination network D reaches Nash equilibrium. Then, the perceptual features of clean speech are generated through the generator network G. Establish the mapping relationship between the perceptual features Y1 of noisy speech and the perceptual features Y2 of clean speech.

2. The speech denoising method based on deep compressed sensing according to claim 1, characterized in that, The voice duration normalization includes the following steps: Set the lengths of the clean speech signal X and the noisy speech signal to integer multiples of the signal frame length.

3. The speech denoising method based on deep compressed sensing according to claim 1, characterized in that, The endpoint detection includes the following steps: A two-level discrimination method with dual thresholds based on short-time energy and zero-crossing rate is used to perform endpoint detection on clean speech signal X, and to distinguish between speech segments and non-speech segments in clean speech signal X.

4. The speech denoising method based on deep compressed sensing according to claim 1, characterized in that, The perceptual feature extraction includes the following steps: A clean speech signal X of N×1 dimensions is perceptually measured on an M×N dimensional random Gaussian matrix Φ, such that Y1=ΦX, to obtain the perceptual features Y1 of the clean speech of M×1 dimensions, where the dimension M of the perceptual features Y1 of the clean speech is less than the dimension N of the clean speech signal X. The Gaussian random matrix used for perceptual feature extraction of noisy speech signal is the same as the Gaussian matrix Φ used for perceptual feature extraction of clean speech signal X, and the dimension of noisy speech signal is the same as the dimension N of clean speech signal X. The perceptual feature Y2 of noisy speech signal is obtained through perceptual feature extraction.

5. A speech denoising method based on deep compressed sensing according to claim 4, characterized in that, The perceptual feature extraction also includes the following steps: First, perceptual features are extracted from each frame of the clean speech signal X and the noisy speech signal. Then, the perceptual features of each frame are spliced ​​together to obtain the perceptual features Y1 of the clean speech signal and the perceptual features Y2 of the noisy speech signal.

6. The speech denoising method based on deep compressed sensing according to claim 1, characterized in that, The signal reconstruction includes the following steps: Choosing the DCT matrix Ψ as the sparse representation matrix, the sensing matrix A = ΦΨ is obtained, according to... =A×theta and =Ψ×theta, solve for theta using the least squares method, and then reconstruct the pure speech signal. .

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the steps of the speech denoising method based on deep compressed sensing as described in any one of claims 1 to 6.

8. A speech denoising device based on deep compressed sensing, characterized in that, include: Memory, used to store software applications; A processor for executing the software application, wherein each program of the software application correspondingly performs the steps of the speech denoising method based on deep compressed sensing as described in any one of claims 1 to 6.

Images