Speech noise separation method in high-noise environment based on independence analysis

Abstract

The invention discloses an independence analysis-based voice noise separation method in a high-noise environment, which comprises the following steps of: firstly, acquiring a mixed signal containing noise, decomposing the mixed signal containing the noise, acquiring parallel independent waveforms based on convolution reasoning, performing independence evaluation on the acquired independent waveforms, and performing resampling; obtaining an improved sampling waveform after marginal probability distribution; the method comprises the following steps: acquiring an independent waveform and an improved sampling waveform, then constructing a discriminator, inputting the acquired independent waveform and the improved sampling waveform into the discriminator, acquiring linear expressions of the two waveforms, then constructing a voice noise separation model, optimizing the model by using SISNR and GAN based on the discriminator, and finally performing noise separation on voice in a high-noise environment based on the optimized model. According to the voice noise separation method, a training method for carrying out independence analysis on mixed signals is adopted, a generative adversarial network and a deep convolutional network are combined, a separation-resampling-independence analysis voice separation method is realized, independence analysis and correlation suppression are combined, and compared with traditional signal processing, the method has the advantages that the method is simple and convenient to operate, and the efficiency is high. And important performance improvement is achieved on the premise of low signal-to-noise ratio.

Description

Technical Field

[0001] The invention belongs to the field of speech noise separation, and in particular relates to a speech noise separation method in a high noise environment based on independence analysis. Background Art

[0002] In the era of booming neural network algorithms, deep learning focuses on how to represent existing inputs and learn the relationship between representations in high-dimensional space. In order to achieve excellent representation learning methods, deep networks usually need to obtain high-dimensional feature expressions through training to express the dependencies between variables with less information and accurately analyze the nature of the variables themselves. For information processing models in nature, speech signals play an important role in deep learning. They can be used to learn their high-dimensional features and perform various processing tasks, such as speech signal separation.

[0003] In daily life, the speech signals heard by people are often interfered by noise, which seriously damages the clarity of speech and may reduce the performance of speech recognition models. Speech separation technology is one of the most commonly used methods for situations with noise or mixed sounds. A good speech signal decomposition module can significantly improve the clarity of speech and the performance of automatic speech recognition models. However, in real environments, the performance of speech separation technology still needs to be improved and faces huge challenges, especially in single-channel and non-stationary noise scenarios. An important algorithmic idea is the component analysis method, which is a useful and relatively traditional blind source signal separation tool. Compared with traditional signal processing methods, the method based on deep learning models uses the pure signal before mixing to build models of speech and noise, and has achieved significant performance improvements under low signal-to-noise ratio conditions.

[0004] Nowadays, deep learning-based methods have made great progress in solving independent component analysis (ICA) methods. Deep learning technology improves the performance of time-frequency masking methods and improves the accuracy of masking estimation. In direct methods and mask estimation methods, the waveform of each sound source is calculated by estimating the inverse short-time Fourier transform (ISTFT) of the amplitude spectrogram of each sound source together with the original phase or modified phase of the mixed sound. Although time-frequency masking is still the most commonly used method in speech separation, this method has some shortcomings. Summary of the invention

[0005] In view of the above problems, the object of the present invention is to provide a method for separating speech noise in a high noise environment based on independence analysis.

[0006] The specific technical solution for achieving the purpose of the present invention is:

[0007] A method for separating speech noise in a high noise environment based on independence analysis comprises the following steps:

[0008] Step 1: Obtain a mixed signal containing noise, decompose the mixed signal containing noise, and obtain parallel independent waveforms based on convolution reasoning;

[0009] Step 2: Perform independence evaluation on the acquired independent waveforms and resample them to obtain improved sampling waveforms after edge probability distribution;

[0010] Step 3: construct a discriminator, input the independent waveform obtained in step 1 and the improved sampling waveform obtained in step 2 into the discriminator, and obtain the linear expression of the two;

[0011] Step 4: Build a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in step 3;

[0012] Step 5: Perform noise separation on speech in high noise environment based on the optimized model.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] The speech noise separation method of the present invention revolves around the training method for independence analysis of mixed signals, combines generative adversarial networks and deep convolutional networks, and implements a "separation-resampling-independence analysis" speech separation method, which combines independence analysis and correlation suppression. Compared with traditional signal processing methods, the method based on deep learning models uses the pure signal before mixing to construct models of speech and noise respectively, and achieves important performance improvement under the premise of low signal-to-noise ratio.

[0015] The present invention is further described below in conjunction with specific implementation modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the speech noise separation method in a high noise environment based on independence analysis of the present invention.

[0017] Figure 2 It is a schematic diagram of the network structure of speech noise separation in a high noise environment based on independence analysis of the present invention. DETAILED DESCRIPTION

[0018] Example

[0019] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an kind" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0021] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so that once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0022] Combination Figure 1 , a method for separating speech noise in a high noise environment based on independence analysis, comprising the following steps:

[0023] Step 1: Obtain a mixed signal containing noise, decompose the mixed signal containing noise, and obtain parallel independent waveforms based on convolution inference:

[0024] Step 1-1, the mixed signal includes multiple speakers' speech and noise speech, and the speech mixing method includes convolution mixing, instantaneous mixing, linear mixing or nonlinear mixing;

[0025] Step 1-2, decompose the mixed signal based on the signal decomposition module, the signal decomposition module generally includes three parts, namely, a mix encoder (Mix-Encoder), a mask separator (Mask-Separator) and a reconstruction decoder (Reconstructor);

[0026] Step 1-2-1, converting the mixed signal fragment into a feature representation based on a mixed encoder (Mix-Encoder), in which each mixed signal fragment is convolutionally mapped to a non-negative feature, and each mixed signal fragment is convolutionally mapped to a non-negative feature;

[0027] Step 1-2-2, learning and generating a mask based on a mask separator, using the mask to separate the feature representation to obtain the features of each independent signal;

[0028] In order to effectively learn the method of separating signals, the mask separator uses a stacked nested one-dimensional convolutional network; since the separation task is performed in the time domain, the information that needs to be predicted is a long time window, so the TCN structure is used instead of the RNN to extract time domain features. The extraction of time domain features is realized based on the TCN structure. The input signal is input from the BatchNormalization layer to the BottleNeck layer, and then to M 1D-Conv layers. After repeating R times, it enters M 1D-Conv layers again, and then inputs to the full convolution Conv layer, and finally outputs the mask of the separated signal;

[0029] Step 1-2-3: Based on the reconstruction decoder (Reconstructor), a one-dimensional linear deconvolution operation is performed on the estimated pure signal, that is, each independent signal feature, to obtain a set of reconstructed independent waveforms Z.

[0030] Step 2: Evaluate the independence of the acquired independent waveforms and resample them to obtain improved sampled waveforms after edge probability distribution. The purpose of these operations is to eliminate the dependencies between data, ensure the independence of each signal in the discriminator, and ultimately improve the effect of speech separation. Specifically, they include:

[0031] Step 2-1: First, normalize the decomposed and reconstructed independent waveform Z to prevent the feature from tending to zero or infinity, and ensure that the data is within a relatively stable range:

[0032]

[0033] Where Z represents the independent waveform Z decomposed in step 1, Z′ represents the normalized data, B is the scaling factor, and μ represents the expectation used for normalization;

[0034] Step 2-2, resample the normalized data: based on the normalized data Z′, shuffle the data in each column, keep the row data unchanged, and resample; the resampled data That is, the Kth column data of the current row is reselected from all the column data of Z′ as the newly sampled sample X sk ;

[0035] Step 2-3, based on the resampled waveform Simulate their marginal distribution without considering their joint distribution. In order to ensure that sampling in each dimension will not be affected by other dimensions, the columns are shuffled to eliminate the influence of the joint distribution. This ensures that when processing each variable, their marginal distribution is independent and not affected by other variables in the joint distribution, thereby better simulating the independence of the signal;

[0036] The resampling operation in this step provides a simulated marginal distribution for the discriminator, and the joint distribution obtained based on the pure waveform is sent to the discriminator together with the marginal distribution for discrimination.

[0037] Step 3: construct a discriminator, input the independent waveform obtained in step 1 and the improved sampling waveform obtained in step 2 into the discriminator, and obtain the linear expression of the two;

[0038] The discriminator is composed of a three-layer linear perceptron MLP, which receives the independent waveform obtained in step 1 and the improved sampling waveform obtained in step 2 respectively, and outputs the discriminant vector of the two as the basis of the GAN loss function.

[0039] Step 4: Build a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in step 3;

[0040] Specifically, a speech noise separation model is built based on a deep learning network, and adversarial strategy optimization and gradient descent optimization are performed on it based on the loss function;

[0041] The loss function is:

[0042] L ALL =λ1SISNR+λ2L GAN (G,D)

[0043] Among them, λ1 and λ2 are used to adjust the importance of SISNR and GAN loss in the total loss function respectively; SISNR represents the scale-invariant signal-to-noise ratio loss function, L GAN (G, D) represents the loss function of the generative adversarial network, where G represents the GAN network discriminator and D represents the GAN network generator. The input of the generator is the mixed speech signal or an initial estimation signal generated from random noise, and the output is an independent speech signal that is attempted to be generated, that is, the clear speech after separation.

[0044] Through this function, a system for learning independent components can be trained, and a separation module can be learned through an adversarial strategy, which can generate prediction signals that meet the mutual independence conditions between source signals;

[0045] One part of the loss function is used to optimize the parameters of the signal decomposition module. The goal of this optimizer is to minimize the SISNR loss related to signal separation. Another optimizer is used to optimize the parameters of the discriminator part of GAN. The goal of this optimizer is to minimize the loss related to GAN.

[0046] And open up two different spaces to store and update weights.

[0047] During the optimization process, the objective function is to optimize the discriminator G and the generator D respectively. First, the generator D is optimized, and then the discriminator G is optimized in several cycles. The algorithm first removes L GAN (G,D), only use SISNR loss function to train the structure described in the above network. After reaching a certain cycle, independence analysis is performed to optimize the GAN objective function of the network as a whole;

[0048] Step 5: Finally, based on the optimized model, perform noise separation on the speech in the high noise environment.

[0049] The present invention also provides a speech noise separation system in a high noise environment based on independence analysis, comprising the following modules:

[0050] Mixed signal decomposition module: used to obtain a mixed signal containing noise, decompose the mixed signal containing noise, and obtain a set of parallel independent waveforms based on convolution reasoning;

[0051] Resampling module: used to evaluate the independence of the acquired independent waveforms and resample them to obtain improved sampling waveforms after edge probability distribution;

[0052] A discriminator building module; used for building a discriminator based on the independent waveform obtained by decomposition and the improved sampling waveform obtained by resampling;

[0053] Speech noise separation model construction module: used to build a speech noise separation model and optimize the model based on the discriminator using SISNR and GAN;

[0054] Noise separation module: used to separate the noise of speech in high noise environment based on the optimized model.

[0055] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0056] Step 1: Obtain a mixed signal containing noise, decompose the mixed signal containing noise, and obtain parallel independent waveforms based on convolution reasoning;

[0057] Step 2: Perform independence evaluation on the acquired independent waveforms and resample them to obtain improved sampling waveforms after edge probability distribution;

[0058] Step 3: construct a discriminator, input the independent waveform obtained in step 1 and the improved sampling waveform obtained in step 2 into the discriminator, and obtain the linear expression of the two;

[0059] Step 4: Build a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in step 3;

[0060] Step 5: Perform noise separation on speech in high noise environment based on the optimized model.

[0061] A computer storable medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the following steps are implemented:

[0062] Step 1: Obtain a mixed signal containing noise, decompose the mixed signal containing noise, and obtain parallel independent waveforms based on convolution reasoning;

[0063] Step 2: Perform independence evaluation on the acquired independent waveforms and resample them to obtain improved sampling waveforms after edge probability distribution;

[0064] Step 3: construct a discriminator, input the independent waveform obtained in step 1 and the improved sampling waveform obtained in step 2 into the discriminator, and obtain the linear expression of the two;

[0065] Step 4: Build a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in step 3;

[0066] Step 5: Perform noise separation on speech in high noise environment based on the optimized model.

[0067] The speech noise separation method in this scheme revolves around the training method of independence analysis of mixed signals, combining generative adversarial networks and deep convolutional networks to implement a "separation-resampling-independence analysis" speech separation method, which combines independence analysis and correlation suppression. Compared with traditional signal processing methods, the method based on deep learning models uses the pure signal before mixing to construct speech and noise models respectively, achieving important performance improvements under the premise of low signal-to-noise ratio.

[0068] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for separating speech noise in a high noise environment based on independence analysis, characterized in that: The following steps are involved: Step 1: Obtain a mixed signal containing noise, decompose the mixed signal containing noise, and obtain parallel independent waveforms based on convolution reasoning; Step 2: Perform independence evaluation on the acquired independent waveforms and resample them to obtain improved sampling waveforms after edge probability distribution; Step 3: construct a discriminator, input the independent waveform obtained in step 1 and the improved sampling waveform obtained in step 2 into the discriminator, and obtain the linear expression of the two; Step 4: Build a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in step 3; Step 5: Perform noise separation on speech in high noise environment based on the optimized model.

2. The method for separating speech noise in a high noise environment based on independence analysis according to claim 1, characterized in that: The step 1 of obtaining a set of independent waveforms in parallel based on convolutional reasoning is specifically as follows: Step 1-1, the mixed signal includes multiple speakers' speech and noise speech, and the speech mixing method includes convolution mixing, instantaneous mixing, linear mixing or nonlinear mixing; Step 1-2: Decompose the mixed signal based on the signal decomposition module.

3. The method for separating speech noise in a high noise environment based on independence analysis according to claim 2, characterized in that: The mixed signal is decomposed based on the signal decomposition module in step 1-2, specifically: Step 1-2-1, converting the mixed signal segment into a feature representation based on a hybrid encoder; Step 1-2-2, learning and generating a mask based on a mask separator, using the mask to separate feature representations to obtain features of each independent signal; The mask separator uses a stacked nested one-dimensional convolutional network and realizes the extraction of time domain features based on the TCN structure. The input signal is input from the Batch Normalization layer to the BottleNeck layer, and then to M 1D-Conv layers. After repeating R times, it enters M 1D-Conv layers again, and then inputs to the full convolution Conv layer, and finally outputs the mask of the separated signal; Step 1-2-3: Perform a one-dimensional linear deconvolution operation on each independent signal feature based on the reconstruction decoder to obtain a set of reconstructed independent waveforms Z.

4. The method for separating speech noise in a high noise environment based on independence analysis according to claim 1, characterized in that: The waveform resampling in step 2 is specifically as follows: Step 2-1: normalize the decomposed and reconstructed independent waveform Z: Where Z represents the independent waveform Z decomposed in step 1, Z′ represents the normalized data, B is the scaling factor, and μ represents the expectation used for normalization; Step 2-2, resample the normalized data: based on the normalized data Z′, shuffle the data in each column, keep the row data unchanged, and resample; the resampled data That is, the Kth column data of the current row is reselected from all the column data of Z′ as the newly sampled sample X sk ; Step 2-3, based on the resampled waveform Simulate its marginal distribution.

5. The method for separating speech noise in a high noise environment based on independence analysis according to claim 1, characterized in that: The construction of the GAN network discriminator in step 3 is specifically as follows: The discriminator is composed of a three-layer linear perceptron MLP, which receives the independent waveform obtained in step 1 and the improved sampling waveform obtained in step 2 respectively, and outputs the discriminant vector of the two as the basis of the GAN loss function.

6. The method for separating speech noise in a high noise environment based on independence analysis according to claim 5, characterized in that: The step 4 of constructing a speech noise separation model and optimizing it is as follows: Build a speech noise separation model based on a deep learning network, and perform adversarial optimization and gradient descent optimization on it based on the loss function; The loss function is: <h2 style=";text-align:left;direction:ltr">L<h2 style=";text-align:left;direction:ltr"> ALL <h2 style=";text-align:left;direction:ltr"> =λ1SISNR+λ2L<h2 style=";text-align:left;direction:ltr"> GAN <h2 style=";text-align:left;direction:ltr"> (G,D) Among them, λ1 and λ2 are used to adjust the importance of SISNR and GAN loss in the total loss function respectively; SISNR represents the scale-invariant signal-to-noise ratio loss function, L GAN (G, D) represents the loss function of the generative adversarial network, where G represents the GAN network discriminator and D represents the GAN network generator.

7. A speech noise separation system in a high noise environment based on independence analysis, characterized in that: Includes the following modules: Mixed signal decomposition module: used to obtain a mixed signal containing noise, decompose the mixed signal containing noise, and obtain a set of parallel independent waveforms based on convolution reasoning; Resampling module: used to evaluate the independence of the acquired independent waveforms and resample them to obtain improved sampling waveforms after edge probability distribution; A discriminator building module; used for building a discriminator based on the independent waveform obtained by decomposition and the improved sampling waveform obtained by resampling; Speech noise separation model construction module: used to build a speech noise separation model and optimize the model based on the discriminator using SISNR and GAN; Noise separation module: used to separate the noise of speech in high noise environment based on the optimized model.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer storable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images