A speech noise separation method in a high noise environment based on independence analysis

Abstract

This invention discloses a speech noise separation method in high-noise environments based on independence analysis. First, a noisy mixed signal is acquired and decomposed. Parallel independent waveforms are obtained based on convolutional inference. The independence of these independent waveforms is evaluated, and resampling is performed to obtain an improved sampled waveform with marginal probability distribution. Then, a discriminator is constructed, inputting the acquired independent waveforms and the improved sampled waveforms to obtain their linear representations. A speech noise separation model is then built, and based on the discriminator, the model is optimized using SISNR and GAN. Finally, the optimized model is used to separate speech noise in high-noise environments. This invention's speech noise separation method revolves around a training method for independence analysis of mixed signals, combining generative adversarial networks and deep convolutional networks to achieve a "separation-resampling-independence analysis" speech separation method. This method combines independence analysis and correlation suppression, achieving significant performance improvements compared to traditional signal processing at low signal-to-noise ratios.

Description

Technical Field

[0001] This invention belongs to the field of speech noise separation, specifically relating to a speech noise separation method in high-noise environments based on independence analysis. Background Technology

[0002] In the era of rapid development of neural network algorithms, deep learning primarily focuses on how to represent existing inputs and learn the relationships between these representations in a high-dimensional space. To achieve excellent representation learning methods, deep networks typically need to acquire high-dimensional feature representations through training, enabling them to express the dependencies between variables with less information and accurately analyze the essence of the variables themselves. For information processing models in nature, speech signals play a crucial role in deep learning, serving as a tool to learn their high-dimensional features and perform various processing tasks, such as speech signal separation.

[0003] In daily life, the speech signals we hear are often interfered with by noise, severely impairing speech clarity and potentially reducing the performance of speech recognition models. Speech separation technology is one of the most commonly used methods for situations involving noisy or mixed sounds. A good speech signal decomposition module can significantly improve speech clarity and the performance of automatic speech recognition models. However, in real-world environments, the performance of speech separation technology still needs improvement and faces significant challenges, especially in single-channel and non-stationary noise scenarios. An important algorithmic approach is component analysis, a useful and relatively traditional tool for separating blind source signals. Compared to traditional signal processing methods, deep learning-based methods utilize the clean signal before mixing to construct models of speech and noise, achieving significant performance improvements under low signal-to-noise ratio conditions.

[0004] Currently, deep learning-based methods have made significant progress in solving Independent Component Analysis (ICA) problems. Deep learning techniques have improved the performance of time-frequency masking methods and increased the accuracy of mask estimation. In direct methods and mask estimation methods, the waveform of each sound source is calculated by estimating the inverse short-time Fourier transform (ISTFT) of the amplitude spectrum of each sound source together with the original or corrected phase of the mixed sound. Although time-frequency masking remains the most commonly used method in speech separation, it has some limitations. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide a speech noise separation method in high-noise environments based on independence analysis.

[0006] The specific technical solution for achieving the objective of this invention is as follows:

[0007] A speech noise separation method based on independence analysis in high-noise environments includes the following steps:

[0008] Step 1: Obtain the noisy mixed signal, decompose the noisy mixed signal, and obtain parallel independent waveforms based on convolution inference;

[0009] Step 2: Evaluate the independence of the acquired independent waveforms and resample them to obtain the improved sampled waveforms after edge probability distribution;

[0010] Step 3: Construct a discriminator by inputting the independent waveform obtained in Step 1 and the improved sampled waveform obtained in Step 2 into the discriminator to obtain their linear representations.

[0011] Step 4: Construct a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in Step 3.

[0012] Step 5: Perform noise separation on speech in high-noise environments based on the optimized model.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0014] The speech noise separation method of this invention revolves around a training method for performing independence analysis on mixed signals. It combines generative adversarial networks and deep convolutional networks to realize a "separation-resampling-independence analysis" speech separation method. It combines independence analysis and correlation suppression. Compared with traditional signal processing methods, the deep learning model-based method uses the clean signal before mixing to construct speech and noise models separately, achieving significant performance improvement under the premise of low signal-to-noise ratio.

[0015] The present invention will be further described below with reference to specific embodiments. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the speech noise separation method in a high-noise environment based on independence analysis according to the present invention.

[0017] Figure 2 This is a schematic diagram of the network structure for speech noise separation in high-noise environments based on independence analysis according to the present invention. Detailed Implementation

[0018] Example

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0021] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0022] Combination Figure 1 A speech noise separation method based on independence analysis in high-noise environments includes the following steps:

[0023] Step 1: Obtain the noisy mixed signal, decompose the noisy mixed signal, and obtain parallel independent waveforms based on convolutional inference:

[0024] Step 1-1: The mixed signal includes multiple speakers' speech and noisy speech. The speech mixing method includes convolutional mixing, instantaneous mixing, linear mixing, or nonlinear mixing.

[0025] Steps 1-2: Decompose the mixed signal based on the signal decomposition module. The signal decomposition module generally includes three parts: a mixed encoder, a mask separator, and a reconstructor.

[0026] Step 1-2-1: Based on the Mix-Encoder, the mixed signal segments are converted into feature representations. In the Mix-Encoder, each mixed signal segment is convolved and mapped to a non-negative feature.

[0027] Step 1-2-2: Learn and generate a mask based on the mask-separator, and use the mask to separate the feature representations to obtain the features of each independent signal;

[0028] To effectively learn the method for separating signals, the mask separator uses stacked nested one-dimensional convolutional networks. Since the separation task is performed in the time domain, the information that needs to be predicted is for a long time window. Therefore, a TCN structure is used instead of an RNN to extract time-domain features. Based on the TCN structure, the input signal is fed into a BottleNeck layer, then into M 1D-Conv layers, repeated R times, then into M more 1D-Conv layers, and finally into a fully convolutional Conv layer. Finally, the mask of the separated signal is output.

[0029] Steps 1-2-3: Based on the reconstruction decoder, perform a one-dimensional linear deconvolution operation on the estimated clean signal, i.e., each independent signal feature, to obtain a set of reconstructed independent waveforms Z.

[0030] Step 2: The acquired independent waveforms are evaluated for independence and resampled to obtain improved sampled waveforms with edge probability distributions. These operations aim to eliminate dependencies between data, ensure the independence of each signal in the discriminator, and ultimately improve speech separation performance. Specifically, this includes:

[0031] Step 2-1: First, normalize the reconstructed independent waveform Z to prevent features from tending to 0 or becoming infinitely large, ensuring the data remains within a relatively stable range.

[0032]

[0033] Where Z represents the independent waveform Z decomposed in step 1, Z′ represents the normalized data, B is the scaling factor, and μ represents the expectation used for normalization.

[0034] Step 2-2: Resample the normalized data: Based on the normalized data Z′, shuffle each column of data while keeping the row data unchanged, and then resample; the resampled data... That is, the Kth column of the current row is reselected from all columns of Z′ and used as the newly sampled sample X. sk ;

[0035] Steps 2-3: Based on the resampled waveform The marginal distributions of the signals are simulated without considering their joint distribution. To ensure that sampling in each dimension is not affected by other dimensions, a shuffled column arrangement is used to eliminate the influence of the joint distribution. This ensures that the marginal distributions of each variable are independent when processing them, unaffected by other variables in the joint distribution, thus better simulating the independence of the signals.

[0036] The resampling operation in this step provides the discriminator with a simulated edge distribution. The joint distribution obtained based on the pure waveform is then fed into the discriminator along with the edge distribution for differentiation.

[0037] Step 3: Construct a discriminator by inputting the independent waveform obtained in Step 1 and the improved sampled waveform obtained in Step 2 into the discriminator to obtain their linear representations.

[0038] The discriminator consists of a three-layer linear perceptron (MLP), which receives the independent waveform obtained in step 1 and the improved sampled waveform obtained in step 2, and outputs the discriminant vectors of the two as the basis for the GAN loss function.

[0039] Step 4: Construct a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in Step 3.

[0040] Specifically, a speech noise separation model is built based on a deep learning network, and adversarial strategy optimization and gradient descent optimization are performed on it based on the loss function;

[0041] The loss function is:

[0042] L ALL =λ1SISNR+λ2L GAN (G,D)

[0043] Where λ1 and λ2 are used to adjust the importance of SISNR and GAN loss in the total loss function, respectively; SISNR represents the scale-invariant signal-to-noise ratio loss function, L GAN (G,D) denotes the loss function of the generative adversarial network, where G represents the discriminator of the GAN network and D represents the generator of the GAN network. The input of the generator is either the mixed speech signal or an initial estimated signal generated from random noise, and the output is the independent speech signal that is attempted to be generated, i.e., the separated clear speech.

[0044] This function can be used to train a system for learning independent components. Through an adversarial strategy, a separation module can be learned, which can generate a predicted signal that satisfies the condition that the source signals are mutually independent.

[0045] One part of the loss function is used to optimize the parameters of the signal decomposition module. The goal of this optimizer is to minimize the SISNR loss associated with signal separation. Another optimizer is used to optimize the parameters of the discriminator part in the GAN. The goal of this optimizer is to minimize the GAN-related loss.

[0046] In addition, two different spaces are allocated for storing and updating weights.

[0047] During the optimization process, the objective function optimizes both the discriminator G and the generator D separately. First, the generator D is optimized, and then the discriminator G is optimized within a few loops. The algorithm first removes L... GAN (G,D), the structure described in the above network is trained using only the SISNR loss function. After a certain number of cycles, independence analysis is performed to optimize the GAN objective function of the network as a whole.

[0048] Step 5: Finally, based on the optimized model, perform noise separation on speech in high-noise environments.

[0049] This invention also provides a speech noise separation system in a high-noise environment based on independence analysis, comprising the following modules:

[0050] Mixed signal decomposition module: used to acquire a noisy mixed signal, decompose the noisy mixed signal, and obtain a set of parallel independent waveforms based on convolutional inference;

[0051] Resampling module: Used to evaluate the independence of the acquired independent waveforms and resample them to obtain improved sampled waveforms with edge probability distribution;

[0052] Discriminator building module; used to build a discriminator based on the independent waveforms obtained from decomposition and the improved sampled waveforms obtained from resampling;

[0053] Speech noise separation model building module: used to build a speech noise separation model, and optimize the model based on the discriminator using SISNR and GAN;

[0054] Noise separation module; used to separate noise from speech in high-noise environments based on an optimized model.

[0055] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, performs the following steps:

[0056] Step 1: Obtain the noisy mixed signal, decompose the noisy mixed signal, and obtain parallel independent waveforms based on convolutional inference;

[0057] Step 2: Evaluate the independence of the acquired independent waveforms and resample them to obtain the improved sampled waveforms after edge probability distribution;

[0058] Step 3: Construct a discriminator by inputting the independent waveform obtained in Step 1 and the improved sampled waveform obtained in Step 2 into the discriminator to obtain their linear representations.

[0059] Step 4: Construct a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in Step 3.

[0060] Step 5: Perform noise separation on speech in high-noise environments based on the optimized model.

[0061] A computer-storable medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, performs the following steps:

[0062] Step 1: Obtain the noisy mixed signal, decompose the noisy mixed signal, and obtain parallel independent waveforms based on convolution inference;

[0063] Step 2: Evaluate the independence of the acquired independent waveforms and resample them to obtain the improved sampled waveforms after edge probability distribution;

[0064] Step 3: Construct a discriminator by inputting the independent waveform obtained in Step 1 and the improved sampled waveform obtained in Step 2 into the discriminator to obtain their linear representations.

[0065] Step 4: Construct a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in Step 3.

[0066] Step 5: Perform noise separation on speech in high-noise environments based on the optimized model.

[0067] This speech noise separation method revolves around training methods for independence analysis of mixed signals. It combines generative adversarial networks and deep convolutional networks to achieve a "separation-resampling-independence analysis" speech separation method. It combines independence analysis and correlation suppression. Compared with traditional signal processing methods, the deep learning model-based method uses the clean signal before mixing to build speech and noise models separately, achieving significant performance improvement under the premise of low signal-to-noise ratio.

[0068] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A speech noise separation method based on independence analysis in high-noise environments, characterized in that, Includes the following steps: Step 1: Obtain the noisy mixed signal, decompose the noisy mixed signal, and obtain parallel independent waveforms based on convolution inference; Step 2: Evaluate the independence of the acquired independent waveforms and resample them to obtain the improved sampled waveforms with marginal probability distributions: Step 2-1: Normalize the decomposed and reconstructed independent waveform Z: ； ； Where Z represents the independent waveform Z decomposed in step 1. This represents the normalized data, where B is the scaling factor. Represents the expectation used for normalization; Step 2-2: Resample the normalized data: Based on the normalized data The data in each column is shuffled while the row data remains unchanged, and then resampled; the resampled data is the data in... The current row was reselected from all columns of data. The column data is used as newly sampled samples ; Steps 2-3: Simulate the edge distribution based on the resampled waveform Ẑ; Step 3: Construct a discriminator by inputting the independent waveform obtained in Step 1 and the improved sampled waveform obtained in Step 2 into the discriminator to obtain their linear representations. Step 4: Construct a speech noise separation model, and optimize the model using SISNR and GAN based on the discriminator in Step 3. Step 5: Perform noise separation on speech in high-noise environments based on the optimized model.

2. The speech noise separation method in a high-noise environment based on independence analysis according to claim 1, characterized in that, The step 1, obtaining a set of parallel independent waveforms based on convolutional inference, specifically involves: Step 1-1: The mixed signal includes multiple speakers' speech and noisy speech. The speech mixing method includes convolutional mixing, instantaneous mixing, linear mixing, or nonlinear mixing. Steps 1-2: Decompose the mixed signal based on the signal decomposition module.

3. The speech noise separation method based on independence analysis in high-noise environments according to claim 2, characterized in that, The decomposition of the mixed signal based on the signal decomposition module in steps 1-2 is specifically as follows: Step 1-2-1: Convert the mixed signal segments into feature representations based on the hybrid encoder; Step 1-2-2: Learn and generate a mask based on the mask separator, and use the mask to separate the feature representation to obtain the features of each independent signal; The mask separator uses stacked nested one-dimensional convolutional networks and extracts temporal features based on a TCN structure. The input signal is fed through a Batch Normalization layer, then to a BottleNeck layer, and finally... One 1D-Conv layer, repeated After that, enter again A 1D-Conv layer is then fed into a fully convolutional Conv layer, and finally the output is a mask for separating the signal. Steps 1-2-3: Perform one-dimensional linear deconvolution operation on each independent signal feature based on the reconstruction decoder to obtain a set of reconstructed independent waveforms Z.

4. The speech noise separation method in a high-noise environment based on independence analysis according to claim 1, characterized in that, The construction of the GAN network discriminator in step 3 specifically involves: The discriminator consists of a three-layer linear perceptron (MLP), which receives the independent waveform obtained in step 1 and the improved sampled waveform obtained in step 2, and outputs the discriminant vectors of the two as the basis for the GAN loss function.

5. The speech noise separation method in a high-noise environment based on independence analysis according to claim 4, characterized in that, The construction and optimization of the speech noise separation model in step 4 specifically involves: A speech noise separation model is built based on a deep learning network, and it is optimized using adversarial strategy and gradient descent based on the loss function. The loss function is: ； in, and These are used to adjust the importance of SISNR and GAN loss in the total loss function, respectively. This represents the scale-invariant signal-to-noise ratio loss function. Let represent the loss function of a generative adversarial network, where This represents the discriminator in a GAN network. This refers to the GAN network generator.

6. A speech noise separation system for high-noise environments based on independence analysis, characterized in that, Includes the following modules: Mixed signal decomposition module: used to acquire a noisy mixed signal, decompose the noisy mixed signal, and obtain a set of parallel independent waveforms based on convolutional inference; Resampling module: Used to evaluate the independence of the acquired independent waveforms and resample them to obtain improved sampled waveforms with marginal probability distributions. The reconstructed independent waveforms Z are normalized: ； ； Where Z represents the independent waveform Z decomposed in step 1. This represents the normalized data, where B is the scaling factor. Represents the expectation used for normalization; Resampling of normalized data: Based on normalized data The data in each column is shuffled while the row data remains unchanged, and then resampled; the resampled data is the data in... The current row was reselected from all columns of data. The column data is used as newly sampled samples ; Based on the resampled waveform Ẑ, its edge distribution is simulated; Discriminator building module; used to build a discriminator based on the independent waveforms obtained from decomposition and the improved sampled waveforms obtained from resampling; Speech noise separation model building module: used to build a speech noise separation model, and optimize the model based on the discriminator using SISNR and GAN; Noise separation module; used to separate noise from speech in high-noise environments based on an optimized model.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-5.

8. A computer-storable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5.

Images