An improved sound source separation method of DPRNN

Abstract

The application relates to the field of acoustic signal separation, in particular to an improved DPRNN sound source separation method, which comprises the following steps: acquiring an original mixed signal; constructing an improved DPRNN acoustic signal separation model, wherein the model comprises an encoder, a separation layer and a decoder; processing the original mixed signal by using the encoder to obtain coding features; performing signal separation by using the separation layer according to the coding features, so as to extract independent source signal features from the mixed signal; and finally, restoring the source signal features by using the decoder to output separated acoustic signals. The application realizes the separation of complex acoustic signals, can be well applied to a speech separation task, can provide clear signal data for a fault diagnosis task, and enhances diagnosis accuracy.

Description

Technical Field

[0001] This invention relates to the field of acoustic signal separation, and more particularly to an improved DPRNN method for sound source separation. Background Technology

[0002] In real-world industrial scenarios, multiple machines may operate simultaneously in the same location. Therefore, when collecting acoustic signal data from one machine, data from other machines will also be collected. Directly using this audio data for machine fault diagnosis would significantly impact diagnostic performance. Therefore, separating the audio from the desired machine using acoustic signal separation algorithms not only improves the accuracy of fault diagnosis but also enables targeted noise control.

[0003] Acoustic signal separation algorithms can be mainly divided into separation methods based on signal processing, separation methods based on computer auditory scene analysis, separation methods based on non-negative matrices, and separation methods based on deep learning.

[0004] With the popularization and development of computers, deep learning technology has become a popular means of solving problems. Researchers have applied this technology to the field of acoustic signal separation. By building a deep neural network model for training, the model can automatically learn various effective features of audio to achieve the separation purpose. This method can not only process and analyze complex acoustic signals, but also has relatively strong generalization ability, and can show better performance compared with other methods.

[0005] Traditional neural network-based acoustic signal separation methods, while capable of addressing sound source separation to some extent, still have limitations. Traditional methods typically rely on short-time Fourier transform to convert the mixed acoustic signal into a time-frequency domain representation, then use a time-frequency mask to separate different sound sources, and finally reconstruct the separated acoustic signal using inverse short-time Fourier transform. This process has several major problems:

[0006] While STFT serves as a fundamental transform for acoustic signal separation, it has not been proven to be the optimal choice. Although STFT can effectively represent signals in certain situations, its balance between time and frequency resolution remains unsatisfactory, making it difficult to achieve the best acoustic signal separation results.

[0007] STFT converts the acoustic signal to the complex domain, requiring simultaneous processing of both amplitude and phase information during separation. However, modifying phase information is quite complex, so most methods only apply a time-frequency mask to the amplitude spectrum, keeping the phase unchanged. Although some studies have attempted to optimize phase information simultaneously, separation performance remains limited due to processing accuracy issues. STFT requires high time-frequency resolution, and the choice of window length directly affects the time-frequency characteristics of the signal; an inappropriate choice will still impact the separation accuracy of the acoustic signal. Summary of the Invention

[0008] To address the problem of low sound signal separation accuracy, this invention provides an improved sound source separation method for DPRNN, mainly comprising:

[0009] S1: Acquire the raw mixed signal;

[0010] S2: Construct an improved DPRNN acoustic signal separation model, which includes an encoder, a separation layer, and a decoder. The encoder is used to process the original mixed signal to obtain coded features.

[0011] S3: Based on the encoded features, a separation layer is used to separate the signal and extract the features of each independent source signal; the separation layer includes a segmentation module, an improved DPRNN processing module, and a separation and restoration module; the improved DPRNN processing module includes Bi-LSTM, a fully connected layer, efficient channel attention ECA, and efficient multi-scale attention EMA;

[0012] S4: The source signal features are restored by the decoder, and the separated acoustic signal is output. After linear layer or transpose convolution operation on the acoustic signal, the reconstructed target acoustic signal is obtained.

[0013] A computer device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0014] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above method.

[0015] A computer program product includes a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0016] The beneficial effects of the technical solution provided by this invention are as follows: This invention adopts a time-domain-based acoustic signal modeling strategy, which directly processes acoustic signals in the time domain, avoiding the limitations of the traditional STFT framework. This allows for more accurate capture of the temporal characteristics of the signal, reducing reliance on time-frequency analysis. It not only effectively reduces signal separation delay but also improves separation accuracy, making it particularly suitable for complex industrial equipment acoustic signal separation tasks. This invention can also better capture long-term dependency information, effectively reducing the loss of temporal information and enhancing the model's understanding of complex signals, thereby improving acoustic signal separation performance. By dynamically adjusting the contribution of each channel through ECA, the model's ability to focus on key features is improved overall, further optimizing the separation effect. Using an EMA parallel convolution structure to process input data helps the model capture the interrelationships between features at different scales, thereby improving model performance. This invention achieves the separation of complex acoustic signals, enabling the separation of different types of signals. It can be well applied to speech separation tasks and provides clear signal data for fault diagnosis tasks, enhancing diagnostic accuracy. This method is applicable to speech separation, multi-device sound source separation, and other aspects, possessing broad applicability. Attached Figure Description

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0018] Figure 1 This is a flowchart of an improved DPRNN sound source separation method according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the segmentation module processing procedure in an embodiment of the present invention;

[0020] Figure 3 This is a flowchart illustrating the structure of the improved DPRNN processing module in an embodiment of the present invention. Detailed Implementation

[0021] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] Example 1

[0023] Please refer to Figure 1 , Figure 1 This is a flowchart of an improved DPRNN sound source separation method according to an embodiment of the present invention, specifically including:

[0024] By adding an efficient channel attention module (ECA) and an efficient multi-scale attention module (EMA) after the DPRNN module, the network's ability to perceive information is enhanced, while the network can still maintain good robustness in the face of complex environmental noise.

[0025] S1: Acquire the raw mixed signal.

[0026] S2: Construct an improved DPRNN acoustic signal separation model, which includes an encoder, a separation layer, and a decoder. The encoder is used to process new features to obtain encoded features.

[0027] The encoder structure of this invention extracts features in the time domain through a convolutional network, implicitly learning the optimal basis functions and feature representations, thus avoiding information loss and computational complexity that may occur during cross-domain transformation. Therefore, this time-domain coding method can model the original signal more directly without explicitly converting the signal to the frequency domain, resulting in greater efficiency during processing.

[0028] S3: The separation layer is used to separate the encoded features from the source signals. The purpose is to extract the individual source signal features from the original mixed signal.

[0029] The separation layer consists of a segmentation module, an improved DPRNN processing module, and a separation and restoration module. The segmentation module's processing procedure is as follows: Figure 2 As shown, the continuous input encoding sequence, i.e. the original mixed signal W∈R, is first encoded. N×L The data is divided into multiple overlapping blocks, where L is the feature length of each channel, P is the number of blocks, and K is the length of each block. For blocks with insufficient length, zero-padding is used. Then, all blocks are concatenated to form a three-dimensional tensor D∈R. N×K×P N is the number of output channels in the encoder.

[0030] The features processed by the segmentation module are then fed into the improved DPRNN processing module for further processing. The flowchart of the improved DPRNN processing module is shown below. Figure 3 As shown, the improved DPRNN processing module consists of a bidirectional long short-term memory network (Bi-LSTM), a fully connected layer, an efficient channel attention module (ECA), and an efficient multi-scale attention module (EMA).

[0031] Bi-LSTM provides richer contextual information when modeling long-term dependencies, improving the learning ability of sequence features. By combining the outputs of forward and backward propagation, Bi-LSTM can provide more accurate prediction results and better adapt to complex time series modeling tasks. Furthermore, the Bi-LSTM structure is highly adaptable, capable of handling dependencies of varying lengths, and excels in global context modeling. Therefore, Bi-LSTM demonstrates stronger modeling capabilities in tasks requiring global contextual information, such as speech recognition, natural language processing, speech separation, and time series prediction, significantly improving processing accuracy, robustness, and generalization ability.

[0032] After Bi-LSTM processing, the weights of each channel are dynamically adjusted and output using ECA, enabling adaptive optimization of the feature representation and improving the quality of acoustic signal separation. First, adaptive global average pooling is used to calculate the average value of each channel's elements, obtaining the statistical features of each channel and thus providing a global channel representation. ECA abandons channel dimensionality reduction and introduces a local cross-channel interaction strategy based on 1D convolution to model the local dependencies between channels. This method adaptively selects the size of the convolution kernel, ensuring efficiency and capturing sufficient inter-channel dependencies.

[0033] EMA is a parallel attention mechanism. Compared with traditional convolutional neural networks, EMA uses a parallel convolutional structure to process input data. This structure can train models faster when dealing with large-scale data and improve accuracy by processing features in parallel at different scales.

[0034] The separation and restoration module takes the output of the improved DPRNN processing module as input. First, it performs feature separation on the DPRNN through convolutional layers to learn the feature mask of the acoustic signal. Then, it restores the mask by overlapping and adding the features, and converts it into a feature tensor of the same dimension as the encoder output. Finally, it combines the mask with the features learned by the encoder using a dot product operation to obtain the separated acoustic signal features.

[0035] S4: The source signal features are restored by the decoder, and the separated acoustic signal is output. After performing linear layer or transposed convolution operation on the acoustic signal, the reconstructed target acoustic signal is obtained. The target acoustic signal refers to the ideal acoustic signal recovered or output by the network.

[0036] Compared to the encoder, the decoder's task is to reconstruct a one-dimensional time-domain waveform from the estimated coded features. This part is typically implemented using one-dimensional transposed convolutions or linear layers. The computation process of the decoder can be represented as follows:

[0037]

[0038] In the formula, The target acoustic signal features estimated by the acoustic signal extraction network are represented by V, where V represents the vector or basis function of the encoder convolution. This represents the estimated target acoustic signal waveform.

[0039] After passing through linear layers or transposed convolution operations, the decoder output typically undergoes a nonlinear mapping using the ReLU activation function to enhance the model's fitting ability. Finally, all overlapping segments need to be reconstructed to output the target acoustic signal waveform, avoiding boundary effects or distortions that may occur in traditional methods.

[0040] The encoder and decoder structure can automatically learn the optimal feature representation from the original time-domain signal, which avoids the computational overhead of frequency domain conversion and reduces the limitations of manually designed features, making its processing method more flexible and efficient in dealing with complex signal separation tasks.

[0041] The loss function used in this invention is a weighted loss function with speech-level permutation invariance training. It calculates the loss of each sound signal separated by the improved DPRNN sound signal separation model according to different labels, and selects the combination with the smallest loss as the final reference sound signal. This reference signal is the input signal used in network training. As the network depth increases, the acquired information becomes closer to the sound signal separation task. The sound signal separation task refers to the process where, as the network depth increases, the final sound signal becomes closer to a clean signal. Therefore, in the weighted scale-invariant signal-to-noise ratio loss function, greater weights are assigned to the outputs of deeper layers. That is, using formula (5), different weights are assigned to the outputs of each layer by accumulating the loss and weighting values ​​of each layer. The calculation formula is as follows:

[0042]

[0043] In the formula, Let and s represent the estimated acoustic signal and the original clean acoustic signal, respectively. The clean acoustic signal waveform is used as the training target to calculate the loss of each layer. This multi-stage aggregation training strategy helps the network adaptively select useful information.

[0044] Example 2

[0045] A computer device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0046] Example 3

[0047] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above method.

[0048] Example 4

[0049] A computer program product includes a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An improved DPRNN sound source separation method, characterized in that, include: S1: Acquire the raw mixed signal; S2: Construct an improved DPRNN acoustic signal separation model, which includes an encoder, a separation layer, and a decoder. The encoder is used to process the original mixed signal to obtain coded features. S3: Based on the encoded features, a separation layer is used to separate the signal and extract the features of each independent source signal; the separation layer includes a segmentation module, an improved DPRNN processing module, and a separation and restoration module. The improved DPRNN processing module includes Bi-LSTM, a fully connected layer, efficient channel attention ECA, and efficient multi-scale attention EMA. In S3, the separation and restoration module takes the output of the improved DPRNN processing module as input, and first performs feature separation on it through a convolutional layer to learn the feature mask of the acoustic signal. Subsequently, the mask is restored by overlapping and adding, and then converted into a feature tensor with the same dimension as the encoder output. The dot product operation is used to combine the mask with the features learned by the encoder to obtain the separated acoustic signal features. Finally, a linear layer is used to perform a linear transformation to reconstruct the target acoustic signal. S4: The source signal features are restored by the decoder, and the separated acoustic signal is output. After linear layer or transposed convolution operation on the acoustic signal, the reconstructed target acoustic signal is obtained. In S4, The separated acoustic signals are processed by calculating the loss of each signal according to its label. The combination of signals with the smallest loss is selected as the final reference signal. As the network depth increases, the information obtained becomes closer to the acoustic signal separation task. Therefore, by accumulating the loss and weighting values ​​of each layer, different weights are assigned to the output of each layer. The calculation formula is as follows: In the formula, Indicates loss, and These represent the estimated acoustic signal and the original clean acoustic signal, respectively. Indicates the target signal. Indicates noise signal, This represents the signal-to-noise ratio loss during signal enhancement, and I represents the total number of training stages.

2. The improved DPRNN sound source separation method as described in claim 1, characterized in that, In S4, The decoder's calculation process is as follows: In the formula, This represents the target acoustic signal features estimated by the acoustic signal extraction network. The vector or basis function representing the encoder convolution. This represents the estimated target acoustic signal waveform.

3. The improved DPRNN sound source separation method as described in claim 1, characterized in that, In S4, the process of ECA dynamically adjusting the weights of each channel is as follows: by using adaptive global average pooling to calculate the average value of the elements of each channel of the feature, the statistical features of each channel are obtained, thereby providing a global channel representation.

4. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes a computer program to implement the steps of the improved DPRNN sound source separation method according to any one of claims 1-3.

5. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the steps of the improved DPRNN sound source separation method according to any one of claims 1-3.

6. A computer program product, characterized in that, Includes a computer program or instructions that, when executed by a processor, implement the steps of the improved DPRNN sound source separation method according to any one of claims 1-3.

Images