A method for single-channel speech separation based on deep learning

Abstract

The application provides a kind of method for separating single-channel speech based on deep learning, comprising the following steps: step 1, encoding stage: receiving mixed speech signals collected by a single microphone, and dividing the time-domain waveform of the received mixed speech signals into multiple short segments, then converting the multiple short segments into corresponding high-dimensional feature representation in the intermediate feature space and output to the next step; Step 2, separation stage: receiving the high-dimensional feature representation of step 1 and outputting the mask estimation corresponding to each source signal, and then obtaining the feature representation of the separated source in the feature space to realize separation, thereby obtaining the separation feature, and outputting the separation feature to the next step; Step 3, decoding stage: receiving the separation feature output by step 2 and converting it into the time-domain waveform estimation of the corresponding source signal. The beneficial effects of the application are: 1. The method improves the performance of speech separation.

Description

Technical Field

[0001] This invention relates to the field of speech separation technology, and in particular to a single-channel speech separation method based on deep learning. Background Technology

[0002] Most mainstream speech separation models are temporal separation networks, which achieve speech separation through three stages: encoding, separation, and decoding. To globally model long input sequences, a dual-path structure is generally used, often neglecting the crucial role of local information in speech separation.

[0003] In complex acoustic environments, humans use an auditory selection mechanism to focus their attention on sound signals of interest, thus solving the cocktail party problem. How to design a general auditory model based on machines to flexibly handle the cocktail party problem has been a research hotspot in recent years. This model can serve as a front-end processing module, extracting the target signal from background noise and interference from other speakers. Speech separation has important applications in areas such as automatic meeting transcription, automatic speech recognition, and hearing aid design.

[0004] After more than half a century of research, speech separation has achieved relatively good results in speaker-dependent scenarios. However, in more general scenarios, prior knowledge about the speaker is often unavailable, making speech separation a challenging task. Nevertheless, this is a problem that must be solved to achieve more robust performance and wider application.

[0005] Before the advent of deep learning, three main algorithms were used to solve the speech separation problem: model-based methods, decomposition-based methods, and rule-based methods. Model-based methods model the speech signal and its statistical information. For example, the Factorial Hidden Markov Model (FHMM) represents the historical information of the speech signal using hidden states to model the time series, or Independent Component Analysis (ICA) achieves separation by assuming that the pure signals are statistically independent. Decomposition-based methods assume that speech can be represented by a basis matrix. For example, Non-negative Matrix Factorization (NMF) decomposes the amplitude spectrum of speech into the product of two non-negative matrices and obtains the target source signal by fixing the basis matrix of the mixture's amplitude spectrum. Rule-based methods perform speech separation by manually setting grouping rules. For example, Computational auditory scene analysis (CASA) uses sound features such as pitch or tone as grouping cues to separate auditory streams belonging to different sources. However, these methods rely on strict assumptions and have overly simplistic models, resulting in poor performance in complex auditory environments. At the same time, the huge computational cost also limits their application scenarios.

[0006] With the tremendous success of deep learning in image and natural language processing, deep learning-based speech separation methods have also been extensively studied. Generally, deep learning techniques for single-channel speech separation can be divided into two categories: frequency domain speech separation methods and time domain speech separation methods. However, due to label arrangement problems and output dimension mismatches, they perform poorly in speech separation tasks with independent speakers. The former is usually caused by the disordered nature of the sources in mixed speech while the output is ordered; incorrect arrangement will also produce significant losses even when the separation is correct, causing the network to update parameters in the wrong direction. The latter refers to the inability of networks with fixed output dimensions to handle speaker variations.

[0007] Frequency-domain-based speech separation methods typically utilize Short-Time Fourier Transform (STFT) to represent speech features and achieve separation by estimating the amplitude spectrum of clean speech. For example, Deep Clustering (DC) assigns an embedding vector to each time-frequency point using a neural network, trains it using an objective function based on affinity matrices, and obtains speaker time-frequency masks through clustering operations during the testing phase. Because affinity matrices are permutation-invariant, this method avoids the label alignment problem. Deep Attractor Networks (DANet), based on DC, maps network outputs to target speech by estimating attractor vectors representing each source signal in the embedding space, thus avoiding the label alignment problem. Both of these methods use clustering operations for mask estimation during the testing phase, so the output dimension mismatch problem can be addressed by setting different numbers of clusters. Permutation-invariant training (PIT) calculates the frame-level loss between all outputs and speaker arrangements and uses the minimum loss for parameter updates to solve the label alignment problem, but this method requires an additional speaker tracking step. To further address the issue of frequent switching of optimal arrangements between different frames, we propose Utterance-level permutation invariant training (uPIT). uPIT uses a discourse-level objective function to enforce that all frames use the same permutation, thus solving the speaker tracking problem.

[0008] Frequency-domain-based methods typically operate on the amplitude spectrum of the mixture, which can lead to problems such as suboptimal feature representation, phase reconstruction issues, and high latency due to high-resolution frequency decomposition. In contrast, time-domain-based speech separation methods directly use the time-domain waveforms of the mixed speech as input, with the network outputting waveform estimates for each source signal. This end-to-end operation avoids many of the problems inherent in frequency-domain methods and often achieves superior separation results. The Fully Convolutional Time-Domain Audio Separation Network (Conv-TasNet) is a pioneer in time-domain-based speech separation networks. This model achieves end-to-end speech separation through three steps: encoding, separation, and decoding. Specifically, a linear encoder is used to obtain self-learned feature representations, the separation module outputs mask estimates for each source, which are applied to the encoder output to obtain estimates of the source signals in the feature space. Finally, the decoder returns the results to the time domain. Building upon Conv-TasNet, the Dual-path Recurrent Neural Network (DPRNN) achieves better separation performance with a very small model size by alternately using inter-block RNNs and intra-block RNNs to model long sequences. Subsequently, the Dual-path Transformer Network (DPTNet) replaces the original RNNs with improved transformer encoding modules in a dual-path structure, allowing input elements to interact directly and be processed in parallel. This avoids the suboptimal separation effect caused by the original model requiring a large number of intermediate states for information transmission.

[0009] Time-domain-based speech separation methods have attracted widespread attention since their emergence and have gradually become the mainstream approach. Previous time-domain separation networks have often focused on how to effectively model long sequence inputs, neglecting the important role of local information in speech separation. Summary of the Invention

[0010] To address the problems in existing technologies, this invention provides a method for single-channel speech separation based on deep learning. This invention provides a method for single-channel speech separation based on deep learning, comprising the following steps:

[0011] Step 1, Encoding stage: Receive the mixed speech signal collected by a single microphone, divide the time-domain waveform of the received mixed speech signal into multiple short segments, and then convert the multiple short segments into the corresponding high-dimensional feature representation in the intermediate feature space and output them to the next step;

[0012] Step 2, Separation Stage: Receive the high-dimensional feature representation from Step 1 and output the masking estimate corresponding to each source signal, thereby obtaining the feature representation of the separated source in the feature space to achieve separation, thus obtaining the separation feature, and outputting the separation feature to the next step;

[0013] Step 3, Decoding Stage: Receive the separation features output from Step 2 and convert them into time-domain waveform estimates of the corresponding source signals. As a further improvement of the present invention, Step 1 specifically involves: representing the time-domain waveform of the mixed signal. Divide into multiple short segments of length L by overlapping, each segment being represented as... Where t = 1, 2, ..., Indicates a short segment index. This represents the total number of feature vectors generated from the input signal. A one-dimensional convolutional module is used as the encoder to convert x... t Convert to an N-dimensional vector representation, and use the ReLU activation function to restrict each element in the feature representation to [0, +∞):

[0014]

[0015] in For encoder output, These are the parameters of the filter in the one-dimensional convolution module. As a further improvement of the present invention, in step 2, the separation stage specifically includes: step 20, firstly, performing layer normalization on the received encoder output, and then inputting the normalized feature representation into the bottleneck layer;

[0016] Step 21: The output Z input of the bottleneck layer is separated into a structure composed of stacked dual-path multi-scale delay modules based on Conformer, wherein each DP-MSTD-Conformer module integrates global information and local information at different scales to form the output of the DP-MSTD-Conformer module.

[0017] Step 22: A one-dimensional gated convolutional layer is used to obtain the masking estimate of each source signal in the feature space. Applying this estimate to the high-dimensional feature representation of the mixture yields the corresponding feature estimate. As a further improvement of the invention, the layer normalization operation formula in step 20 is as follows:

[0018]

[0019] in, Let W be the t-th column. The scaling factor is ε, which is a very small positive number and can take the value 10. -8It maintains numerical stability by avoiding the denominator from taking the value of zero. E[·] and Var[·] represent the mean and variance of the calculated vector, respectively.

[0020] As a further improvement of the present invention, the bottom layer and the top layer of the DP-MSTD-Conformer module are each a feedforward module. The feedforward module advances the layer normalization operation and uses the Swish activation function for nonlinear transformation. Dropout operation is added after the Swish activation function and the second linear layer to improve the generalization of the network. In order to learn position information, the first linear layer of the feedforward module is replaced with a recurrent neural network.

[0021] The DP-MSTD-Conformer module finally standardizes the feature dimensions through a normalization layer to obtain the output of the entire DP-MSTD-Conformer module. The output of the entire DP-MSTD-Conformer module will be used as the input of the next DP-MSTD-Conformer module to continue feature learning.

[0022] As a further improvement of the present invention, the DP-MSTD-Conformer module uses an improved dual-path transformer module to learn the global features of long sequences, and the specific steps are as follows:

[0023] Step S1, Segmentation Step: Input the long sequence at P time intervals. Divide into short blocks of length K;

[0024] Step S2, processing steps: Input the segmented features from step S1 into the dual-path structure and perform intra-block feature processing and inter-block feature processing respectively to obtain global features;

[0025] Step S3, Overlapping Accumulation Step: The output of the inter-block improved converter is... Arrange the segments into overlapping short segments according to the segmentation pattern, and perform accumulation operations at the same time step. Finally, delete the feature vector corresponding to the initial zero-fill position to obtain the output of the DPTi module. On this basis, perform residual connection to obtain the output of the entire module.

[0026] As a further improvement of the present invention, in step S1, to ensure that there are no redundant samples remaining and that each sample appears the same number of times in different short blocks, the input sequence is zero-padding before segmentation. The specific steps are as follows:

[0027] The first step is to pad the end of the sequence with M zero-based feature vectors for each time step, where M is calculated as follows:

[0028]

[0029] Where % represents the remainder operation;

[0030] The second step is to fill the input sequence with P zero feature vectors at each time step at both ends.

[0031] As a further improvement of the present invention, step S2 further includes:

[0032] Step S20, Intra-block Feature Processing Step: The intra-block improved transformer processing module models each of the S short blocks individually, that is, in D r Feature learning is carried out on the second dimension, as shown in the following formula:

[0033]

[0034] in f is the output of the submodule within the block. r (·) is a mapping function defined for the improved Transformer encoder;

[0035] Step S21, Inter-block Feature Processing Step: The improved inter-block converter processing module in By learning features along the last dimension, information from all the short blocks can be associated, ultimately yielding an output based on global information:

[0036]

[0037] in For the output of the inter-block submodule, h r (·) denotes the mapping function of the improved Transformer encoder.

[0038] As a further improvement of the present invention, the DP-MSTD-Conformer module uses a multi-scale delay multi-head attention module as a local feature learning module and incorporates it into the Conformer structure to improve the speech separation effect. The specific steps are as follows:

[0039] Step Y1, Segmentation Step: Before segmentation, zero-padding is performed on the long input sequence. After zero-padding, the long input sequence is segmented into lengths of K·2. r-1 A short segment with an overlap rate of half its length;

[0040] Step Y2, multi-scale time delay sampling step: Perform multi-scale time delay sampling in each short segment, and concatenate all the results after time delay sampling of each short segment along a new dimension to obtain the sampled output;

[0041] Step Y3, Sequence Feature Learning Step: The MHSA module is used to learn sequence features in each short block of length K to capture the relationship between the current scale downsampling time steps.

[0042] Q = [y r (H[:,:,s]),s=1,2,…,S r (3-14)

[0043] in This is the output of sequence feature learning. For the s-th short block, y r (.) represents the mapping function corresponding to the MHSA module; it returns a short segment, along the last dimension of the output Q in increments of 2. r-1 Dividing the blocks at intervals, a total of S′ is obtained. r Three-dimensional matrices:

[0044] Q i =Q[:,:,(i-1)·2 r-1 +1: i·2 r-1 ], i = 1, 2, ..., S′ r (3-15)

[0045] in Let be a three-dimensional matrix formed by the i-th block; in the second dimension, K blocks of length 2... r-1 The features are spliced ​​together to obtain

[0046]

[0047] Step Y4: Obtain the long sequence output by overlapping and accumulating.

[0048] As a further improvement of the present invention, in step 3, during the decoding stage, a one-dimensional transposed convolution module is used to reconstruct the time-domain waveform of the corresponding separated signal based on the estimated source signal features. Taking the k-th input signal as an example, the output after the one-dimensional transposed convolution operation is:

[0049]

[0050] in These are the learnable parameters in the transpose convolution module. For the k-th output generated by the separation module, the estimated time-domain waveform is finally obtained by overlapping and accumulating the outputs based on the filter length and stride set in the one-dimensional transpose convolution operation. The specific calculation method is as follows:

[0051]

[0052] in The vector is initialized to all zeros, and l is the stride of the transpose convolution.

[0053] The beneficial effects of this invention are: 1. The method proposed in this invention uses MSTD to sample long sequence inputs, which can be used for local feature learning; 2. This invention can use the improved dual-path module for global feature learning; 3. This invention proposes a Conformer-based dual-path multi-scale time delay network, which integrates global features and multi-scale local features based on the Conformer structure, and can obtain the masking estimate of each source signal in the feature space; 4. Experimental results show that, under the evaluation of multiple performance indicators, the model proposed in this invention has achieved significant performance improvement (reaching 19.09dB, 19.27dB, 3.5789, and 0.9638 on the TIMIT-2mix dataset for SI-SNRi, SDRi, PESQ, and STOI, respectively), and the model has a certain ability to be tested across datasets. Attached Figure Description

[0054] Figure 1 This is a block diagram illustrating the principle of the DP-MSTD-CNet of this invention;

[0055] Figure 2 This is a block diagram illustrating the overall structural principle of the DP-MSTD-Conformer module of this invention;

[0056] Figure 3 This is a block diagram illustrating the FFM principle of the present invention;

[0057] Figure 4 This is a block diagram illustrating the specific operating principle of the DPTi module of this invention;

[0058] Figure 5 This is a block diagram illustrating the structural principle of the Transformer encoder in the DPTNet of this invention.

[0059] Figure 6 This is a block diagram illustrating the structural principle of the Transformer encoder in the DPTi module of this invention.

[0060] Figure 7 This is a schematic diagram of the operation of the MHSA-MSTD module of the present invention;

[0061] Figure 8 This is a separation example diagram on the TIMIT-2mix dataset of this invention;

[0062] Figure 9 This is a separation example diagram on the Libri2Mix dataset of this invention. Detailed Implementation

[0063] likeFigure 1 As shown, this invention discloses a single-channel speech separation method based on deep learning, comprising the following steps:

[0064] Step 1, Encoding stage: Receive the mixed speech signal collected by a single microphone, divide the time-domain waveform of the received mixed speech signal into multiple short segments, and then convert the multiple short segments into the corresponding high-dimensional feature representations in the intermediate feature space and output them to the next step.

[0065] Step 2, Separation Stage: Receive the high-dimensional feature representation from Step 1 and output the masking estimate corresponding to each source signal, thereby obtaining the feature representation of the separated source in the feature space to achieve separation, thus obtaining the separation features, and outputting the separation features to the next step;

[0066] Step 3, Decoding stage: Receive the separation features output from step 2 and convert them into time-domain waveform estimates of the corresponding source signals.

[0067] This invention proposes a Conformer-based dual-path multi-scale delay network. This network learns global information using an improved dual-path module, and then introduces a multi-scale delay module to learn local information of varying coarseness. The network is organized based on the Conformer network structure. Experiments show that the method of this invention not only achieves significant performance improvements in two-speaker mixed speech separation tasks with independent speakers, but also possesses some cross-dataset testing capabilities.

[0068] The following is a detailed description of the present invention:

[0069] 1. Overall structure of a Conformer-based dual-path multi-scale delay network

[0070] DP-MSTD-CNet consists of three parts: an encoder, a separation module, and a decoder, consistent with mainstream temporal separation networks. First, the encoder module transforms short segments of the mixed waveform into corresponding high-dimensional representations in the intermediate feature space through self-learning. Then, the separation stage receives features of the mixed speech signals and outputs mask estimates for each source signal to achieve separation. Finally, the decoding stage is the inverse transform of the encoding stage, used to return temporal waveform estimates for each source signal. The specific design of the entire network is as follows... Figure 1 As shown, detailed descriptions of each part will be provided in subsequent sections.

[0071] 1.1 Single-channel speech separation

[0072] Consider a single-channel speech separation task involving C speakers. Ignoring reverberation and noise interference, the problem can be described as follows:

[0073]

[0074] Where x(t) is the mixed speech signal captured by the microphone at time t, and s k x(t) represents the k-th clean speech signal at time t, and C represents the total number of sound sources in the mixed speech. The goal of single-channel speech separation is to process the mixed speech signal x(t) received by a single microphone according to the model described by formula (2-1), and finally obtain the time-domain waveform estimates of C clean speech signals. The closer the estimated speech signals are to the clean speech signals, the better.

[0075] 1.2 Encoder and Decoder

[0076] The encoder and decoder are symmetrical in structure and function. The former is mainly used to represent the features of the input time-domain signal, while the latter is mainly used to convert the separated features of the output into the time-domain waveform estimation of the corresponding source signal. The existence of these two ensures that the network can decide on the feature representation method itself, and that both the network's input and output are time-domain waveforms, creating conditions for end-to-end training of the network.

[0077] The specific design of the encoder and decoder is as follows: Figure 1 As shown. During the encoding stage, the time-domain waveform of the mixed signal is represented. Divide into multiple short segments of length L by overlapping, each segment being represented as... Where t = 1, 2, ..., Indicates a short segment index. This represents the total number of feature vectors generated from the input signal. A one-dimensional convolutional module is used as the encoder to convert x... t Convert to an N-dimensional vector representation, and use the ReLU activation function to restrict each element in the feature representation to [0, +∞):

[0078]

[0079] in For encoder output, Let be the parameters of the filters in the one-dimensional convolution module, which can be described as a filter bank with N filters of length L. During the decoding stage, the one-dimensional transposed convolution module is used to reconstruct the time-domain waveform of the corresponding separated signal based on the estimated source signal features. Taking the k-th input signal as an example, the output after the one-dimensional transposed convolution operation is:

[0080]

[0081] in These are the learnable parameters in the transpose convolution module. For the k-th output generated by the separation module, the estimated time-domain waveform is finally obtained by overlapping and accumulating the outputs based on the filter length and stride set in the one-dimensional transpose convolution operation. The specific calculation method is as follows:

[0082]

[0083] in The vector is initialized to all zeros, and l is the stride of the transposed convolution. The entire decoding module can be viewed as the inverse transformation of the aforementioned convolutional encoding module.

[0084] 1.3 Separation Module

[0085] The cascaded multi-head self-attention (MHSA) module in the Transformer excels at capturing long-range dependencies and effectively models global information, but it often overlooks the crucial role of local information. CNN networks, due to their fixed receptive field, cannot perform global modeling, but they are highly effective at extracting local information. The Transformer network structure combines the strengths of both, utilizing self-attention mechanisms and convolutional operations to enhance feature learning, and has been successfully applied in the ASR (Automatic Feature Recognition) field. This invention borrows the ideas of the Transformer network to design a temporal separation model, and further improves the global and local feature learning modules. Finally, it summarizes the learning results of the two types of features to achieve the final separation.

[0086] Specifically, the separation module adopts a Conformer-based network structure with two main improvements. First, the MHSA module is replaced with an improved dual-path transformer (DPT) module, which uses intra-block and inter-block transformers alternately for global feature learning. The dual-path approach addresses the difficulty of modeling long input sequences in temporal speech separation and also captures local information to some extent through feature learning between short segments. Second, the CNN network is replaced with a multi-scale time-delay MHSA module for local feature learning. In this module, a multi-scale time-delay sampling segmentation method is proposed, which ensures that local information at different time intervals can be extracted at different stacked layers. Finally, a one-dimensional gated convolutional layer is used to obtain the masking estimate of each source signal in the feature space, which is then applied to the high-dimensional feature representation of the mixture to obtain the corresponding feature estimate. The specific network structure is as follows: Figure 1 As shown, each module will be described in detail in Section 2.

[0087] 2. Separation Module of Conformer-Based Dual-Path Multi-Scale Delay Network

[0088] In the DP-MSTD-CNet model, the main function of the separation module is to output a mask estimate for each source signal, thereby obtaining the feature representation of the separated sources in the feature space. Specifically, it includes the following steps:

[0089] Step 20: First, perform layer normalization on the received encoder output:

[0090]

[0091] in, Let W be the t-th column. The scaling factor is ε, which is a very small positive number, typically taken as 10. -8 It maintains numerical stability by avoiding the denominator from being zero. E[·] and Var[·] represent the mean and variance of the vector, respectively, and the specific calculation method is as follows:

[0092]

[0093]

[0094] The standardized feature representation is input into a bottleneck layer consisting of 11 convolutional layers. This layer determines the feature dimension of subsequent inputs, and the number of its filters is generally set to a small number to reduce the amount of computation.

[0095] Step 21: Next, the output Z of the bottleneck layer is input by a separation structure composed of stacked Conformer-based dual-path multiscale time-delay with conformer (DP-MSTD-Conformer) modules. Each DP-MSTD-Conformer module integrates global information and local information at different scales to form the output of the module. It is the key to the entire DP-MSTD-CNet separation model and will be introduced in detail in the following subsections.

[0096] The output of the last DP-MSTD-Conformer module is Z R+1 First, a nonlinear transformation is performed using the PReLU activation function. Then, the signal is input into a 1×1 convolutional layer to assign a corresponding masking estimate to each source signal. The resulting masking estimate is denoted as... Let's take the k-th masking estimate as an example for illustration. After processing by a one-dimensional gated convolutional layer, the result is...

[0097]

[0098] Where U, The parameters for the 1×1 convolution operation are: Tanh, representing the tanh activation function, which transforms the input to constrain it to the range [-1, 1]; and σ, the sigmoid activation function, which performs gating here to selectively output the input. The final 1×1 convolutional layer transforms the estimated mask back to the original feature dimensions, followed by a ReLU activation function to ensure the mask estimation is non-negative. The resulting mask estimate is applied to the encoder output to obtain the estimate for each source in the feature space.

[0099] Next, Section 2.1 introduces the overall structure of the DP-MSTD-Conformer module, followed by descriptions of its key components. Specifically, Section 2.2 introduces the feedforward module, Section 2.3 introduces the improved dual-path transformer module, and Section 2.4 introduces the multi-scale delay sampling module. Finally, Section 2.5 describes the loss function used for training this network.

[0100] 2.1 Dual-path multi-scale delay module based on Conformer

[0101] The main components of the DP-MSTD-Conformer module are as follows: Figure 2 As shown, it generally follows the architecture of the Conformer model. Specifically, the bottom and top layers of this module are a feedforward module, sandwiched by an improved DPT module and a multi-scale delay MHSA module. Each DP-MSTD-Conformer module corresponds to a delay factor tf = 2. r-1 , where r represents the number of times the module is stacked. Finally, a normalization layer is used to normalize the feature dimensions to obtain the output of the entire module. To avoid degradation during the training of deep neural networks, residual connections are used between each layer, with half-step residual connections used between the first and last feedforward modules. For the input Z of the r-th DP-MSTD-Conformer module... r In other words, the output of this module can be calculated as follows:

[0102]

[0103] Z r " = Z r ′+DPTi(Z r ′) (3-6)

[0104] Z r″′=Zr″+MHSA|-MSTD(Zr″) (3-7)

[0105]

[0106] Here, FFM represents the feedforward module, DPTi represents the improved dual-path Transformer module, MHSA-MSTD represents the multi-head self-attention module under multi-scale time-delay sampling, and LN represents the layer normalization operation. The output of the entire module will be used as the input of the next DP-MSTD-Conformer module for further feature learning.

[0107] 2.2 Feedforward Module

[0108] In the Transformer encoder architecture, a feed-forward module (FFM) is deployed after the MHSA module. It consists of two linear layers and an intermediate nonlinear activation function. The entire FFM is residually connected and followed by a layer normalization operation to obtain the final output. In the DP-MSTD-Conformer module, the FFM follows the structure in the Conformer, advancing the layer normalization operation and using the Swish activation function for nonlinear transformation. The Swish activation function can prevent gradient saturation to some extent, which helps in network optimization. Its calculation formula is as follows:

[0109] f(x) = x· sigmoid(βx) (3-9)

[0110] β can take different values; in this invention, β = 1. In addition, dropout operations are added after the activation function and the linear layer to improve the network's generalization ability.

[0111] In the DP-MSTD-Conformer module, MHSA is used for multi-scale local feature learning. However, the module's input does not include positional encoding, nor does it integrate a relative sinusoidal positional encoding scheme like the multi-head self-attention module in the Conformer. To learn positional information, the first linear layer of the FFM is replaced with a recurrent neural network. The final FFM design in the DP-MSTD-Conformer module is as follows: Figure 3 As shown.

[0112] 2.3 Improved Dual-Path Converter Module

[0113] In the DP-MSTD-Conformer module, an improved dual-path transformer (DPTi) structure is used to learn the global features of long sequences. For example... Figure 4 As shown, DPTi consists of three parts: segmentation, improved dual-path transformer processing, and overlapping accumulation. Its overall structure is roughly the same as the segmented modules of DPTNet. The only difference is that in this invention, multiple DPT modules are not directly stacked here. Instead, after segmentation, only intra-block and inter-block feature learning is performed once, and then the original input size is immediately returned through overlapping accumulation, which is then used as the input of the next module.

[0114] Taking the r-th DPTi module as an example, the entire process is explained in the following steps:

[0115] Step S1, Segmentation Step: First, in the segmentation stage, the long sequence is input at intervals of P time steps. The sequence is divided into short blocks of length K. To ensure that no extra samples remain and that each sample appears the same number of times in different blocks, zero-padding is performed on the input sequence before segmentation. The padding process consists of two steps: First, the end of the sequence is padded with M zero feature vectors at each time step, where M is calculated as follows:

[0116]

[0117] Where % represents the modulo operation; the second step is to fill both ends of the input sequence with P time-step all-zero feature vectors. After the above two padding steps, the input sequence has a total of The input features are segmented into S short blocks of the same shape. s = 1, ..., S, piecing together all the short blocks yields a three-dimensional matrix.

[0118] Step S2, Processing Step: The segmented features are input into a dual-path structure. This structure utilizes two improved transformers to learn intra-block local features and integrate inter-block global information, respectively. The improved transformer design is based on the Transformer encoder and mainly consists of an MHSA module and an FFM module.

[0119] MHSA has many advantages over traditional RNNs and CNNs. Firstly, the dot product operation in attention calculation can directly associate two inputs at arbitrary intervals, easily capturing long-term dependencies in the sequence. Secondly, since the calculation results at each step are independent, calculations at multiple time steps can be processed in parallel, reducing model training time. However, MHSA also has a significant drawback: input sequences with different orders may produce the same output. The main reason for this problem is that the module does not utilize the input positional information, which is crucial for speech separation tasks. To address this issue, the DPTNet model replaces the first linear layer in the feedforward neural network with an RNN to learn positional information. The modified Transformer encoder structure is as follows: Figure 5 As shown.

[0120] In Macaron-Net, replacing the FFM following MHSA with two half-step FFMs, located before and after the attention layer respectively, results in superior performance in machine translation compared to a typical Transformer. Inspired by Macaron-Net and DPTNet, this invention proposes a novel Transformer encoder architecture. Figure 6 As shown, this structure sets up FFM before and after the MHSA module, and performs half-step residual connections in each FFM. The result of feature learning is processed by layer normalization to obtain the output of the entire module. The FFM in the entire encoder structure is consistent with that described in Section 2.2. Here, the original first linear layer is replaced with an RNN to learn positional information.

[0121] Step S20, Intra-block Feature Processing: During the entire dual-path block processing, the segmented features undergo intra-block feature processing and inter-block feature processing sequentially. Since an attention mechanism is used for intra-block feature modeling, the output at each time step is a weighted sum of value vectors, containing all the input information from the entire short block. Therefore, inter-block modeling allows for the learning of features from the entire long sequence, ultimately achieving global modeling.

[0122] First, the improved converter processing module within the block models each of the S short blocks individually, i.e., in D... r Feature learning is carried out on the second dimension, as shown in the following formula:

[0123]

[0124] in f is the output of the submodule within the block. r(·) is the mapping function defined for the improved Transformer encoder. After processing by the Transformer within the block, each time step in the short block carries information from the entire short block.

[0125] Step S21, Inter-block Feature Processing Step: The improved inter-block converter processing module in By learning features along the last dimension, information from all the short blocks can be associated, ultimately yielding an output based on global information:

[0126]

[0127] in For the output of the inter-block submodule, h r (·) denotes the mapping function of the improved Transformer encoder.

[0128] To obtain sequence features, the output D of the inter-block improved transformer is... interr The segments are arranged into overlapping short segments according to the segmentation pattern, and accumulated at the same time step. The specific operation of overlapping accumulation here is the same as in the decoder. Finally, the feature vector corresponding to the initial zero-padding position is deleted to obtain the output of the DPTi module. Residual connection is then performed on this to obtain the final output.

[0129] 2.4 Multi-head self-attention module under multi-scale time delay

[0130] Human cognition is generally hierarchical and abstract, initially focusing on small, subtle local features, then delving deeper layer by layer, and finally building a holistic understanding. While dual-path network structures can acquire global features by scanning in both directions and effectively model long input sequences, their learning of local features is limited to each fixed-length block, thus failing to capture local information at different levels. The DP-MSTD-Conformer module uses a multi-scale delay sampling module for block segmentation. This sampling method, similar to dilated convolution (DC) expanding the receptive field, allows for attention to local information at different scales within blocks of the same length by setting an exponentially increasing delay factor. Within the DP-MSTD-Conformer module, the MHSA mechanism is used to model the information carried in each block. As this module is stacked, the entire model learns features from fine to coarse. Incorporating the Multi-headed Self-Attention with Multiscale Time-Delay (MHSA-MSTD) module as a local feature learning module into the Conformer structure can improve speech separation performance.

[0131] The MHSA-MSTD module mainly consists of four parts: segmentation, multi-scale time-delay sampling, sequence feature learning, and feature merging. The specific operations of each part are as follows: Figure 7 As shown, the specific steps include the following:

[0132] Step Y1, Segmentation: To ensure no samples remain during segmentation and that the total number of short blocks obtained after multi-scale delay sampling for different delay factors is the same, zero-padding is required for the long sequence input before segmentation. After zero-padding, the long sequence input is segmented into lengths of K·2. r-1 The overlap rate is half that of the short segment. The entire process generates a total of S′. r The short segments are denoted as follows:

[0133] Step Y2, Multi-scale Delay Sampling Step: After segmentation, multi-scale delay sampling is performed in each short segment. Taking the i-th short segment as an example, with a delay factor tf = 2... r-1 Feature vectors are extracted from short segments at equal intervals, and the extracted feature vectors are concatenated to form a short block at that time delay scale. Each short segment can form 2 r-1 A short block:

[0134] h i，j =H i [:,j::2 r-1], j = 1, 2, ..., 2 r-1 (3-13)

[0135] in Let j represent the j-th short block obtained by sampling the i-th short segment, where j::2 r-1 This indicates starting from j, with 2 as the starting point. r-1 Take the last value at the interval. Set S′... r The sampled output is obtained by concatenating all the results after each short time delay sampling along a new dimension.

[0136] Step Y3, Sequence Feature Learning Step: Subsequently, the MHSA module is used to learn sequence features in each short block of length K to capture the relationship between the current scale downsampling time steps.

[0137] Q = [y r (H[:,:,s]),s=1,2,...,S r (3-14)

[0138] in This is the output of sequence feature learning. For the s-th short block, y r (.) represents the mapping function corresponding to the MHSA module; it returns a short segment, along the last dimension of the output Q in increments of 2. r-1 Dividing the blocks at intervals, a total of S′ is obtained. r Three-dimensional matrices:

[0139] Q i =Q[:,;,(i-1)·2 r-1 +1: i·2 r-1 ], i = 1, 2, ..., S′ r (3-15)

[0140] in Let i be the three-dimensional matrix formed by the i-th block;

[0141] In the second dimension, K elements of length 2 r-1 The features are spliced ​​together to obtain

[0142]

[0143] Step Y4: Obtain the long sequence output using an overlapping and accumulating method. Since the number of times each time step is used may not be equal, it needs to be divided by the corresponding coefficient. After deleting the feature vectors at the zero-padding positions before segmentation, the output of the entire MHSA-MSTD module is obtained. After residual concatenation, the output can be obtained...

[0144] 2.5 Network Loss Function

[0145] In temporal separation networks, the SI-SNR metric is often used as the loss function. This metric estimates the temporal waveform of the k-th clean speech. Orthogonal decomposition is used to avoid the influence of output size on the index results, and can more accurately measure the magnitude of loss. The specific calculation method of the SI-SNR index of the k-th separated signal is as follows:

[0146]

[0147] This invention, based on SI-SNR, adds a measure of the difference between the sum of the separated signals and the original mixed speech signal:

[0148]

[0149] During training, permutation-invariant training at the discourse level is used to address the label arrangement problem in speaker-independent cases.

[0150] The loss function calculated by formula (3-18) achieved better results in the experiment. It requires, to a certain extent, that the sum of the separated signals be as close as possible to the original mixed signal, which is consistent with the facts. Specifically, the smaller value of the second term in formula (3-18) does not dominate the total loss, but only plays a certain auxiliary role.

[0151] 3. Experimental Results

[0152] 3.1 Dataset

[0153] This invention evaluates the performance of its model on a single-channel speech separation task involving two speakers, based on the TIMIT and Libri2Mix datasets.

[0154] The TIMIT corpus is an acoustic-phoneme continuous speech corpus built collaboratively by institutions such as Texas Instruments and MIT. The primary language used in the corpus is English. This dataset, completed in the United States, consists of 6300 sentences recorded by 438 men and 192 women from eight major dialect regions, with each sentence sampled at 16kHz. Based on the TIMIT corpus, this invention constructs two speaker-mixed datasets (TIMIT-2mix), consisting of a training set, a validation set, and a test set. Each set contains 10,000, 5,000, and 3,000 mixed speech samples, respectively, along with their corresponding clean speech samples. The specific construction method for the mixed speech is as follows: two speech samples are randomly selected from the TIMIT training set, truncated to the same length based on the shorter segment, and then mixed using a random signal-to-noise ratio between -5dB and 5dB to obtain the speech samples in the training and validation sets. The test set is generated in the same manner from 168 previously unseen speakers. Finally, the validation set was used as test data under closed conditions (i.e., speaker-dependent cases), and the test set was used as test data under open conditions (i.e., speaker-independent cases) to compare the performance differences between the two different scenarios. To reduce computational cost, all audio was downsampled to 8kHz.

[0155] The clean train-100 subset of the Libri2Mix dataset was used as the training data for the network. This training set contains 13,900 sentences with a total duration of approximately 58 hours. The validation and test sets each consist of 3,000 sentences with a total duration of approximately 11 hours. Each sentence in the Libri2Mix dataset comes from the corresponding train-clean-100, dev-clean, and test-clean subsets of the LibriSpeech dataset, which contains approximately 110 hours of audio data from 331 speakers, with roughly equal numbers of male and female speakers. Sentences from two different speakers were randomly selected for mixing. Full-scale loudness units (LUFS) were used to scale each sentence, maintaining a uniform loudness distribution between -25 and -33 LUFS for each sentence. During mixing, each sentence was used only once, and the length of the mixed sentence was the same as the shorter sentence before mixing. All audio data were sampled at 8kHz.

[0156] 3.2 Experimental Setup

[0157] Two filters of different lengths are used for encoding and decoding, with a step size of half the filter length. The short segment length in the segmentation operation is set according to the length of the encoded input sequence, and segmentation is performed using a semi-overlapping method. All MHSA modules in the entire model have a head count of 4, and the entire DP-MSTD-Conformer module is stacked 6 times to perform feature learning at different scales. The symbols, descriptions, and values ​​of each parameter in the network are given in Table 3-1.

[0158] Table 3-1 Network Parameter Settings

[0159]

[0160] In the experiment, the training and validation sets of sentences in the TIMIT-2mix dataset were pruned to a length of 2 seconds before training. Two different training generations were set: 70 or 100 generations, to observe the impact of different training generations on separation performance. For the Libri2Mix dataset, sentences were pruned to a length of 2.5 seconds before training, for a total of 70 generations. Adam was used as the network optimizer, and gradient pruning with a maximum 2L norm of 5 was employed. During network training, training was stopped if the loss on the validation set did not decrease for 10 consecutive generations. The learning rate increased linearly in the first 4000 training steps, and then decreased to 0.98 times the current learning rate every two generations in subsequent training steps.

[0161]

[0162] Where n is the number of training steps and en is the number of training algebras.

[0163] 3.3 Evaluation Indicators

[0164] This invention evaluates the performance of speech separation from two aspects. Specifically, it uses Signal-to-Distortion Ratio Improvement (SDRi) and Scale-Invariant Signal-to-Noise Ratio Improvement (SI-SNRi) to measure the degree of distortion of the separated signal compared to the original clean signal; and it uses Short-Time Objective Intelligence (STOI) and Perceptual Evaluation of Speech Quality (PESQ) to objectively evaluate the quality of the separated speech. For all the speech evaluation metrics listed above, higher values ​​indicate better separation performance.

[0165] 3.4 Results

[0166] Figure 8 This is an example of using the DP-MSTD-CNet model for two-speaker mixed speech separation on the TIMIT-2mix dataset. Comparison shows that the time-domain waveforms of the two separated signals obtained by the network are basically consistent with the corresponding target clean signal. For further verification, the difference between the clean signal and the corresponding separated signal is calculated. The results show that the difference also fluctuates around zero, proving that the model proposed in this invention can successfully separate mixed speech and achieve good separation performance.

[0167] Figure 9 This paper demonstrates a single-channel speech separation example using the DP-MSTD-CNet model on the Libri2Mix dataset. The results show that the time-domain waveforms of the two separated signals are essentially identical to their corresponding clean signals, demonstrating the model's good separation performance across different datasets. Similarly, subtracting the corresponding signals reveals a significant difference between the clean signal 2 and the separated signal 2. However, the waveform of the difference is quite similar to that of the clean signal 2, indicating that the difference is simply due to a multiple relationship and does not affect the speech separation performance.

[0168] Table 3-2 presents the performance comparison results of the DP-MSTD-CNet model and the representative temporal separation model in the single-channel speech separation task under different experimental configurations. For the model based on the dual-path structure, experiments were conducted with coding filter lengths of 8 and 16, respectively, and the results after 70 and 100 generations of training are provided. All models were evaluated on the TIMIT-2mix dataset, and the optimal parameter configurations in the literature were used for reproduction. Separation results were tested under CC and OC, respectively. The results show that the SI-SNR values ​​obtained by all experiments in these two cases are not significantly different, indicating that the listed models are effective in speaker-independent cases. In addition, a smaller value of L corresponds to better separation performance. Therefore, when conditions permit, the overall performance of the model can be improved by continuously reducing the length of the coding stage filter. Further comparison of the separation effects after 70 and 100 generations of training reveals that the performance increase of the Conv-TasNet model is not significant, while the dual-path model can achieve a significant improvement in the SI-SNR of the separated signal through continued training. The comparison shows that the DP-MSTD-CNet model significantly outperforms the original temporal separation network and also shows a significant improvement compared to the recently outstanding DPTNet model. Specifically, the DP-MSTD-CNet model achieves SI-SNR values ​​of 19.17 dB and 19.10 dB under CC and OC conditions, respectively, representing an improvement of nearly 1.8 dB compared to the corresponding results of the DPTNet model. Further comparison of the separation results of the DP-MSTD-CNet model after 70 generations of training and other models after 100 generations reveals that the DP-MSTD-CNet model still exhibits a performance improvement of over 1.3 dB, indicating that the network model proposed in this invention can produce better results than other models with fewer training generations, demonstrating a clear advantage.

[0169] Table 3-2 Separation results of each model under different configurations

[0170]

[0171] To investigate the impact of gender on separation performance, Table 3-3 presents the separation performance of each model under same-sex and opposite-sex mixing scenarios based on the TIMIT-2mix dataset. According to the data in Table 3-3, the separation performance under opposite-sex mixing is generally better than that under same-sex mixing. This is likely because the speech patterns of same-sex speakers are more similar, increasing the difficulty of separation. Comparing the absolute and percentage differences in the SI-SNR index of all models under same-sex and opposite-sex mixing scenarios, it is found that the DP-MSTD-CNet model proposed in this invention improves the overall separation performance while reducing the difference in separation performance between the two scenarios.

[0172] Table 3-3 Separation results of each model under homogeneous and heterogeneous mixtures.

[0173]

[0174] We further separated the same-sex and opposite-sex mixtures to explore the average separation performance of different models across all mixture scenarios. The specific results are shown in Tables 3-4. In all models, the separation performance for the two opposite-sex mixtures was not significantly different. However, in the same-sex mixture scenario, the separation performance of the (female-female) mixture was significantly lower than that of the (male-male) mixture. This may be due to uneven distribution of the mixture data. Future research could try increasing the number of samples in the (female-female) mixture to improve this phenomenon.

[0175] Table 3-4 Separation results for different mixing conditions of each model

[0176]

[0177] To demonstrate the effectiveness of the proposed algorithm on other datasets, experiments were conducted on the Libri2Mix dataset. Four additional evaluation metrics—SI-SNRi, SDRi, PESQ, and STOI—were used to quantify the separation performance of all models on the test set. The experimental results are shown in Tables 3-5. Comparison reveals that the improved model proposed in this section still achieves the best separation performance under the four metrics, and shows a significant improvement over the second-best model. Specifically, the DP-MSTD-CNet model achieved SI-SNRi of 14.69 dB, SDRi of 15.13 dB, PESQ of 2.8734, and STOI of 0.9241.

[0178] Table 3-5 Separation performance on the LibriMix dataset

[0179]

[0180] Finally, models trained on the Conv-TasNet, DPTNet, and DP-MSTD-CNet systems were saved on the Libri2Mix dataset and used to test the separation performance of mixed speech on the TIMIT-2mix dataset without any fine-tuning, to verify the cross-dataset test performance of each model. Table 3-6 shows the results of the corresponding experiments. According to the SI-SNR values ​​obtained from the tests, the DP-MSTD-CNet model also achieved the best separation performance in the cross-dataset test.

[0181] Table 3-6 Comparison of cross-dataset testing capabilities of each model

[0182]

[0183] This invention improves the separation module within the framework of the time-domain separation model and proposes a segmentation method for multi-scale time-delay (MSTD) to learn local information at different scales within the same time step.

[0184] The beneficial effects of this invention are: 1. This invention proposes the MSTD method to sample long sequence inputs, which can be used for local feature learning; 2. This invention can use the improved dual-path module for global feature learning; 3. This invention proposes a Conformer-based dual-path multi-scale time-delay network, which integrates global features and multi-scale local features based on the Conformer structure, and can obtain the masking estimate of each source signal in the feature space; 4. Experimental results show that, under the evaluation of multiple performance indicators, the model proposed in this invention has achieved significant performance improvements (reaching 19.09dB, 19.27dB, 3.5789, and 0.9638 on the TIMIT-2mix dataset for SI-SNRi, SDRi, PESQ, and STOI, respectively), and the model has a certain ability to be tested across datasets.

[0185] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for single-channel speech separation based on deep learning, characterized in that, Includes the following steps: Step 1, Encoding stage: Receive the mixed speech signal collected by a single microphone, divide the time-domain waveform of the received mixed speech signal into multiple short segments, and then convert the multiple short segments into the corresponding high-dimensional feature representation in the intermediate feature space and output them to the next step; Step 2, Separation Stage: Receive the high-dimensional feature representation from Step 1 and output the masking estimate corresponding to each source signal, thereby obtaining the feature representation of the separated source in the feature space to achieve separation, thus obtaining the separation feature, and outputting the separation feature to the next step; Step 3, Decoding stage: Receive the separation features output from step 2 and convert them into time-domain waveform estimates of the corresponding source signals; In step 2, the separation stage specifically includes: Step 20: First, perform layer normalization on the received encoder output, and then input the normalized feature representation into the bottleneck layer. Step 21: The output Z input of the bottleneck layer is separated into a structure composed of stacked dual-path multi-scale delay modules based on Conformer, wherein each DP-MSTD-Conformer module integrates global information and local information at different scales to form the output of the DP-MSTD-Conformer module. Step 22: Use a one-dimensional gated convolutional layer to obtain the masking estimate of each source signal in the feature space, and apply it to the high-dimensional feature representation of the mixture to obtain the corresponding feature estimate; The DP-MSTD-Conformer module has a feedforward module at both the bottom and top layers. The feedforward module advances the layer normalization operation and uses the Swish activation function for nonlinear transformation. Dropout operations are added after the Swish activation function and the second linear layer to improve the generalization of the network. In order to learn position information, the first linear layer of the feedforward module is replaced with a recurrent neural network. The DP-MSTD-Conformer module finally standardizes the feature dimensions through a normalization layer to obtain the output of the entire DP-MSTD-Conformer module. The output of the entire DP-MSTD-Conformer module will be used as the input of the next DP-MSTD-Conformer module to continue feature learning. The DP-MSTD-Conformer module uses an improved dual-path transformer module to learn the global features of long sequences. The specific steps are as follows: Step S1, Segmentation Step: Input the long sequence at P time intervals. Divide into short blocks of length K; Step S2, processing steps: Input the segmented features from step S1 into the dual-path structure and perform intra-block feature processing and inter-block feature processing respectively to obtain global features; Step S3, Overlapping Accumulation Step: The output of the inter-block improved converter is... Arrange the segments into overlapping short segments according to the segmentation pattern, and perform accumulation operations at the same time step. Finally, delete the feature vector corresponding to the initial zero-fill position to obtain the output of the DPTi module. On this basis, perform residual connection to obtain the output of the entire module. Step S2 further includes: Step S20, Intra-block Feature Processing Step: The intra-block improved transformer processing module models each of the S short blocks individually, that is, in D r Feature learning is carried out on the second dimension, as shown in the following formula: in f is the output of the submodule within the block. r (·) is a mapping function defined for the improved Transformer encoder; Step S21, Inter-block Feature Processing Step: The improved inter-block converter processing module in By learning features along the last dimension, information from all the short blocks can be associated, ultimately yielding an output based on global information: in For the output of the inter-block submodule, h r (·) denotes the mapping function of the improved Transformer encoder.

2. The method according to claim 1, characterized in that, Step 1 specifically involves: representing the time-domain waveform of the mixed signal. Divide into multiple short segments of length L by overlapping, each segment being represented as... in Indicates a short segment index. This represents the total number of feature vectors generated from the input signal. A one-dimensional convolutional module is used as the encoder to convert x... t Convert to an N-dimensional vector representation, and use the ReLU activation function to restrict each element in the feature representation to [0, +∞): in For encoder output, These are the parameters of the filter in the one-dimensional convolution module.

3. The method according to claim 1, characterized in that, In step 20, the formula for layer normalization is as follows: in, Let W be the t-th column. The scaling factor is ε, which is a very small positive number with a value of 10. -8 It maintains numerical stability by avoiding the denominator from taking the value of zero. E[·] and Var[·] represent the mean and variance of the calculated vector, respectively.

4. The method according to claim 1, characterized in that, In step S1, to ensure that there are no extra samples remaining and that each sample appears the same number of times in different short blocks, the input sequence is zero-padding before segmentation. The specific steps are as follows: The first step is to pad the end of the sequence with M zero-based feature vectors for each time step, where M is calculated as follows: Where % represents the remainder operation; The second step is to fill the input sequence with P zero feature vectors at each time step at both ends.

5. The method according to claim 1, characterized in that, The DP-MSTD-Conformer module uses a multi-scale latency multi-head attention module as a local feature learning module and incorporates it into the Conformer structure to improve speech separation performance. The specific steps are as follows: Step Y1, Segmentation Step: Before segmentation, zero-padding is performed on the long input sequence. After zero-padding, the long input sequence is segmented into lengths of k·2. r-1 A short segment with an overlap rate of half its length; Step Y2, multi-scale time delay sampling step: Perform multi-scale time delay sampling in each short segment, and concatenate all the results after time delay sampling of each short segment along a new dimension to obtain the sampled output; Step Y3, Sequence Feature Learning Step: The MHSA module is used to learn sequence features in each short block of length K to capture the relationship between the current scale downsampling time steps. Q=[y r (H[:,:,s]),s=1,2,…,S r ] (3-14) in This is the output of sequence feature learning. For the s-th short block, y r (.) represents the mapping function corresponding to the MHSA module; it returns a short segment, along the last dimension of the output Q in increments of 2. r-1 Dividing the blocks at intervals, a total of S′ is obtained. r Three-dimensional matrices: Q i =Q[:,:,(i-1)·2 r-1 +1:i·2 r-1 ],i=1,2,…,S′ r (3-15) in Let be a three-dimensional matrix formed by the i-th block; in the second dimension, K blocks of length 2... r-1 The features are spliced ​​together to obtain Step Y4: Obtain the long sequence output by overlapping and accumulating.

6. The method according to claim 1, characterized in that, In step 3, during the decoding stage, a one-dimensional transposed convolution module is used to reconstruct the time-domain waveform of the corresponding separated signal based on the estimated source signal features. Taking the k-th input signal as an example, the output after the one-dimensional transposed convolution operation is: in These are the learnable parameters in the transpose convolution module. For the k-th output generated by the separation module, the estimated time-domain waveform is finally obtained by overlapping and accumulating the outputs based on the filter length and stride set in the one-dimensional transpose convolution operation. The specific calculation method is as follows: in The vector is initialized to all zeros, and l is the stride of the transpose convolution.

Images