One-dimensional time series signal enhancement method based on double-branch convolution enhanced window attention

Abstract

This invention belongs to the field of speech signal processing technology, specifically relating to a one-dimensional temporal signal enhancement method based on bi-branch convolution with enhanced window attention. The method comprises five steps: collecting clean and noisy one-dimensional temporal signals and performing preprocessing; constructing a noisy one-dimensional temporal signal X; constructing a one-dimensional temporal signal enhancement network; training the one-dimensional temporal signal enhancement network; and enhancing the actual one-dimensional temporal signal. These steps address the limitations of traditional Transformers in capturing local contextual information and the high computational complexity in calculating global attention, enabling better enhancement of one-dimensional temporal signals.

Description

Technical Field

[0001] This invention belongs to the field of speech signal processing technology, specifically relating to a one-dimensional temporal signal enhancement method based on bi-branch convolution to enhance window attention. Background Technology

[0002] In recent years, the rapid development of deep learning and high-performance computing has brought significant progress to one-dimensional time-series signal enhancement techniques. Especially in the field of deep neural networks (DNNs), models such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have achieved groundbreaking results. Although a large amount of research has attempted to build deep learning models in the time domain of one-dimensional signals, researchers have found in experiments that processing one-dimensional time-series signals using the Short-Time Fourier Transform (STFT) yields more significant enhancement results.

[0003] Early research focused primarily on processing amplitude information, using the phase information of noisy one-dimensional time-series signals for audio reconstruction. However, this approach overemphasized amplitude information, neglecting the importance of phase information. Recent research indicates that both amplitude and phase information play crucial roles in the perceptual quality and intelligibility of time-series signals; therefore, researchers have begun to focus on methods that simultaneously estimate amplitude and phase.

[0004] Phase-Sensitive Mask (PSM) was an early method that incorporates phase information into mask estimation. Further developments included Complex-Valued Ratio Mask (CRM) for estimating real and complex components. This algorithm effectively reconstructs speech by simultaneously enhancing both clean and noisy one-dimensional temporal signals, segmenting the spectrogram into real and imaginary parts.

[0005] In recent years, the development of deep and complex networks has driven further improvements in the performance of one-dimensional temporal signal enhancement. However, these networks rely too heavily on the representational capabilities of convolutional layers. To address this issue, attention mechanisms have been introduced, which can fuse spatial and channel information to enhance the representational capabilities of convolutional layers. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a one-dimensional temporal signal enhancement method based on bi-branch convolution to enhance window attention.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A one-dimensional temporal signal enhancement method based on bi-branch convolution enhanced window attention includes the following steps:

[0009] Step 1: Collect clean one-dimensional time series signals and noisy one-dimensional time series signals and perform preprocessing;

[0010] Step 2: Construct a noisy one-dimensional time series signal X by combining the preprocessed clean one-dimensional time series signal with the noisy one-dimensional time series signal. Obtain the frequency domain representation of the noisy one-dimensional time series signal X through short-time Fourier transform: Mag=[B,C,H,W], Phase=[B,C,H,W];

[0011] Step 3: Construct a one-dimensional temporal signal enhancement network based on dual-branch convolutional enhanced window attention, including an encoder module, a convolutional enhanced window attention module, a decoder module, and a CRM mask module;

[0012] Step 4: Train the one-dimensional time-series signal enhancement network obtained in Step 3;

[0013] Step 5: Use the trained one-dimensional time series signal enhancement network to enhance the actual one-dimensional time series signal to obtain the estimated clean signal.

[0014] Furthermore, the specific steps of the preprocessing in step 1 are as follows:

[0015] Step 1.1: Resample the original clean one-dimensional time-series signal and the noisy one-dimensional time-series signal with a sampling rate of 48000Hz to 16000Hz. The purpose is to reduce the frequency and the amount of data.

[0016] Step 1.2: Use zero-padding to fill the signal to 10s.

[0017] Furthermore, the encoder module consists of two branches, which are used to predict the amplitude and phase of the signal, respectively. These two branches have the same network architecture, which includes 2D convolutional layers, BN layers and ReLU layers. The downsampling operation is implemented through the 2D convolutional layers. After the convolution operation is completed in each branch, a connection is established with the other branch through the complex value calculation module.

[0018] The decoder module consists of two branches, which are used to estimate the amplitude and phase of the signal, respectively. These two branches have the same network architecture, which includes a 2D deconvolution layer, a BN layer, and a ReLU layer. After the deconvolution operation is completed in each branch, a connection is established with the other branch through a complex value calculation module. After passing through the last 2D deconvolution layer, the estimated amplitude and phase information are constrained by the hyperbolic tangent function to obtain the constrained amplitude and phase information.

[0019] The convolutional enhancement window attention module includes a non-overlapping window attention module and a convolutional enhancement forward propagation module. The non-overlapping window attention module adopts a multi-head self-attention mechanism for non-overlapping windows. The convolutional enhancement forward propagation module includes a convolutional enhancement module and an LN layer. The convolutional enhancement module has an asymmetric structure, including a fully connected layer, an activation function, a transpose, a 2D convolutional layer, an activation function, a transpose, and a fully connected layer connected in sequence.

[0020] The CRM mask module obtains a mask M based on the constrained amplitude and phase information. The mask M is then multiplied by the frequency domain representation of the noisy one-dimensional time-series signal X, and converted back to the time domain representation through inverse short-time Fourier transform.

[0021] Furthermore, the multi-head self-attention mechanism of the non-overlapping window is specifically as follows:

[0022] Based on the window size M, the feature information H and W in the signal X = [B, C, H, W] are padded with zeros. Then, the signal X is divided into non-overlapping windows of size M × M, and a multi-head self-attention mechanism is executed within each window to obtain translation and transpose features from each window.

[0023] Assuming the multi-head self-attention mechanism has K heads, the non-overlapping window calculation process for the Kth head is described as follows:

[0024] X = {X 1 ,X 2 ,…,X N}, N=HW / M 2 ,

[0025]

[0026]

[0027] in These represent the query for the Kth head, key, and value, respectively. This represents the output of the Kth head. Finally, the outputs of all heads will be concatenated and a linear transformation will be performed to obtain the final result of the multi-head attention process.

[0028] Furthermore, the convolutional enhanced window attention module is represented as follows:

[0029] X′ l =W-MSA(LN(X) l-1 ))+X l-1

[0030] X l =CE(LN(X′) l ))+X′ l

[0031] Where X′ l and X l represents the outputs of the non-overlapping window attention module and the convolutional enhanced forward propagation module, respectively, and LN represents normalization.

[0032] Furthermore, the specific steps for training the one-dimensional time-series signal enhancement network obtained in step 3 in step 4 are as follows:

[0033] Step 4.1: Input the frequency domain representation of the one-dimensional time-series signal into the one-dimensional time-series signal enhancement network in batches;

[0034] Step 4.2: Output the enhanced one-dimensional timing signal;

[0035] Step 4.3: Perform error calculation on the enhanced one-dimensional time series signal and the input one-dimensional time series signal using a joint loss function;

[0036] Step 4.4: Backpropagate the error and update the parameters of the one-dimensional time-series signal enhancement network;

[0037] Step 4.5: Repeat steps 4.1 to 4.4 until the convergence condition is met.

[0038] Furthermore, the joint loss function is established by introducing an amplitude loss function and a phase loss function, and combining them with a time-domain information loss function, as follows:

[0039] Loss=SI-SNR+Loss_D_R+Loss_D_I

[0040] Where SI-SNR is the time-domain information loss function, Loss_D_R is the magnitude loss function, and Loss_D_I is the phase loss function;

[0041] The SI-SNR is represented as:

[0042]

[0043] Among them S and Let represent the pure and estimated time-domain waveforms, respectively; <·, ·> represent the dot product between two vectors; and ||·||2 represents the L2 norm.

[0044] The calculation steps for Loss_D_R are as follows: First, the enhanced amplitude information and the amplitude information of the clean signal are input into the D_Real function. The D_Real function contains five convolutional layers and two attention layers. Except for the last convolutional layer, all convolutional layers use the Leaky ReLU activation function for nonlinear transformation to obtain Loss_D_R, as shown below:

[0045] Loss_D_R=((D_real(mag_enh)-s) 2 +(D_real(mag_cle)) 2 ) / 2

[0046] Where mag_enh represents the amplitude information of the enhanced signal, mag_cle represents the amplitude information of the clean signal, and s represents an all-1 vector;

[0047] The calculation steps for Loss_D_I are as follows: First, the enhanced phase information and the phase information of the clean signal are input into the D_Imag function. The D_Imag function contains five convolutional layers and two attention layers. Except for the last convolutional layer, all convolutional layers use the Leaky ReLU activation function for nonlinear transformation to obtain Loss_D_I, as shown below:

[0048] Loss_D_I=((D_imag(phase_enh)-s) 2 +(D_imag(phase_cle)) 2 ) / 2

[0049] Where phase_enh represents the phase information of the enhanced signal, phase_cle represents the phase information of the clean signal, and s represents an all-1 vector.

[0050] Compared with the prior art, the present invention has the following advantages:

[0051] (1) This method proposes a convolution-enhanced window attention module to better capture the features of one-dimensional temporal signals. The design of this module aims to solve the problems of insufficient ability of traditional Transformer to capture local context information and high computational complexity when calculating global attention.

[0052] (2) The superiority of the proposed convolutional enhancement module was verified through ablation experiments. Furthermore, the method was compared with state-of-the-art algorithms on the public VCTK corpus. The results show that the proposed method has significant performance advantages and potential, and can better enhance one-dimensional time-series signals. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of a one-dimensional temporal signal enhancement network based on bi-branch convolution to enhance window attention;

[0054] Figure 2 This is a graph of the convolutional layer parameters in the encoder-decoder module;

[0055] Figure 3 This is a schematic diagram of the convolution-enhanced window attention module;

[0056] Figure 4 This is a schematic diagram of a non-overlapping window attention module;

[0057] Figure 5 This is a schematic diagram of the convolutional augmentation forward propagation module;

[0058] Figure 6 It refers to the arrangement of the five convolutional layers and two attention layers in the D_Real and D_Imag functions. Detailed Implementation

[0059] To further illustrate the technical solution of the present invention, the present invention will be further described below through embodiments.

[0060] Example 1

[0061] A one-dimensional temporal signal enhancement method based on bi-branch convolution enhanced window attention includes the following steps:

[0062] Step 1: Collect clean one-dimensional time series signals and noisy one-dimensional time series signals and perform preprocessing;

[0063] Step 1.1: Resample the original clean one-dimensional time-series signal and the noisy one-dimensional time-series signal with a sampling rate of 48000Hz to 16000Hz. The purpose is to reduce the frequency and the amount of data.

[0064] Step 1.2: The signal was padded to 10s using a zero-padding method, employing a Hanning window of 40ms size and a frame length of 10ms.

[0065] Step 2: Construct a noisy one-dimensional time series signal X by combining the preprocessed clean one-dimensional time series signal with the noisy one-dimensional time series signal. Obtain the frequency domain representation of the noisy one-dimensional time series signal X through short-time Fourier transform: Mag=[B,C,H,W], Phase=[B,C,H,W];

[0066] Step 3: Construct a one-dimensional temporal signal enhancement network based on bi-branch convolution to enhance window attention (e.g., Figure 1 As shown), it includes an encoder module, a convolutional enhanced window attention module, a decoder module, and a CRM mask module;

[0067] The encoder-decoder module is designed based on the DCUNET model, but with the difference of separating amplitude and phase information and modeling them separately. The specific operation is as follows:

[0068] The encoder module consists of two branches, which are used to predict the amplitude and phase of the signal, respectively. These two branches have the same network architecture, which includes 2D convolutional layers, BN layers and ReLU layers. The downsampling operation is implemented through the 2D convolutional layers. After the convolution operation is completed in each branch, a connection is established with the other branch through the complex value calculation module.

[0069] This process can be represented as:

[0070] Mag=Conv2d(Mag)-Conv2d(Phase),

[0071] Phase=Conv2d(Mag)+Conv2d(Phase),

[0072] Here, Mag represents amplitude information, Phase represents phase information, and Conv2d represents a 2D convolutional layer. The encoder module implements the interaction between amplitude and phase information. Subsequently, the amplitude and phase information are processed through batch normalization (BN) layers and ReLU layers.

[0073] The decoder module consists of two branches, which are used to estimate the amplitude and phase of the signal, respectively. These two branches have the same network architecture, which includes a 2D deconvolution layer, a BN layer, and a ReLU layer. After the deconvolution operation is completed in each branch, a connection is established with the other branch through a complex value calculation module. After passing through the last 2D deconvolution layer, the estimated amplitude and phase information are constrained by the hyperbolic tangent function to obtain constrained amplitude and phase information, so as to better estimate the application of the mask M in the CRM mask module.

[0074] In designing the encoder-decoder module, we referenced the encoder-decoder operations of DCUNET. For example... Figure 2 As shown, a 10-layer encoder was designed, which indicates the number of convolutional layers used and the parameters of the convolution. Each convolutional operation is followed by BN and ReLU processing.

[0075] like Figure 3As shown, the convolution-enhanced window attention module includes a non-overlapping window attention module (Window-based Multi-head Self-Attention, W-MSA) and a convolution-enhanced forward propagation module (CEFP).

[0076] The convolution-enhanced window attention module is represented as follows:

[0077] X′ l =W-MSA(LN(X) l-1 ))+X l-1

[0078] X l =CE(LN(X′) l ))+X′ l

[0079] Where X′ l and X l represents the outputs of the non-overlapping window attention module and the convolutional enhanced forward propagation module, respectively, and LN represents normalization.

[0080] The non-overlapping window attention module employs a multi-head self-attention mechanism for non-overlapping windows (e.g., ...). Figure 4 As shown in the figure, by computing the self-attention mechanism within the non-overlapping window, the computational cost is effectively reduced, enabling the attention mechanism to focus more accurately on local contextual information and improving the model's performance.

[0081] The multi-head self-attention mechanism for the non-overlapping window is as follows:

[0082] Based on the window size M, the feature information H and W in the signal X = [B, C, H, W] are padded with zeros to align them to a multiple of the window size. Then, the signal X is divided into non-overlapping windows of size M × M, and a multi-head self-attention mechanism is executed within each window to obtain translation and transpose features from each window.

[0083] Assuming the multi-head self-attention mechanism has K heads, the non-overlapping window calculation process for the Kth head is described as follows:

[0084] X = {X 1 ,X 2 ,…,X N}, N=HW / M 2 ,

[0085]

[0086]

[0087] in These represent the query (Q), key (K), and value (V) of the Kth header, respectively. This represents the output of the Kth head. Finally, the outputs of all heads will be concatenated and a linear transformation will be performed to obtain the final result of the multi-head attention process.

[0088] The convolutional enhancement forward propagation module (such as...) Figure 5 As shown, it includes a Convolution Enhancement (CE) module and an LN layer to normalize the distribution of data features and ensure its stability.

[0089] Capturing local contextual information in traditional Transformer modules presents numerous challenges. However, for one-dimensional temporal signal enhancement tasks, neighboring temporal information is crucial. Therefore, a convolutional enhancement module is proposed to effectively capture local contextual information and better integrate it. This convolutional enhancement module has an asymmetric structure, comprising a fully connected layer, an activation function, a transpose, a 2D convolutional layer, an activation function, a transpose, and a fully connected layer connected in sequence.

[0090] Ablation experiments demonstrate that introducing a convolutional enhancement module into the forward propagation module significantly improves model performance. This result shows that applying the convolutional enhancement module to tasks involving capturing local contextual information has a significant positive impact on enhancing model performance. This further validates the effectiveness of the convolutional enhancement module in one-dimensional temporal signal enhancement.

[0091] The CRM mask module obtains a mask M based on the constrained amplitude and phase information. The mask M is then multiplied by the frequency domain representation of the noisy one-dimensional time-series signal X, and converted back to the time domain representation through inverse short-time Fourier transform.

[0092] Given a noisy one-dimensional time-series signal x(n) = y(n) + z(n), where x(n) represents the noisy signal, y(n) represents the clean signal, and z(n) represents the noisy signal, the task of one-dimensional time-series signal enhancement is to estimate y(n) from x(n). After performing a short-time Fourier transform on x and y, we obtain X = Xn. r +jX i Y = Y r +jY i ,

[0093] CRM can be defined as:

[0094]

[0095] To achieve this goal, we employ the Signal Approximation (SA) method, directly minimizing the difference between the amplitude or complex spectrogram of a pure one-dimensional time-series signal and the amplitude or complex spectrogram of masked noise. We define the loss function as: The specific formula is expressed as follows:

[0096]

[0097] Step 4: Train the one-dimensional time-series signal enhancement network obtained in Step 3;

[0098] Step 4.1: Input the frequency domain representation of the one-dimensional time-series signal into the one-dimensional time-series signal enhancement network in batches;

[0099] Step 4.2: Output the enhanced one-dimensional timing signal;

[0100] Step 4.3: Perform error calculation on the enhanced one-dimensional time series signal and the input one-dimensional time series signal using a joint loss function;

[0101] Step 4.4: Backpropagate the error and update the parameters of the one-dimensional time-series signal enhancement network;

[0102] Step 4.5: Repeat steps 4.1 to 4.4 until the convergence condition is met.

[0103] Furthermore, the joint loss function is established by introducing an amplitude loss function and a phase loss function, and combining them with a time-domain information loss function, as follows:

[0104] Loss=SI-SNR+Loss_D_R+Loss_D_I

[0105] Where SI-SNR is the time-domain information loss function, Loss_D_R is the magnitude loss function, and Loss_D_I is the phase loss function;

[0106] The SI-SNR is represented as:

[0107]

[0108] Among them S and Let represent the pure and estimated time-domain waveforms, respectively; <·, ·> represent the dot product between two vectors; and ||·||2 represents the L2 norm.

[0109] The calculation steps for Loss_D_R are as follows: First, the enhanced amplitude information and the amplitude information of the clean signal are input into the D_Real function. The D_Real function contains five convolutional layers and two attention layers, arranged as follows: Figure 6As shown, except for the last convolutional layer, all convolutional layers use the Leaky ReLU activation function for nonlinear transformation, resulting in Loss_D_R, as shown below:

[0110] Loss_D_R=((D_real(mag_enh)-s) 2 +(D_real(mag_cle)) 2 ) / 2

[0111] Where mag_enh represents the amplitude information of the enhanced signal, mag_cle represents the amplitude information of the clean signal, and s represents an all-1 vector;

[0112] The calculation steps for Loss_D_I are as follows: First, the enhanced phase information and the phase information of the clean signal are input into the D_Imag function. The D_Imag function contains five convolutional layers and two attention layers. Except for the last convolutional layer, all convolutional layers use the Leaky ReLU activation function for nonlinear transformation to obtain Loss_D_I, as shown below:

[0113] Loss_D_I=((D_imag(phase_enh)-s) 2 +(D_imag(phase_cle)) 2 ) / 2

[0114] Where phase_enh represents the phase information of the enhanced signal, phase_cle represents the phase information of the clean signal, and s represents an all-1 vector.

[0115] Step 5: Use the trained one-dimensional time series signal enhancement network to enhance the actual one-dimensional time series signal to obtain the estimated clean signal.

[0116] Example 2

[0117] The network model proposed in this invention will be compared with benchmark models and existing state-of-the-art (SOTA) methods, including temporal and time-frequency (TF) methods. For temporal methods, we selected Wave-u-net, Segan, and TSTNN. For TF methods, we selected several models for comparison, including DCUNET16, DCUNET20, MetricGAN, MetricGAN+, and FTDDN. All these models will be experimentally evaluated on the VCTK corpus.

[0118] In this embodiment, we used the open-source standard speech dataset from the University of Edinburgh as our experimental dataset. The speech samples in this dataset come from the publicly available Voice Bank corpus, while the noise samples come from the publicly available DEMAND noise dataset. This selection ensures that we use widely accepted and representative speech and noise data, making the experimental results more credible and comparable.

[0119] The experimental dataset uses the open-source standard speech dataset from the University of Edinburgh. The speech in this dataset comes from the publicly available corpus Voice Bank, and the noise in this dataset comes from the publicly available noise dataset DEMAND.

[0120] In assessing speech quality, we used several common metrics:

[0121] Perceptual Speech Quality Assessment (PESQ) ranges from -0.5 to 4.5; Short-Time Objective Intelligibility (STOI) ranges from 0 to 1 and is used to assess speech intelligibility, with scores closer to 1 indicating higher intelligibility; Subjective Mean Opinion Score (MOS): CSIG for signal distortion (from 1 to 5), CBAK for noise distortion assessment (from 1 to 5), and COVL for overall quality assessment (from 1 to 5).

[0122] The performance evaluation results of different one-dimensional time-series signal enhancement methods are shown in Table 1.

[0123] Table 1 Performance evaluation results of different one-dimensional time-series signal enhancement methods

[0124]

[0125]

[0126] In Table 1, PESQ, CSIG, CBAK, and COVL represent the speech quality perception assessment score, the signal distortion comprehensive measurement score, the noise distortion comprehensive measurement score, and the overall speech quality comprehensive measurement score, respectively.

[0127] In the experiments, we compared the one-dimensional time-series signal enhancement methods in the time domain and the frequency domain to ensure the sufficiency of the experimental results. According to the results in Table 1, we found that the method proposed in this invention outperforms other comparative algorithms in PESQ, CSIG, CBAK, and COVL metrics, which confirms the superiority of this invention in one-dimensional time-series signal enhancement.

[0128] Specifically, compared to the baseline model DCUNET20, this invention improves PESQ, CSIG, CBAK, and COVL scores by 0.47, 0.15, 0.06, and 0.30, respectively. Notably, this invention outperforms other comparative models across all metrics, with a particularly significant improvement in PESQ, far exceeding the performance of other models.

[0129] The foregoing has shown and described the main features and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

[0130] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A one-dimensional temporal signal enhancement method based on bi-branch convolution to enhance window attention, characterized in that, Includes the following steps: Step 1: Collect clean one-dimensional time series signals and noisy one-dimensional time series signals and perform preprocessing; Step 2: Construct a noisy one-dimensional time series signal X using the preprocessed clean one-dimensional time series signal and the noisy one-dimensional time series signal. Obtain the frequency domain representation of the noisy one-dimensional time series signal X through short-time Fourier transform: , ; Step 3: Construct a one-dimensional temporal signal enhancement network based on dual-branch convolutional enhanced window attention, including an encoder module, a convolutional enhanced window attention module, a decoder module, and a CRM mask module; Step 4: Train the one-dimensional time-series signal enhancement network obtained in Step 3; Step 5: Use the trained one-dimensional time series signal enhancement network to enhance the actual one-dimensional time series signal to obtain the estimated clean signal; The encoder module consists of two branches, which are used to predict the amplitude and phase of the signal, respectively. These two branches have the same network architecture, which includes 2D convolutional layers, BN layers and ReLU layers. The downsampling operation is implemented through the 2D convolutional layers. After the convolution operation is completed in each branch, a connection is established with the other branch through the complex value calculation module. The decoder module consists of two branches, which are used to estimate the amplitude and phase of the signal, respectively. These two branches have the same network architecture, which includes a 2D deconvolution layer, a BN layer, and a ReLU layer. After the deconvolution operation is completed in each branch, a connection is established with the other branch through a complex value calculation module. After passing through the last 2D deconvolution layer, the estimated amplitude and phase information are constrained by the hyperbolic tangent function to obtain the constrained amplitude and phase information. The convolutional enhancement window attention module includes a non-overlapping window attention module and a convolutional enhancement forward propagation module. The non-overlapping window attention module adopts a multi-head self-attention mechanism for non-overlapping windows. The convolutional enhancement forward propagation module includes a convolutional enhancement module and an LN layer. The convolutional enhancement module has an asymmetric structure, including a fully connected layer, an activation function, a transpose, a 2D convolutional layer, an activation function, a transpose, and a fully connected layer connected in sequence. The CRM mask module obtains a mask M based on the constrained amplitude and phase information. The mask M is then multiplied by the frequency domain representation of the noisy one-dimensional time-series signal X, and converted back to the time domain representation through inverse short-time Fourier transform.

2. The one-dimensional temporal signal enhancement method based on bi-branch convolution enhanced window attention according to claim 1, characterized in that, The specific steps of the preprocessing in step 1 are as follows: Step 1.1: Resample the original clean one-dimensional time-series signal and the noisy one-dimensional time-series signal with a sampling rate of 48000Hz to 16000Hz; Step 1.2: Use zero-padding to fill the signal to 10s.

3. The one-dimensional temporal signal enhancement method based on bi-branch convolution enhanced window attention according to claim 1, characterized in that, The multi-head self-attention mechanism for the non-overlapping window is as follows: Based on window size Signal Feature information Zero-padding is performed, and then the signal X is divided into windows of size . Non-overlapping windows are used, and a multi-head self-attention mechanism is performed within each window to obtain translation and transpose features from each window. ; Assuming the head of the multi-head self-attention mechanism is , No. The process of calculating the non-overlapping window size is described as follows: in Representing the first Query by size, key, value. It represents the first The outputs of each head will be concatenated and a linear transformation will be performed to obtain the final result of the multi-head attention process.

4. The one-dimensional temporal signal enhancement method based on bi-branch convolution enhanced window attention according to claim 1, characterized in that, The convolution-enhanced window attention module is represented as follows: in and These represent the outputs of the non-overlapping window attention module and the convolutional enhanced forward propagation module, respectively. It represents normalization.

5. A one-dimensional temporal signal enhancement method based on bi-branch convolution to enhance window attention, as described in claim 1, is characterized in that... The specific steps for training the one-dimensional temporal signal enhancement network obtained in step 3 in step 4 are as follows: Step 4.1: Input the frequency domain representation of the one-dimensional time-series signal into the one-dimensional time-series signal enhancement network in batches; Step 4.2: Output the enhanced one-dimensional timing signal; Step 4.3: Perform error calculation on the enhanced one-dimensional time series signal and the input one-dimensional time series signal using a joint loss function; Step 4.4: Backpropagate the error and update the parameters of the one-dimensional time-series signal enhancement network; Step 4.5: Repeat steps 4.1 to 4.4 until the convergence condition is met.

6. A one-dimensional temporal signal enhancement method based on bi-branch convolution to enhance window attention, as described in claim 5, is characterized in that... The joint loss function is established by introducing an amplitude loss function and a phase loss function, and combining them with a time-domain information loss function, and is expressed as follows: Where SI-SNR is the time-domain information loss function, Loss_D_R is the magnitude loss function, and Loss_D_I is the phase loss function; The SI-SNR is represented as: in and These are represented as the pure and estimated time-domain waveforms, respectively. Represents the dot product between two vectors. Represents the L2 paradigm; The calculation steps for Loss_D_R are as follows: First, the enhanced amplitude information and the amplitude information of the clean signal are input into the D_Real function. The D_Real function contains five convolutional layers and two attention layers. Except for the last convolutional layer, all convolutional layers use the Leaky ReLU activation function for nonlinear transformation to obtain Loss_D_R, as shown below: in This represents the amplitude information of the enhanced signal. The amplitude information representing a clean signal. Represents a vector consisting entirely of 1s; The calculation steps for Loss_D_I are as follows: First, the enhanced phase information and the phase information of the clean signal are input into the D_Imag function. The D_Imag function contains five convolutional layers and two attention layers. Except for the last convolutional layer, all convolutional layers use the Leaky ReLU activation function for nonlinear transformation to obtain Loss_D_I, as shown below: in Phase information representing the enhanced signal, Phase information representing a clean signal. It represents a vector consisting entirely of 1s.

Images