Lightweight speech enhancement method based on dual-path one-dimensional convolution group loop network

Abstract

The invention belongs to the technical field of speech enhancement, and particularly relates to a lightweight speech enhancement method based on a dual-path one-dimensional convolution group loop network. The method comprises the following steps: firstly, acquiring a noisy voice signal, and extracting a spectrum feature of the noisy voice signal; then, constructing a dual-path one-dimensional convolution group cycle network model which comprises an encoder module, a decoder module and a dual-path grouping cycle module located between the encoder module and the decoder module, and performing complex multiplication on the output characteristics of the decoder module and the input characteristics of the encoder module to obtain an estimated complex ratio mask; and finally, extracting the spectrum characteristics of a to-be-enhanced noisy voice signal, and inputting the spectrum characteristics of the to-be-enhanced noisy voice signal into the trained model to obtain an estimated complex ratio mask. Performing complex multiplication on the spectrum characteristics of the noisy voice signal to be enhanced and the estimated complex ratio mask to obtain the spectrum characteristics of the enhanced voice signal; and performing inverse short-time Fourier transform and overlap-add on the spectrum characteristics of the enhanced voice signal in sequence to obtain the enhanced voice signal. According to the method, the model parameter quantity and the calculation quantity are reduced, and the problems that an existing voice enhancement technology is insufficient in real-time processing performance, large in calculation resource demand and not suitable for being deployed on edge equipment are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech enhancement, and in particular relates to a lightweight speech enhancement method based on a dual-path one-dimensional convolutional grouping recurrent network. Background Art

[0002] Speech enhancement aims to restore pure speech signals from distorted speech signals interfered by noise, improve the intelligibility and perceptual quality of distorted speech signals, and thus improve application effects in scenarios such as speech recognition, voice calls, hearing assistance, and voiceprint recognition. Modern deep learning-based models perform well in speech enhancement tasks, and performance has been significantly improved.

[0003] With the popularity of mobile communications and smart devices, the demand for real-time lightweight speech enhancement technology is growing. For example, smartphone video conferencing requires real-time noise reduction to ensure clear conversations, smart speakers accurately recognize commands in real time under various noise environments, and hearing aids require real-time speech enhancement technology to effectively filter background noise. Real-time speech enhancement technology requires low-latency, low-energy, and efficient processing on resource-constrained devices to meet modern communication and interaction requirements.

[0004] Although existing deep learning-based speech enhancement methods have made some progress in improving speech quality, they also generally have limitations such as high computational complexity, high processing delay, high energy consumption, and a large number of parameters, making it difficult to meet the needs of real-time and lightweight applications. Summary of the invention

[0005] In view of the shortcomings of the prior art, the technical problem that the present invention intends to solve is to provide a lightweight speech enhancement method based on a dual-path one-dimensional convolutional grouping recurrent network. The present invention reduces the number of model parameters and the amount of calculation, and solves the problem that the existing speech enhancement technology has insufficient real-time processing performance, large computing resource requirements, and is not suitable for deployment on edge devices.

[0006] The present invention solves the technical problem by adopting the following technical solution:

[0007] A lightweight speech enhancement method based on a dual-path one-dimensional convolutional grouping recurrent network, characterized in that the method comprises the following steps:

[0008] Step 1: Mix the clean speech signal and the noise signal at different signal-to-noise ratios to obtain a data set consisting of several noisy speech signals;

[0009] Step 2: Frame the noisy speech signal, perform windowing and short-time Fourier transform to obtain the spectrum of each frame signal; splice the spectrum of all frame signals in the time dimension to obtain the spectrum characteristics of the noisy speech signal;

[0010] Step 3: construct a dual-path one-dimensional convolution group recurrent network model; the dual-path one-dimensional convolution group recurrent network model includes an encoder module, a decoder module and a dual-path group recurrent module located between the encoder module and the decoder module, the encoder module includes multiple encoder layers, the decoder module includes multiple decoder layers, and each encoder layer is jump-connected to the corresponding decoder layer; the output features of the decoder module are complex multiplied with the input features of the encoder module to obtain an estimated complex ratio mask;

[0011] The dual-path grouping circulation module comprises an intra-frame grouping circulation module, an inter-frame grouping circulation module and a convolution gating module which are connected in sequence; wherein the input features of the dual-path grouping circulation module are dimensionally reorganized to obtain the input features of the intra-frame grouping circulation module; the input features of the intra-frame grouping circulation module are sequentially subjected to a grouped bidirectional gating circulation unit, a linear layer and layer normalization, and then added to themselves to obtain the output features of the intra-frame grouping circulation module; the input features of the intra-frame grouping circulation module are dimensionally reorganized to obtain the input features of the inter-frame grouping circulation module; the output features of the inter-frame grouping circulation module The input features are sequentially subjected to the grouped gated recurrent unit, the linear layer and the layer normalization, and then added to themselves to obtain the output features of the inter-frame grouped recurrent module; the output features of the inter-frame grouped recurrent module are dimensionally reorganized to obtain the input features of the convolutional gated module; the input features of the convolutional gated module are sequentially subjected to point-by-point convolution, channel-by-channel convolution and Mish activation function, and then point-by-point multiplied with the features obtained by point-by-point convolution of the input features of the convolutional gated module, and the features obtained by point-by-point multiplication are then added to the input features of the convolutional gated module to obtain the output features of the convolutional gated module;

[0012] Step 4: train the dual-path one-dimensional convolution group recurrent network model, and use the trained dual-path one-dimensional convolution group recurrent network model to estimate the complex ratio mask;

[0013] Step 5: Extract the spectral features of the noisy speech signal to be enhanced, input the spectral features of the noisy speech signal to be enhanced into the trained dual-path one-dimensional convolutional group recurrent network model to obtain an estimated complex ratio mask; perform complex multiplication of the spectral features of the noisy speech signal to be enhanced and the estimated complex ratio mask to obtain the spectral features of the enhanced speech signal; subject the spectral features of the enhanced speech signal to inverse short-time Fourier transform and overlap-addition in sequence to obtain an enhanced speech signal.

[0014] Furthermore, the encoder layer includes a one-dimensional convolution layer, batch normalization, PReLU activation function and multi-dimensional attention layer on the frequency axis connected in sequence; the multi-dimensional attention layer includes a channel attention block and a time-frequency attention block; the input features of the channel attention block are sequentially subjected to time-frequency axis average pooling, one-dimensional convolution, PReLU activation function, one-dimensional convolution and Sigmoid activation function, and then multiplied with themselves point by point to obtain the output features of the channel attention block, which are used as the input features of the time-frequency attention block; the time-frequency attention block includes two branches: time domain attention and frequency domain attention. The domain attention branch includes a time axis average pooling layer, a one-dimensional convolution layer, a PReLU activation function, a one-dimensional convolution layer and a Sigmoid activation function connected in sequence. The frequency domain attention branch includes a frequency axis average pooling layer, a one-dimensional convolution layer, a PReLU activation function, a one-dimensional convolution layer and a Sigmoid activation function connected in sequence. The output features of the time domain attention branch and the frequency domain attention branch are multiplied to obtain the output features of the time-frequency attention block; the output features of the time-frequency attention block are multiplied point by point with the output features of the channel attention block to obtain the output features of the multi-dimensional attention layer.

[0015] Furthermore, in the grouped bidirectional gated recurrent unit, the input features of the intra-frame grouped recurrent module are divided into two parts in the channel dimension through the segmentation layer to obtain two features x1 and x2; features x1 and x2 are respectively passed through a bidirectional gated recurrent unit to obtain features x1' and x2'; features x1' and x2' are integrated and shuffled in the channel dimension through the rearrangement layer to obtain the output features of the grouped bidirectional gated recurrent unit;

[0016] In the grouped gated recurrent unit, the input features of the inter-frame grouped recurrent module pass through the segmentation layer and are divided into two parts in the channel dimension to obtain two features y1 and y2; features y1 and y2 pass through a gated recurrent unit respectively to obtain features y1' and y2'; features y1' and y2' pass through the rearrangement layer, are integrated and shuffled in the channel dimension to obtain the output features of the grouped gated recurrent unit.

[0017] Furthermore, during the model training process, the training loss is calculated by the following formula;

[0018]

[0019] In the formula, is the pure speech signal s(t) and the enhanced speech signal The negative signal-to-noise ratio, is the real part of the spectrum characteristic of the pure speech signal S r Enhance the real part of the spectral characteristics of speech signals The mean square error, is the imaginary part of the spectrum characteristic of the pure speech signal S iEnhance the imaginary part of the speech signal spectrum The mean square error, is the characteristic amplitude of the pure speech signal spectrum |S| and the characteristic amplitude of the enhanced speech signal spectrum The mean square error of .

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1. In the cascaded encoder path, the two-dimensional convolution of sliding calculation on the time axis will cause the same time frame to be repeatedly calculated, and there will be redundant calculations between adjacent frames, making the model calculation inefficient. Therefore, the encoder layer focuses on extracting frequency domain features through one-dimensional convolution on the frequency axis, which improves the calculation rate of the model and enables the model to process single-frame signals in real time. The multi-dimensional attention layer can improve the model's adaptability to different noisy speech signals by dynamically assigning importance weights to different features, more accurately capture key speech features, and has a simple structure. It can retain more speech harmonic information and suppress noise without significantly increasing the computational burden.

[0022] 2. In the dual-path grouped recurrent module, the segmentation layer of the intra-grouped recurrent module and the inter-grouped recurrent module divides the input features into two non-overlapping groups in the channel dimension, and the intra-group features are learned separately in their respective groups. The grouping operation significantly reduces the amount of training parameters of the recurrent unit, thereby reducing the complexity of the model. However, the output features of this grouping method only depend on the input features of the corresponding group, resulting in the loss of inter-group dependency information and reducing the modeling ability of the recurrent unit. To solve this problem, a parameter-free rearrangement layer is used to concatenate and rearrange the output features of the two groups in the channel dimension to capture the inter-group dependency. The point-by-point convolution, channel-by-channel convolution and Mish activation function in the convolutional gating module are used to fuse the inter-channel information and obtain fine-grained channel attention. The gating mechanism is implemented through element-by-element multiplication to further supplement the inter-group dependency information. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the overall flow chart of the present invention;

[0024] Figure 2 It is a structural diagram of the dual-path one-dimensional convolution group recurrent network model of the present invention;

[0025] Figure 3 is a structural diagram of the encoder layer and the decoder layer of the present invention;

[0026] Figure 4 It is a structural diagram of the dual-path grouping circulation module of the present invention;

[0027] Figure 5It is a structural diagram of the grouped bidirectional gated cyclic unit and the grouped gated cyclic unit of the present invention. DETAILED DESCRIPTION

[0028] Specific embodiments are given below in conjunction with the accompanying drawings. The specific embodiments are only used to introduce the technical solutions of the present invention in detail and are not intended to limit the protection scope of the present application.

[0029] The present invention provides a lightweight speech enhancement method based on a dual-path one-dimensional convolutional group recurrent network (hereinafter referred to as the method, see Figures 1 to 5 ), including the following steps:

[0030] Step 1: Mix the clean speech signal and the noise signal at different signal-to-noise ratios such as -5, 0, 5, 10, and 15 dB to obtain several noisy speech signals; several noisy speech signals and clean speech signals together constitute a data set;

[0031] Step 2: Extract the spectral features of the noisy speech signal;

[0032] The noisy speech signal is framed, windowed, and short-time Fourier transformed to obtain the spectrum of each frame signal; the spectrum of all frame signals is spliced ​​in the time dimension to obtain the spectrum characteristics of the noisy speech signal, that is, the input characteristics of the dual-path one-dimensional convolution group recurrent network model, with a dimension of T×F×C, where T represents the number of frames, F is the number of frequency points, and C is the number of channels; in this embodiment, the sampling rate of the speech signal is 16kHz, the frame length is 32ms, the frame shift is 16ms, the FFT point number is 512, the window function uses a Hamming window, and the window length is the same as the frame length; the spectrum of each frame signal contains 257 frequency points, and the number of channels C is 2, indicating two channels of real and imaginary parts.

[0033] Step 3: Construct a dual-path one-dimensional convolutional group recurrent network model to estimate the complex ratio mask (CRM);

[0034] like Figure 2 As shown, the dual-path one-dimensional convolutional grouped recurrent network model includes an encoder module, a decoder module and multiple dual-path grouped recurrent modules (this embodiment includes two) located between the encoder module and the decoder module, the encoder module is used for downsampling, and the decoder module is used for upsampling; the encoder module includes multiple encoder layers, the decoder module includes multiple decoder layers with the same number as the encoder layers, and each encoder layer is jump-connected to the corresponding decoder layer.

[0035] like Figure 3As shown, the encoder layer includes a one-dimensional convolution layer on the frequency axis, batch normalization, a PReLU activation function and a multi-dimensional attention layer connected in sequence; the encoder module of this embodiment includes three encoder layers, and the numbers of convolution kernels of the one-dimensional convolution layers on the frequency axis of the three encoder layers are 32, 64 and 128 respectively, the convolution kernel sizes are all 5, the step sizes are all 2, and the zero padding is (2, 2). The encoder module finally downsamples the frequency dimension of the spectral features of the noisy speech signal from 257 to 33, and the time dimension remains unchanged.

[0036] The multi-dimensional attention layer includes a channel attention block and a time-frequency attention block; the input features of the channel attention block are sequentially subjected to time-frequency axis average pooling, one-dimensional convolution, PReLU activation function, one-dimensional convolution and Sigmoid activation function, and then multiplied with themselves point by point to obtain the output features of the channel attention block; the time-frequency axis average pooling performs average pooling on the time axis and frequency axis of the features at the same time. The output features of the channel attention block are used as the input features of the time-frequency attention block. The time-frequency attention block includes two branches, time domain attention and frequency domain attention. The time domain attention branch includes a time axis average pooling layer, a one-dimensional convolution layer, a PReLU activation function, a one-dimensional convolution layer and a Sigmoid activation function connected in sequence, and the frequency domain attention branch includes a frequency axis average pooling layer, a one-dimensional convolution layer, a PReLU activation function, a one-dimensional convolution layer and a Sigmoid activation function connected in sequence. The output features of the time domain attention branch and the frequency domain attention branch are multiplied to obtain the output features of the time-frequency attention block; the output features of the time-frequency attention block are multiplied point by point with the output features of the channel attention block to obtain the output features of the multi-dimensional attention layer. The number of convolution kernels of each one-dimensional convolution layer of the multi-dimensional attention layer is the same as the number of channels of its input features, the convolution kernel size is 5, the stride is 1, the zero padding is (2, 2), and the feature dimension is not changed.

[0037] The structure of the decoder layer is similar to that of the encoder layer, except that the one-dimensional convolution layer on the frequency axis of the encoder layer is replaced by a one-dimensional transposed convolution layer on the frequency axis. The numbers of convolution kernels of the one-dimensional transposed convolution layers on the frequency domain axes of the three decoder layers are 64, 32, and 2, respectively. The sizes of the transposed convolution kernels are all 5, the step sizes are all 2, and the zero padding is (2, 2). The decoder module finally restores the feature dimension to its original size to obtain the estimated CRM.

[0038] like Figure 4As shown, the dual-path grouped loop module includes an intra-frame grouped loop module, an inter-frame grouped loop module and a convolutional gating module connected in sequence; wherein the input features of the dual-path grouped loop module are dimensionally reorganized to obtain the input features x of the intra-frame grouped loop module; the input features x of the intra-frame grouped loop module are sequentially passed through a grouped bidirectional gated loop unit (Bidirectional Grouped Gated Recurrent Unit, Bi-GGRU), a linear layer and layer normalization, and then added to themselves to obtain the output features of the intra-frame grouped loop module. Figure 5 As shown in the figure, in the grouped bidirectional gated recurrent unit, the input feature x of the intra-frame grouped recurrent module passes through the segmentation layer and is split into two parts in the channel dimension to obtain two features x1 and x2; features x1 and x2 pass through a bidirectional gated recurrent unit respectively to obtain features x1' and x2'; features x1' and x2' pass through the rearrangement layer, are integrated and shuffled in the channel dimension to obtain the output features of the grouped bidirectional gated recurrent unit. The number of hidden states of Bi-GGRU is 32, which is used to model the frequency dependency within the frame without affecting the causality of the model.

[0039] The input features of the intra-frame grouped recurrent module are reshaped to obtain the input features y of the inter-frame grouped recurrent module; the input features y of the inter-frame grouped recurrent module are sequentially passed through the grouped gated recurrent unit (GGRU), linear layer and layer normalization, and then added to themselves to obtain the output features of the inter-frame grouped recurrent module. Figure 5 As shown in the figure, in the group gated recurrent unit, the input feature y of the inter-frame grouped recurrent module passes through the segmentation layer and is divided into two parts in the channel dimension to obtain two features y1 and y2; features y1 and y2 pass through a gated recurrent unit respectively to obtain features y1' and y2'; features y1' and y2' pass through the rearrangement layer, are integrated and shuffled in the channel dimension to obtain the output features of the group gated recurrent unit. The number of hidden states of GGRU is 64, which is used to model the temporal dependency of frequencies between consecutive frames.

[0040] The output features of the inter-frame grouping cycle module are reshaped to obtain the input features of the convolution gate module; the input features of the convolution gate module are successively subjected to point-by-point convolution, channel-by-channel convolution and Mish activation function, and then point-by-point multiplication is performed with the features obtained by point-by-point convolution of the input features of the convolution gate module. The features obtained by point-by-point multiplication are then added to the input features of the convolution gate module to obtain the output features of the convolution gate module. The number of convolution kernels for point-by-point convolution is 128; the convolution kernel size for channel-by-channel convolution is 5, the stride is 1, and the zero padding is (2, 2).

[0041] The output features of the decoder module are complex multiplied with the input features of the encoder module to obtain the output of the dual-path one-dimensional convolution group recurrent network model, that is, the estimated complex ratio mask; the spectral features of the noisy speech signal are complex multiplied with the estimated complex ratio mask to obtain the spectral features of the enhanced speech signal; the spectral features of the enhanced speech signal are successively subjected to inverse short-time Fourier transform and overlap-addition to restore the enhanced speech signal from the frequency domain to the time domain to obtain the enhanced speech signal, thus completing the enhancement of the noisy speech signal. The process of obtaining the enhanced speech signal from the estimated complex ratio mask can be expressed as:

[0042] X=X r +iX i (1)

[0043] M=M r +iM i (2)

[0044]

[0045] Where, X r , X i are the real and imaginary parts of the spectral feature X of the noisy speech signal, M r 、M i are the real and imaginary parts of the estimated complex ratio mask M, respectively, It is the spectral feature of the enhanced speech signal. represents the enhanced speech signal, and STFT(.) represents the inverse short-time Fourier transform and overlap-add operation.

[0046] Step 4: Train the dual-path one-dimensional convolution group recurrent network model and calculate the training loss using the following loss function; use the Adam optimizer for training with an initial learning rate of 0.001. If the training loss does not decrease for five consecutive cycles, the learning rate is halved.

[0047] The loss function is:

[0048]

[0049] In the formula, is the pure speech signal s(t) and the enhanced speech signal The negative signal-to-noise ratio, is the real part of the spectrum characteristic of the pure speech signal S r Enhance the real part of the spectral characteristics of speech signals The mean square error, is the imaginary part of the spectrum characteristic of the pure speech signal S i Enhance the imaginary part of the speech signal spectrum The mean square error, is the characteristic amplitude of the pure speech signal spectrum |S| and the characteristic amplitude of the enhanced speech signal spectrum The logarithm of the sum of the mean square errors is taken to ensure that it is of the same order of magnitude as the negative signal-to-noise ratio.

[0050] This embodiment uses two data sets, VCTK-DEMAND and DNS3, to train a dual-path one-dimensional convolution group recurrent network model; the VCTK-DEMAND data set contains clean speech signals and noisy speech signals. The training set and the test set contain 11,572 speech signals from 28 speakers and 824 speech signals from 2 speakers, respectively, with a sampling rate of 16kHz; 1,572 speech signals are selected from the training set as the validation set. The DNS3 data set contains a set of clean speech signals, a set of noisy speech signals, and a room impulse response (RIR). During the mixing process, the clean speech signal is convolved with a randomly selected RIR, and then mixed with a randomly selected noisy speech signal in the signal-to-noise ratio range from -5 to 15dB, generating a total of 720,000 pairs of 10-second noisy-clean speech signals as the training set, while the validation set and the test set generate 840 pairs and 800 pairs of noisy-clean speech signals, respectively. The batch size of the VCTK-DEMAND data set is 4, and the batch size of the DNS3 data set is 16. During training on the DNS3 dataset, the speech signals are segmented into 8-second segments, and 40,000 pairs of noisy-clean training speech signals are randomly selected in each cycle.

[0051] Step 5: Extract the spectral features of the noisy speech signal to be enhanced, input the spectral features of the noisy speech signal to be enhanced into the trained dual-path one-dimensional convolutional group recurrent network model to obtain an estimated complex ratio mask; perform complex multiplication of the spectral features of the noisy speech signal to be enhanced and the estimated complex ratio mask to obtain the spectral features of the enhanced speech signal; subject the spectral features of the enhanced speech signal to inverse short-time Fourier transform and overlap-addition in sequence to obtain an enhanced speech signal.

[0052] In order to verify the speech enhancement effect of the present invention, the method is tested on the VCTK-DEMAND dataset and compared with the advanced real-time speech enhancement model. The test results are shown in Table 1. The three indicators of parameter quantity, MACs and RTF are selected to evaluate the model performance, and the two indicators of perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) are selected to evaluate the speech enhancement effect. The original PESQ and STOI of the VCTK-DEMAND dataset are 1.97 and 0.921 respectively.

[0053] Table 1 Comparison of test results of different models on the VCTK-DEMAND dataset

[0054]

[0055] DPT-FSNet uses a dual-path Transformer to improve the dual-path recurrent network, but it consumes a lot of computing resources when processing long sequences, with a computational cost of 6.2GMACs. Fast FullSubNet designs a cascaded full-band and sub-band fusion model, with a computational cost of 4.14GMACs. SICRN combines a multidimensional state space model with in-place convolution for speech enhancement, with a computational cost of 4.24GMACs. The dual-path one-dimensional convolutional group recurrent network model of the present invention has achieved the best performance in terms of parameter quantity, MACs and RTF. The parameter quantity is only 0.38M, the computational cost is 0.4G MACs per second, and the RTF is 0.008. This is because one-dimensional convolution is used to perform up- and down-sampling on the frequency axis, which improves the processing rate of single-frame signals; the multi-dimensional attention layer is used to improve the generalization performance of the model for different noisy speech signals with low computational burden; the dual-path grouping recurrent module uses Bi-GGRU and GGRU to model the intra-frame frequency dependency and inter-frame time dependency respectively. This grouping operation significantly reduces the computational complexity, and the convolution gating mechanism can supplement the lost inter-group dependency information.

[0056] The sources of the above existing models are as follows:

[0057] [1] F.Dang, H.Chen and P.Zhang, "DPT-FSNet: Dual-Path Transformer BasedFull-Band and Sub-Band Fusion Network for Speech Enhancement," ICASSP 2022-2022IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, Singapore, pp.6857-6861, 2022.

[0058] [2]

[0059] [3]C.Zhao, S.He and

[0060] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A lightweight speech enhancement method based on a dual-path one-dimensional convolutional grouping recurrent network, characterized in that: The method comprises the following steps: Step 1: Mix the clean speech signal and the noise signal at different signal-to-noise ratios to obtain a data set consisting of several noisy speech signals; Step 2: Frame the noisy speech signal, perform windowing and short-time Fourier transform to obtain the spectrum of each frame signal; splice the spectrum of all frame signals in the time dimension to obtain the spectrum characteristics of the noisy speech signal; Step 3: construct a dual-path one-dimensional convolution group recurrent network model; the dual-path one-dimensional convolution group recurrent network model includes an encoder module, a decoder module and a dual-path group recurrent module located between the encoder module and the decoder module, the encoder module includes multiple encoder layers, the decoder module includes multiple decoder layers, and each encoder layer is jump-connected to the corresponding decoder layer; the output features of the decoder module are complex multiplied with the input features of the encoder module to obtain an estimated complex ratio mask; The dual-path grouping circulation module comprises an intra-frame grouping circulation module, an inter-frame grouping circulation module and a convolution gating module which are connected in sequence; wherein the input features of the dual-path grouping circulation module are dimensionally reorganized to obtain the input features of the intra-frame grouping circulation module; the input features of the intra-frame grouping circulation module are sequentially subjected to a grouped bidirectional gating circulation unit, a linear layer and layer normalization, and then added to themselves to obtain the output features of the intra-frame grouping circulation module; the input features of the intra-frame grouping circulation module are dimensionally reorganized to obtain the input features of the inter-frame grouping circulation module; the output features of the inter-frame grouping circulation module The input features are sequentially subjected to the grouped gated recurrent unit, the linear layer and the layer normalization, and then added to themselves to obtain the output features of the inter-frame grouped recurrent module; the output features of the inter-frame grouped recurrent module are dimensionally reorganized to obtain the input features of the convolutional gated module; the input features of the convolutional gated module are sequentially subjected to point-by-point convolution, channel-by-channel convolution and Mish activation function, and then point-by-point multiplied with the features obtained by point-by-point convolution of the input features of the convolutional gated module, and the features obtained by point-by-point multiplication are then added to the input features of the convolutional gated module to obtain the output features of the convolutional gated module; Step 4: train the dual-path one-dimensional convolution group recurrent network model, and use the trained dual-path one-dimensional convolution group recurrent network model to estimate the complex ratio mask; Step 5: Extract the spectral features of the noisy speech signal to be enhanced, input the spectral features of the noisy speech signal to be enhanced into the trained dual-path one-dimensional convolutional group recurrent network model to obtain an estimated complex ratio mask; perform complex multiplication of the spectral features of the noisy speech signal to be enhanced and the estimated complex ratio mask to obtain the spectral features of the enhanced speech signal; subject the spectral features of the enhanced speech signal to inverse short-time Fourier transform and overlap-addition in sequence to obtain an enhanced speech signal.

2. The lightweight speech enhancement method based on dual-path one-dimensional convolutional grouping recurrent network according to claim 1 is characterized in that: The encoder layer includes a one-dimensional convolution layer, batch normalization, PReLU activation function and a multi-dimensional attention layer on the frequency axis connected in sequence; the multi-dimensional attention layer includes a channel attention block and a time-frequency attention block; the input features of the channel attention block are sequentially subjected to time-frequency axis average pooling, one-dimensional convolution, PReLU activation function, one-dimensional convolution and Sigmoid activation function, and then multiplied with themselves point by point to obtain the output features of the channel attention block, and used as the input features of the time-frequency attention block; the time-frequency attention block includes two branches, time domain attention and frequency domain attention, the time domain attention branch includes a time axis average pooling layer, a one-dimensional convolution layer, a PReLU activation function, a one-dimensional convolution layer and a Sigmoid activation function connected in sequence, and the frequency domain attention branch includes a frequency axis average pooling layer, a one-dimensional convolution layer, a PReLU activation function, a one-dimensional convolution layer and a Sigmoid activation function connected in sequence, and the output features of the time domain attention branch and the frequency domain attention branch are multiplied to obtain the output features of the time-frequency attention block; The output features of the time-frequency attention block are multiplied point by point with the output features of the channel attention block to obtain the output features of the multi-dimensional attention layer.

3. The lightweight speech enhancement method based on dual-path one-dimensional convolutional grouping recurrent network according to claim 1 or 2, characterized in that: In the grouped bidirectional gated recurrent unit, the input features of the intra-frame grouped recurrent module are divided into two parts in the channel dimension through the segmentation layer to obtain two features x1 and x2; features x1 and x2 are respectively passed through a bidirectional gated recurrent unit to obtain features x1' and x2'; features x1' and x2' are integrated and shuffled in the channel dimension through the rearrangement layer to obtain the output features of the grouped bidirectional gated recurrent unit; In the grouped gated recurrent unit, the input features of the inter-frame grouped recurrent module pass through the segmentation layer and are divided into two parts in the channel dimension to obtain two features y1 and y2; features y1 and y2 pass through a gated recurrent unit respectively to obtain features y1' and y2'; features y1' and y2' pass through the rearrangement layer, are integrated and shuffled in the channel dimension to obtain the output features of the grouped gated recurrent unit.

4. The lightweight speech enhancement method based on dual-path one-dimensional convolutional grouping recurrent network according to claim 1 is characterized in that: During model training, the training loss is calculated using the following loss function: In the formula, is the pure speech signal s(t) and the enhanced speech signal The negative signal-to-noise ratio, is the real part of the spectrum characteristic of the pure speech signal S r Enhance the real part of the spectral characteristics of speech signals The mean square error, is the imaginary part of the spectrum characteristic of the pure speech signal S i Enhance the imaginary part of the speech signal spectrum The mean square error, is the characteristic amplitude of the pure speech signal spectrum | S | The amplitude of the spectral feature of the enhanced speech signal The mean square error of .

Images