Low-delay speech enhancement method based on microphone array

Abstract

The invention discloses a low-delay speech enhancement method based on a microphone array, and the method comprises the steps: setting a group of initial pole parameters, optimizing the initial pole parameters through employing an artificial neural network, and obtaining a real number pole; constructing an orthogonal basis function model of each channel of the microphone array by using real number poles, and calculating the response of each order of filter; framing and time domain feature extraction are carried out on microphone array receiving signals, and an improved sequential network is used for estimating the weight of a self-adaptive beam former formed by an orthogonal basis function model; and calculating the system response of each channel of the beam forming network according to the filter response and the weight of the beam former, and obtaining an enhanced complete voice signal. By using the orthogonal basis structure beam forming network, poles can be flexibly adjusted, the degree of freedom of the network can be improved, the length of a filter can be shortened, and network delay can be reduced; under the condition that the length of the filter is shorter, a better speech enhancement effect is achieved.

Description

Technical Field

[0001] The present invention relates to speech processing technology, and in particular to a low-delay speech enhancement method based on a microphone array. Background Art

[0002] Speech enhancement is a technology that improves speech quality by eliminating background noise or directional interference during speech propagation. Among them, multi-channel beamforming technology can make full use of time and space information to effectively reduce speech distortion. The speech enhancement method based on deep learning (DL) uses the complex structure and strong expression ability of deep neural network (DNN) to learn the relationship between the microphone received signal and the expected signal and its characteristics. It is applicable under various background noises and is an efficient technical means.

[0003] The existing DL-based frequency domain beamforming method mainly estimates the covariance matrix of the signal and then uses it in the minimum variance distortionless response (MVDR) beamforming. This type of method has a certain improvement in speech quality, but the residual noise is large. At the same time, in order to ensure a long frame length for frequency domain resolution, it is necessary to extract the contextual information-rich features of the signal during the training process, so the system delay is also large.

[0004] The existing time-domain filter-and-Sum Network (FaSNet) adaptively estimates the beamformer weights in the time domain, which can process speech signals with shorter frame lengths, ensure the real-time performance of the system, and achieve better noise reduction effects. However, the FaSNet delay is related to the filter length, and a longer filter length needs to be set to achieve a good filtering effect, which increases the network delay. Summary of the invention

[0005] Purpose of the invention: In view of the above problems, the purpose of the present invention is to provide a low-latency speech enhancement method based on a microphone array. By using OBF filters in each channel of the microphone array to improve the degree of freedom, good speech enhancement performance can be achieved when the filter length is short. The entire beamforming framework is implemented using an artificial neural network, while reducing network latency.

[0006] Technical solution: A low-latency speech enhancement method based on a microphone array of the present invention comprises the following steps:

[0007] Step 1, setting a set of initial pole parameters, optimizing the initial pole parameters using an artificial neural network, obtaining real poles, and using the real poles as parameters of a multi-pole orthogonal basis function model of each channel of the microphone array;

[0008] Step 2, constructing an orthogonal basis function model of each channel of the microphone array using real poles, and calculating the filter responses of each order;

[0009] Step 3, performing frame division and time domain feature extraction on the microphone array received signal, and using an improved timing network to estimate the weights of an adaptive beamformer formed by an orthogonal basis function model;

[0010] Step 4: Calculate the system response of each channel of the beamforming network according to the filter response and the beamformer weight, filter and sum the microphone array received signal to obtain an enhanced frame-level speech signal, and perform overlap-addition operation on the enhanced frame-level speech signal to obtain an enhanced complete speech signal.

[0011] Further, step 1 comprises:

[0012] For microphone array channels m, m = 1, 2, ..., M, an unbiased one-dimensional convolutional neural network with 1 input channel and L output channels is used to train the initial pole parameters M is the number of microphones, L is the filter length, the kernel size and step size of the one-dimensional convolutional neural network are set to 1, and the data with a constant value of 1 is used as the input item. The output item is the learnable weight of the one-dimensional convolutional neural network, which is expressed as:

[0013]

[0014] In the formula, Conv1d represents a one-dimensional convolutional neural network; l represents the order of the filter;

[0015] The extreme points obtained from each training Perform boundary processing and process the pole a ml As the poles of the orthogonal basis function model; where the boundary treatment is expressed as:

[0016]

[0017] Where HardTanh(·) is the activation function, expressed as:

[0018]

[0019] In the formula, a max and a min Respectively represent the maximum and minimum values ​​of the linear range of the HardTanh(·) function.

[0020] Further, step 2 includes:

[0021] The state space equation of the orthogonal basis function model of the mth channel of the microphone array is:

[0022] x m (t) = A m x m (t-1)+b m u m (t)

[0023]

[0024] In the formula, is the state vector at the tth discrete time point, is the state matrix, is the input-state vector, u m (t) is the signal received by the microphone, w m is the state-output vector, i.e., the weight of the orthogonal basis function model, is the orthogonal basis function model output, [·] T Represents the transpose of a matrix or vector;

[0025] Calculate the state matrix A based on the state space equation of the orthogonal basis function model m The (j,l) element and input-state vector b m The lth elements of are:

[0026]

[0027] in, is the orthogonal state space realization matrix of the lth first-order all-pass filter that constitutes the orthogonal basis function model of the mth channel, and the elements in the matrix are related to the pole a ml The relationship is: d ml =-a ml ;

[0028] Set the initial state vector x m (0) = 0, then the state space equation of the orthogonal basis function model is:

[0029]

[0030] Where i represents the discrete time index, Represents the state matrix A m the i-1th power;

[0031] This gives a length of L h The L-order filter response is:

[0032]

[0033] in, represents the L-order filter response vector at the i-th time point, i=1,2,…,Lh , L h ≥L.

[0034] Further, step 3 includes:

[0035] The microphone received signal is divided into frames, the frame length is W, and the frame shift J=W / 2. Then the k-th frame signal of the m-th channel of the microphone array is expressed as:

[0036] u m (k)=[u m ((k-1)J+1),…,u m ((k-1)J+W)] T

[0037] In the case of non-causal feature extraction and filtering, the current frame signal and its context information are spliced ​​to obtain the spliced ​​signal:

[0038] v m (k)=[u m ((k-1)J-C+1),…,u m ((k-1)J+W+C)] T

[0039] In the formula, C represents the length of the previous or next concatenation information;

[0040] The reference microphone receives the signal u ref (k) and v m (k) Extract the feature vector ξ of each channel of the microphone array m (k), and the eigenvector ξ m (k) is input into the improved sequential network SeqNet(·) to obtain the weights of the adaptive wave former.

[0041] Furthermore, the improved sequential network SeqNet(·) starts with a linear bottleneck layer with B output channels, and the weights of the adaptive wave formers at the frame level corresponding to the state space equation are estimated as follows:

[0042] w m (k) = OutputLayer(SeqNet(ξ m (k)))

[0043] In the formula, w m (k)=[w m,1 (k),…,w m,L (k)] T is the component vector of the adaptive wave former weight;

[0044] The output layer OutputLayer(·) of the improved sequential network SeqNet(·) consists of a one-dimensional convolutional neural network and an activation function, which is specifically set as follows:

[0045] OutputLayer(p)=PReLU(n(Wp+q))

[0046] In the formula, is the output of the timing network, and are the weights and biases of the one-dimensional convolutional neural network, η is the scale factor that controls the output of the one-dimensional convolutional neural network, and PReLU(·) is the parameter rectified linear unit activation function.

[0047] Furthermore, in step 4, the system responses of each channel of the beamforming network are:

[0048]

[0049] When L h > L, add L before the convolution concatenation signal h -L zeros are added to ensure that the length of the convolved signal is equal to the expected signal length. The splicing information after zero padding is:

[0050]

[0051] A one-dimensional convolutional neural network is used to implement the beamforming network to filter the received signals of each channel, and the output signals of all channels are summed to obtain the enhanced k-th frame speech signal:

[0052]

[0053] In the formula, * represents the convolution operation of the one-dimensional convolutional neural network, and the relationship between the filter length and the context signal length is L=2C+1.

[0054] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0055] 1. In the present invention, the FasNet with all zero poles is expanded into an orthogonal basis structure beamforming network, which can flexibly adjust the poles and improve the network freedom.

[0056] 2. The non-convex multi-pole optimization problem of the OBF model is solved by training a one-dimensional convolutional neural network. A unified orthogonal basis expansion result is given for speech under different conditions, and the pole solution and beamformer weight estimation are decoupled, which is more flexible and suitable for adaptive beamforming in actual situations.

[0057] 3. Compared with the frequency domain method, the present invention has better noise reduction effect; compared with the time domain filtering and summation method, the present invention can obtain better speech enhancement performance when the filter length is short.

[0058] 4. The speech enhancement method proposed in the present invention is universal and is not affected by the type of timing network module. It has a good enhancement effect under different types of networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a flow chart of a low-latency speech enhancement method based on microphone array;

[0060] Figure 2 A flowchart of a low-latency speech enhancement method based on a microphone array;

[0061] Figure 3 Schematic diagram of the indoor microphone array and sound source location distribution. DETAILED DESCRIPTION

[0062] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0063] The low-latency speech enhancement method based on a microphone array described in this embodiment includes at least the following steps 1 to 4, as shown in the flowchart. Figure 1 and Figure 2 As shown, the speech enhancement of microphone arrays applicable to any array aims to suppress the background noise of the microphone receiving signal by using beamforming technology without considering the removal of room reverberation.

[0064] In this embodiment, a microphone array consisting of M array elements is used to receive a speech signal, and the noisy signal received by the mth microphone is:

[0065] u m (t) = y m (t)+n m (t),m=1,2,…,M

[0066] In the formula, is the noise-free signal received by the microphone, i.e., the desired signal in the present invention, g m (t) is the room impulse response from the unknown sound source s(t) to the mth microphone, is the convolution operator, n m (t) is the additive noise at the mth microphone. It is assumed here that the noise and the sound source signal are uncorrelated and the additive noises are independent of each other.

[0067] In this example, a low-latency speech enhancement method based on a microphone array includes the following steps:

[0068] Step 1, setting a set of initial pole parameters, optimizing the initial pole parameters using an artificial neural network, obtaining real poles, and using the real poles as parameters of a multi-pole orthogonal basis function model of each channel of the microphone array;

[0069] Step 2, constructing an orthogonal basis function model of each channel of the microphone array using real poles, and calculating the filter responses of each order;

[0070] Step 3, performing frame division and time domain feature extraction on the microphone array received signal, and using an improved timing network to estimate the weights of an adaptive beamformer formed by an orthogonal basis function model;

[0071] Step 4: Calculate the system response of each channel of the beamforming network according to the filter response and the beamformer weight, and filter and sum the microphone array received signal to obtain an enhanced frame-level speech signal. Perform overlap-addition operation on the enhanced frame-level speech signal to obtain an enhanced complete speech signal.

[0072] Further, step 1 comprises:

[0073] A microphone array consisting of M array elements is used to receive the signal. Different orthonormal basis function (OBF) models (i.e., OBF filters) are used for different microphone channels. For microphone array channel m, m = 1, 2, ..., M, an unbiased one-dimensional convolutional neural network with 1 input channel and L output channels is used to train the initial pole parameters. M is the number of microphones, L is the filter length, the kernel size and step size of the one-dimensional convolutional neural network are set to 1, and the data with a constant value of 1 is used as the input item. The output item is the learnable weight of the one-dimensional convolutional neural network, and the weight is used as the pole parameter of the orthogonal basis function model. The training process is expressed as:

[0074]

[0075] In the formula, Convld represents a one-dimensional convolutional neural network; l represents the order of the filter;

[0076] The initial extreme parameters obtained from each training Boundary processing is performed to ensure the stability of the system, and the pole a after boundary processing is ml As the poles of the orthogonal basis function model; where the boundary treatment is expressed as:

[0077]

[0078] Where HardTanh(·) is the activation function, expressed as:

[0079]

[0080] In the formula, a max and a min They represent the maximum and minimum values ​​of the linear range of the HardTanh(·) function. In this example, the linear range can be set to [0,0.9999], that is, a min Corresponding to 0, a max Corresponds to 0.9999.

[0081] Further, step 2 includes:

[0082] The state space equation of the orthogonal basis function model of the mth channel of the microphone array is expressed as:

[0083] x m (t) = A m x m (t-1)+b m u m (t)

[0084]

[0085] In the formula, is the state vector at t discrete time points, is the state matrix, is the input-state vector, u m (t) is the signal received by the microphone, w m is the state-output vector, i.e., the weight of the orthogonal basis function model, is the orthogonal basis function model output, [·] T Represents the transpose of a matrix or vector;

[0086] According to the state space equations and real poles a of the orthogonal basis function model ml Calculate the state matrix A m The (j,l) element and input-state vector b m The lth element of is represented as:

[0087]

[0088] in, is the orthogonal state space realization matrix of the lth first-order all-pass filter that constitutes the orthogonal basis function model of the mth channel, and the elements in the matrix are related to the pole a ml The relationship is: d ml =-a ml ;

[0089] Set the initial state vector x m(0) = 0, then the state space equation of the orthogonal basis function model is expressed as:

[0090]

[0091] Where i represents the discrete time index, Represents the state matrix A m the i-1th power;

[0092] Through the state matrix A m and the input-state vector b m , the calculated length is L h The L-order filter response is:

[0093] f m =[f mLh ,…,f mi ,…,f m2 ,f m1 ]

[0094] in, represents the L-order filter response vector at the i-th time point, i=1,2,…,L h , L h ≥L.

[0095] Further, step 3 includes:

[0096] The microphone received signal is divided into frames, the frame length is W, and the frame shift J=W / 2. Then the k-th frame signal of the m-th channel of the microphone array is expressed as:

[0097] u m (k)=[u m ((k-1)J+1),…,u m ((k-1)J+W)] T

[0098] In the case of non-causal feature extraction and filtering, the current frame signal and its context information are spliced ​​to obtain the spliced ​​signal:

[0099] v m (k)=[u m ((k-1)J-C+1),…,u m ((k-1)J+W+C)] T

[0100] In the formula, C represents the length of the previous or next concatenation information;

[0101] The reference microphone receives the signal u ref (k) and v m (k) Extract the feature vector ξ of each channel of the microphone array m(k), and the eigenvector ξ m (k) is input into the improved sequential network SeqNet(·) to obtain the weights of the adaptive wave former.

[0102] In one example, the microphone received signal is framed, such as frame length W = 80, that is, 10ms is one frame, frame shift H = W / 2, then the kth frame signal of the mth channel can be obtained. If microphone No. 1 is selected as the reference channel, under the condition of non-causal feature extraction and filtering, the current frame signal of each channel and its context information are spliced ​​to obtain the splicing information, and the feature vector ξ of each channel input into the timing network is calculated m (k).

[0103] The temporal convolutional network (TCN) or dual-path recurrent neural network (DPRNN) modules are stacked into a temporal network SeqNet(·), and then the temporal network is improved, that is, a one-dimensional convolutional neural network and an activation function layer are stacked after the output layer of SeqNet(·). Among them, the TCN network is directly set to process data in real-time online mode, and the network delay of the TCN module is W+C; the DPRNN network is changed to real-time online mode to process data, that is, the intra-block RNN ​​uses a bidirectional long short-term memory unit, the inter-block RNN ​​uses a unidirectional long short-term memory unit, and the normalization layer uses a causal normalization form, then the network delay of the DPRNN module is (K+1)W / 2+C, K=4 is the block length of the DPRNN for the temporal sequence.

[0104] The improved sequential network SeqNet(·) starts with a linear bottleneck layer with B output channels. The weights of the adaptive wave formers at the frame level corresponding to the state space equation are estimated as follows:

[0105] w m (k) = OutputLayer(SeqNet(ξ m (k)))

[0106] In the formula, w m (k)=[w m,1 (k),…,w m,L (k)] T is the component vector of the adaptive wave former weight;

[0107] The output layer OutputLayer(·) of the improved sequential network SeqNet(·) consists of a one-dimensional convolutional neural network and an activation function, which is specifically set as follows:

[0108] OutputLayer(p)=PReLU(n(Wp+q))

[0109] In the formula, is the output of the timing network, and are the weights and biases of the one-dimensional convolutional neural network, η is the scale factor that controls the output of the one-dimensional convolutional neural network, and PReLU(·) is the parameter rectified linear unit activation function.

[0110] Furthermore, through the calculation of the filter responses of each order and the weight vector of the adaptive beamformer, the system response of each channel in the beamforming network is obtained as follows:

[0111]

[0112] When L h > L, add L before the convolution concatenation signal h -L zeros are added to ensure that the length of the convolved signal is equal to the expected signal length. The splicing information after zero padding is:

[0113]

[0114] A one-dimensional convolutional neural network is used to implement the beamforming network to filter the received signals of each channel, and the output signals of all channels are summed to obtain the enhanced k-th frame speech signal:

[0115]

[0116] In the formula, * represents the convolution operation of the one-dimensional convolutional neural network, and the relationship between the filter length and the context signal length is L=2C+1.

[0117] In this example, the scale-invariant source-to-noise ratio (SI-SNR) is used as the loss function, and the Adam optimizer with an initial learning rate of 0.001 is used to train the beamforming network proposed in the present invention. The improvement of the objective speech quality (PESQ), the short-time objective intelligibility (STOI), the SI-SNR and the signal to distortion ratio (SDR) are used as performance evaluation indicators.

[0118] The invention is described in detail below with reference to a simulation example.

[0119] Example 1

[0120] Figure 3 The figure shows the indoor microphone array and the distribution of the sound source positions used in this example. Assume that the length, width, and height of the room are x room ,y room 、h room , the reference microphone position is (x ref ,y ref ,h ref ), the sound source position is (x s ,y s ,h s ), h s =h ref , the speed of sound propagation in the air is c = 340m / s. Four omnidirectional microphones M1, M2, M3, and M4 are used to form a uniform linear array to receive speech signals, and the array element spacing D = 0.04m. Taking M1 as the reference channel, the distance between the sound source and the reference microphone is r, and the azimuth angle is θ. The virtual source method is used to establish the room reverberation model. The sampling frequency of the signal is set to 8kHz, the frame length is 10ms, and the frame shift is half of the frame length. The number of sampling points of a frame of speech signal is W = 80. The speaker's voice is randomly selected from the TIMIT speech library as the sound source signal to ensure that the speech of the training, verification, and test sets are different, accounting for 65%, 10%, and 25%. Assuming that the additive noise between the channels is uncorrelated, during the training and verification process, 9 types of noise are selected from the Freesound sound effect library, and white noise is selected from the NOISEX-92 noise library to be added to the reverberant speech received by the microphone array. During the test, three noise signals, babble, destroyerops, and factory, were selected from the NOISEX-92 noise library and added to the reverberated speech. Data was generated under different combinations of reverberation time (RT60), signal-to-noise ratio (SNR), noise type, room size, microphone array position, sound source position, and speaker voice signal conditions. Among them, the reverberation time (RT60), room length x room 、Room width room 、Room height h room The distance r between the sound source and the reference microphone and the azimuth angle θ are randomly selected in the range of [0.2, 0.6] s, [6.5, 9.5] m, [3.5, 6.5] m, [2.5, 3.5] m, [1.5, 2] m and [0, 180] ° respectively. The signal-to-noise ratio during training and validation is randomly selected in the range of [-5, 5] dB. The signal-to-noise ratio during testing is fixed at 0 dB and 5 dB. The distance between the reference microphone position and the center of the room is randomly selected in the range of [0, 0.5] m. The sound source position is at least 0.5 m away from the room wall.

[0121] Table 1 shows the network performance comparison of the method of the present invention (denoted as OBFNet) and the filter summation network FasNet under the condition of non-causal feature extraction and filtering, when the filter length L=51, using TCN and DPRNN for beamformer weight estimation. Since the present invention takes contextual information into account in the feature extraction process, the speech enhancement effect under non-causal feature extraction is more significant. Under the same filter order, the method of the present invention can show better performance than FasNet under various evaluation indicators; compared with TCN with a fixed receptive field, DPRNN can make full use of intra-frame and inter-frame information, and the network training effect is also better. This trend remains unchanged after adding the degree of freedom expansion, indicating that the method proposed by the present invention is suitable for speech enhancement of various time series networks.

[0122] Table 1

[0123]

[0124] Comparative Example 1

[0125] The data generation method is the same as that in Example 1. Table 2 shows the comparison of speech enhancement performance of the method of the present invention and FasNet under different filter lengths L under non-causal feature extraction and filtering, using DPRNN to train beamformer weights. Obviously, the method of the present invention can achieve the speech enhancement effect of FasNet with a longer filter length under a shorter filter length, while reducing the network algorithm delay.

[0126] Table 2

[0127]

[0128]

[0129] Comparative Example 2

[0130] In this comparative example, the frame length of the short-time Fourier transform of the frequency domain method is set to 64ms, and the frame shift is half of the frame length. The data generation method is the same as that described in Example 1. Table 3 shows the comparison of speech enhancement performance of the method of the present invention with the traditional beamforming methods TI-MVDR, BeamformIt and the existing DL-based frequency domain beamforming methods GRN-IRM-MVDR and BeamTasNet-sigMVDR when the filter length L=51 under non-causal feature extraction and filtering. Compared with the frequency domain method, the method of the present invention as a time domain method can show good noise reduction performance under shorter frame length and filter length, among which the advantages of the OBFNet-DPRNN method are more prominent.

[0131] Table 3

[0132] method △PESQ △STOI(%) SI-SNR(dB) SDR(dB) Oracle TI-MVDR 0.45 14.91 7.41 8.53 BeamformIt 0.20 5.72 4.13 5.56 GRN-IRM-MVDR 0.37 9.33 7.38 8.66 BeamTasNet-sigMVDR 0.51 12.70 8.55 9.87 OBFNet-TCN 0.85 19.73 11.43 12.33 OBFNet-DPRNN 0.92 20.76 11.82 12.62

Claims

1. A low-latency speech enhancement method based on a microphone array, characterized in that: The steps include: Step 1, setting a set of initial pole parameters, optimizing the initial pole parameters using an artificial neural network, obtaining real poles, and using the real poles as parameters of a multi-pole orthogonal basis function model of each channel of the microphone array; Step 2, constructing an orthogonal basis function model of each channel of the microphone array using real poles, and calculating the filter responses of each order; Step 3, performing frame division and time domain feature extraction on the microphone array received signal, and using an improved timing network to estimate the weights of an adaptive beamformer formed by an orthogonal basis function model; Step 4: Calculate the system response of each channel of the beamforming network according to the filter response and the beamformer weight, filter and sum the microphone array received signal to obtain an enhanced frame-level speech signal, and perform overlap-addition operation on the enhanced frame-level speech signal to obtain an enhanced complete speech signal.

2. A low-latency speech enhancement method based on microphone array according to claim 1, characterized in that: Step 1 includes: For microphone array channels m, m = 1, 2, ..., M, an unbiased one-dimensional convolutional neural network with 1 input channel and L output channels is used to train the initial pole parameters M is the number of microphones, L is the filter length, the kernel size and step size of the one-dimensional convolutional neural network are set to 1, and the data with a constant value of 1 is used as the input item. The output item is the learnable weight of the one-dimensional convolutional neural network, which is expressed as: In the formula, Conv1d represents a one-dimensional convolutional neural network; l represents the order of the filter; The extreme points obtained from each training Perform boundary processing and process the pole a ml As the poles of the orthogonal basis function model; where the boundary treatment is expressed as: Where HardTanh(·) is the activation function, expressed as: In the formula, a max and a min Respectively represent the maximum and minimum values ​​of the linear range of the HardTanh(·) function.

3. The low-latency speech enhancement method based on microphone array according to claim 1, characterized in that: Step 2 includes: The state space equation of the orthogonal basis function model of the mth channel of the microphone array is: x m (t)=A m x m (t-1)+b m u m (t) In the formula, is the state vector at the tth discrete time point, is the state matrix, is the input-state vector, u m (t) is the signal received by the microphone, w m is the state-output vector, i.e., the weight of the orthogonal basis function model, is the orthogonal basis function model output, [·] T Represents the transpose of a matrix or vector; Calculate the state matrix A based on the state space equation of the orthogonal basis function model m The (j,l) element and input-state vector b m The lth elements of are: in, is the orthogonal state space realization matrix of the lth first-order all-pass filter that constitutes the mth channel orthogonal basis function model, and the elements in the matrix are related to the pole a ml The relationship is: d ml =-a ml ; Set the initial state vector x m (0) = 0, then the state space equation of the orthogonal basis function model is: Where i represents the discrete time index, Represents the state matrix A m the i-1th power; This gives a length of L h The L-order filter response is: in, represents the L-order filter response vector at the i-th time point, i=1,2,…,L h , L h ≥L.

4. The low-latency speech enhancement method based on microphone array according to claim 3, characterized in that: Step 3 includes: The microphone received signal is divided into frames, the frame length is W, and the frame shift J=W / 2. Then the k-th frame signal of the m-th channel of the microphone array is expressed as: u m (k)=·u m ((k-1)J+1),…,u m ((k-1)J+W)] T In the case of non-causal feature extraction and filtering, the current frame signal and its context information are spliced ​​to obtain the spliced ​​signal: v m (k)=[u m ((k-1)J-C+1),…,u m ((k-1)J+W+C)] T In the formula, C represents the length of the previous or next concatenation information; The reference microphone receives the signal u ref (k) and v m (k) Extract the feature vector ξ of each channel of the microphone array m (k), and the eigenvector ξ m (k) is input into the improved sequential network SeqNet(·) to obtain the weights of the adaptive wave former.

5. A low-latency speech enhancement method based on microphone array according to claim 4, characterized in that: The improved sequential network SeqNet(·) starts with a linear bottleneck layer with B output channels. The weights of the adaptive wave formers at the frame level corresponding to the state space equation are estimated as follows: w m (k)=OutputLayer(SeqNet(ξ m (k))) In the formula, w m (k)=[w m,1 (k),…,w m,L (k)] T is the component vector of the adaptive wave former weight; The output layer OutputLayer(·) of the improved sequential network SeqNet(·) consists of a one-dimensional convolutional neural network and an activation function, which is specifically set as follows: OutputLayer(p)=PReLU(n(Wp+q)) In the formula, is the output of the timing network, and are the weights and biases of the one-dimensional convolutional neural network, η is the scale factor that controls the output of the one-dimensional convolutional neural network, and PReLU(·) is the parameter rectified linear unit activation function.

6. The low-latency speech enhancement method based on microphone array according to claim 1, characterized in that: In step 4, the system response of each channel of the beamforming network is: When L h > L, add L before the convolution concatenation signal h -L zeros are added to ensure that the length of the convolved signal is equal to the expected signal length. The splicing information after zero padding is: A one-dimensional convolutional neural network is used to implement the beamforming network to filter the received signals of each channel, and the output signals of all channels are summed to obtain the enhanced k-th frame speech signal: In the formula, * represents the convolution operation of the one-dimensional convolutional neural network, and the relationship between the filter length and the context signal length is L=2C+1.

Images