An online speech separation method and device based on speech spectrum domain sparsity constraint

Abstract

This invention discloses an online speech separation method and apparatus based on the sparsity constraints of the speech spectrum domain, belonging to the field of information processing. The method includes the following steps: S1: Perform a short-time Fourier transform on the time-domain signal of each microphone to obtain a time-frequency domain representation, and construct the original frequency domain vector; S2: Construct an initialization separation matrix of dimension 2×M according to the direction of the target speech relative to the microphone array; S3: Construct an optimization function and calculate the separation matrix used to separate the target speech from the remaining signal; S4: Obtain the frequency domain signal of the target speech based on the calculated separation matrix, and then obtain the time domain signal of the target speech. This invention, based on the sparsity characteristics of speech signals in the spectrum domain, designs a new optimization function that does not require pre-whitening processing, ensuring the real-time performance of conference communication.

Description

Technical Field

[0001] The present invention belongs to the field of information processing, and in particular relates to an online speech separation method and device based on speech spectrum domain sparsity constraint. Background Art

[0002] Online conference systems based on microphone arrays as voice signal acquisition devices have been increasingly widely used. In actual use, the presence of significant noise, reverberation and other factors in conference rooms will reduce voice quality and further reduce the listening experience of online conferences. Beamforming based on multiple elements of microphone arrays is the most common method to reduce signal noise and improve communication quality. Furthermore, during conference calls, there are often multiple speakers speaking at the same time. How to selectively extract the voice of a certain person can further reduce the interference of competing speakers and improve the quality of the meeting. Furthermore, in application technologies such as automatic meeting minutes generation, extracting the voice of a target speaker is more important for improving speech recognition rate and the accuracy of meeting minutes.

[0003] Independent vector analysis (IVA) is currently the most commonly used blind source separation technology. This technology first converts the time domain signals picked up by all array elements into the time-frequency domain through short-time Fourier transform, then constructs an optimization function based on the principle of minimum mutual entropy of separated speech, iteratively updates the separation matrix based on the optimization function, and after estimating the separation matrix, the frequency domain estimation of the target signal can be obtained, and finally the time domain estimation is obtained based on the inverse Fourier transform.

[0004] The main disadvantages of the prior art are as follows:

[0005] 1) Based on the principle of minimum mutual entropy to optimize the function, it is necessary to buffer the signal for a long time to ensure the accuracy of the mutual entropy calculation. Therefore, the existing IVA technology has a large delay and cannot guarantee real-time communication.

[0006] 2) The existing IVA technology needs to perform pre-whitening processing first to ensure that all frequency bands can equally participate in the update of the separation matrix. However, the pre-whitening technology also needs to buffer the data for a period of time and cannot guarantee real-time performance.

[0007] In view of this, the present invention is proposed. Summary of the invention

[0008] The purpose of the present invention is to provide an online speech separation method and device based on the sparsity constraint of speech spectrum domain. Based on the sparsity characteristics of speech signals in the spectrum domain, a new optimization function is designed, which does not require pre-whitening processing and can ensure the real-time nature of conference communication.

[0009] In order to achieve the above object, the present invention provides an online speech separation method based on speech spectrum domain sparsity constraint, which is applied to a system based on a microphone array, comprising the following steps:

[0010] S1: Perform short-time Fourier transform on the time domain signal of each microphone to obtain the time-frequency domain expression and construct the original frequency domain vector;

[0011] S2: Construct an initial separation matrix with a dimension of 2×M according to the direction of the target speech relative to the microphone array;

[0012] S3: construct an optimization function and calculate a separation matrix for separating the target speech and the remaining signal;

[0013] S4: According to the calculated separation matrix, the frequency domain signal of the target speech is obtained, and then the time domain signal of the target speech is obtained.

[0014] Furthermore, before step S1, the method further includes: obtaining a time domain signal x of each microphone m (n);

[0015] The step S1 comprises:

[0016] For the time domain signal x m (n) Perform short-time Fourier transform to obtain the time-frequency domain expression:

[0017]

[0018] For each frequency band k, construct a frequency domain original vector X(l, k):

[0019] X(l,k)=[X1(l,k),X2(l,k),…,X M (l, k)] T .

[0020] Furthermore, in step S2, the initialized separation matrix is:

[0021]

[0022] Among them, the target speech separation matrix is ​​initialized as:

[0023]

[0024]

[0025] q(θ)=[cos(θ), sin(θ)]:

[0026] The residual signal separation matrix is ​​initialized to a zero vector:

[0027] G2(0,k)=[0,0,....,0].

[0028] Furthermore, the step S3 comprises:

[0029] S301: Construct a new optimization function J(G(k)):

[0030]

[0031] y1(l,k)=G1(k)X(l,k), y2(l,k)=G2(k)X(l,k)

[0032] Wherein, y1(l, k) and y2(l, k) represent the frequency domain estimation of the target speech and the residual signal based on the separation matrix, respectively;

[0033] S302: Calculate the gradient vector of the optimization function:

[0034]

[0035]

[0036] S303: Update the separation matrix according to the separation matrix of the previous frame and the gradient descent method:

[0037] G1(l,k)=G1(l-1,k)-εΔ(G1(k))

[0038] G2(l,k)=G2(l-1,k)-εΔ(G2(k))

[0039] Wherein, G1(l, k) and G2(l, k) are separation matrices used to separate the target speech and the residual signal respectively.

[0040] Furthermore, the step S4 comprises:

[0041] S401: Obtain frequency domain estimation of the target speech according to the obtained separation matrix:

[0042]

[0043] S402: Perform inverse Fourier transform on the frequency domain estimate to obtain the time domain signal of the target speech:

[0044]

[0045] The present invention also provides an online speech separation device based on speech spectrum domain sparsity constraint, which is applied to a system based on a microphone array, including an initialization module, a signal decomposition module, a separation filter calculation module and a target speech estimation module:

[0046] The initialization module is used to perform short-time Fourier transform on the time domain signal of each microphone to obtain the time-frequency domain expression and construct the original vector in the frequency domain;

[0047] The signal decomposition module is used to construct an initial separation matrix with a dimension of 2×M according to the direction of the target speech relative to the microphone array;

[0048] The separation filter calculation module is used to construct an optimization function and calculate a separation matrix for separating the target speech and the remaining signal;

[0049] The target speech estimation module is used to obtain the frequency domain signal of the target speech according to the calculated separation matrix, and then obtain the target speech time domain signal.

[0050] Furthermore, the initialization module is also used to obtain the time domain signal x of each microphone. m (n);

[0051] The signal decomposition module is used to decompose the time domain signal x m (n) Perform short-time Fourier transform to obtain the time-frequency domain expression:

[0052]

[0053] For each frequency band k, construct a frequency domain original vector X(l, k):

[0054] X(l,k)=[X1(l,k),X2(l,k),…,X M (l, k)] T .

[0055] Furthermore, in the signal decomposition module, the initialized separation matrix is:

[0056]

[0057] Among them, the target speech separation matrix is ​​initialized as:

[0058]

[0059]

[0060] q(θ)=[cos(θ),sin(θ)];

[0061] The residual signal separation matrix is ​​initialized to a zero vector:

[0062] G2(0,k)=[0,0,....,0].

[0063] Furthermore, the operation steps of the separation filter calculation module are as follows:

[0064] First, construct a new optimization function J(G(k)):

[0065]

[0066] y1(l,k)=G1(k)X(l,k), y2(l,k)=G2(k)X(l,k)

[0067] Wherein, y1(l, k) and y2(l, k) represent the frequency domain estimation of the target speech and the residual signal based on the separation matrix, respectively;

[0068] Second, calculate the gradient vector of the optimization function:

[0069]

[0070]

[0071] Finally, the separation matrix is ​​updated according to the separation matrix of the previous frame and the gradient descent method:

[0072] G1(l,k)=G1(l-1,k)-εΔ(G1(k))

[0073] G2(l,k)=G2(l-1,k)-εΔ(G2(k))

[0074] Wherein, G1(l, k) and G2(l, k) are separation matrices used to separate the target speech and the residual signal respectively.

[0075] Furthermore, the operation steps of the speech estimation module are as follows:

[0076] First, according to the solved separation matrix, the frequency domain estimation of the target speech is obtained:

[0077]

[0078] Secondly, the frequency domain estimate is inverse Fourier transformed to obtain the time domain signal of the target speech:

[0079]

[0080] The present invention provides an online speech separation method and device based on speech spectrum domain sparsity constraint, which has the following beneficial effects:

[0081] 1. Compared with traditional IVA, the present invention optimizes the objective function based on sparsity constraints, does not require pre-whitening processing, and therefore does not need to buffer data for a period of time, and can support real-time communication needs.

[0082] 2. The objective function adopted by the present invention conforms to the assumption of speech signal sparsity. The separated speech contains less noise and voices of competing speakers, which can significantly improve the quality of voice communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 This is a flowchart of the online speech separation method based on speech spectrum domain sparsity constraint in this specific implementation.

[0084] Figure 2 It is a schematic diagram of the Hamming window function used in this specific implementation.

[0085] Figure 3 It is a schematic diagram of an online speech separation device based on speech spectrum domain sparsity constraint in this specific implementation manner. DETAILED DESCRIPTION

[0086] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods.

[0087] like Figure 1 As shown, one embodiment of the present invention is an online speech separation method based on sparsity constraints in the speech spectrum domain, which can be applied to a voice conferencing system with a microphone array, can extract speech signals in a certain direction in real time, helps to improve the quality of conference voice communications, and can also be combined with speech recognition technology to automatically record meeting minutes.

[0088] The specific implementation steps include the following four steps:

[0089] S1: Perform short-time Fourier transform on the time domain signal of each microphone to obtain the time-frequency domain expression and construct the original frequency domain vector.

[0090] Before step S1, the voice signal of each microphone is obtained. The obtained voice signal is as follows: Assume x m (n) represents the original time domain signal picked up by M microphone array elements in real time, where m represents the microphone serial number label, whose value ranges from 1 to M, and n represents the time label.

[0091] Specifically, the method for performing short-time Fourier transform is as follows:

[0092] For the time domain signal x m (n) Perform short-time Fourier transform to obtain the time-frequency domain expression:

[0093]

[0094] Where N is the frame length, N = 512; w(n) is a Hamming window of length 512, where n represents the time sequence number, so w(n) represents the value at each corresponding time sequence number n; 1 is the time frame sequence number, in frames; k is the frequency band sequence number, where the frequency band refers to the signal component corresponding to a certain frequency; j represents the imaginary unit X m (l, k) is the spectrum of the kth frequency band in the first frame of the mth microphone signal. In the present invention, the Hamming window function used is as follows: Figure 2 shown.

[0095] The method to construct the original vector in the frequency domain is as follows:

[0096] For each frequency band k, construct a frequency domain original vector X(l, k):

[0097] X(l,k)=[X1(l,k),X2(l,k),…,X M (l, k)] T

[0098] The superscript T represents the transposition operator, and it can be seen that the original vector is an M-dimensional column vector.

[0099] Through the above step S1, the transformation of the time domain signal into the time-frequency domain can be completed.

[0100] S2: Construct an initial separation matrix with a dimension of 2×M according to the direction of the target speech relative to the microphone array.

[0101] Among them, the direction of the target speech relative to the microphone array is θ. The target speech refers to the speech signal corresponding to the target direction. For the speech separation task, the target direction is usually used as prior information based on the actual application and is known in advance based on the extracted signal. For example, for large-screen voice communication equipment, it is hoped that the speech signal separated is at a 90-degree direction.

[0102] Therefore, the initial separation matrix is:

[0103]

[0104] Among them, in order to extract the speech in the target direction, the target speech separation matrix is ​​initialized as:

[0105]

[0106]

[0107] q(θ)=[cos(θ),sin(θ)]

[0108] Among them, f kis the frequency of the kth frequency band, k = 1, 2, ... K, where the value of K is determined according to the subsequent Fourier transform. If the frame length is 512, then the value of K is half of the frame length; c is the speed of sound, c = 340 m / s; d m is the two-dimensional coordinate value of the mth microphone; q(θ) is the direction vector, ω k is the band circle frequency.

[0109] To extract the residual signal, the residual signal separation matrix is ​​initialized to a zero vector:

[0110] G2(0,k)=[0,0,....,0]

[0111] Through the above step S2, the initialization of the separation matrix can be completed.

[0112] S3: Construct an optimization function and calculate a separation matrix for separating the target speech from the remaining signal.

[0113] Specifically, this step S3 includes the following steps:

[0114] S301: Construct a new optimization function J(G(k)):

[0115]

[0116] y1(l,k)=G1(k)X(l,k), y2(l,k)=G2(k)X(l,k)

[0117] Where α is a weight factor, |.| represents taking the modulus of a complex number, and the superscript * represents taking the conjugate of a complex number. y1(l, k) and y2(l, k) represent the frequency domain estimates of the target speech and the residual signal based on the separation matrix, respectively. Where G1(0, k) and G2(0, k) represent the initial values ​​of G1(k) and G2(k), respectively.

[0118] In the above optimization function formula, the first term of the function constrains the sum of the two separated signals to be as close as possible to X1(l, k), and the second term constrains the correlation between the separated signals y1(l, k) and y2(l, k) to be as small as possible. That is, the principle of this optimization function is to ensure that the difference between the sum of y1(l, k) and y2(l, k) and X1(l, k) is as small as possible while the correlation between y1(l, k) and y2(l, k) is as small as possible.

[0119] Among them, the residual signal is relative to y1(l, k), y2(l, k) and X1(l, k). As mentioned above, the optimization function J(G(k)) is constrained to make the difference between the sum of y1(l, k) and y2(l, k) and X1(l, k) as small as possible, that is, y1(l, k) and y2(l, k) are separated from X1(l, k). y1(l, k) represents the target speech, and y2(l, k) is the residual signal after y1(l, k) is separated from X1(l, k). The residual signal is not the same as the background signal. The background signal refers to the background noise signal outside the sound source.

[0120] The first term in the optimization function J(G(k)) ensures that the two separated signals are as equal as possible to the microphone recording signals, and the second term constrains the output two signals to be as sparse as possible.

[0121] The larger the value of the weight factor α, the stronger the sparsity constraint, making the separated speech sparser, but it will cause the inconsistency between the two separated signals and the original microphone signal to become higher. In the present invention, the recommended reference value is between 1 and 10. Preferably, the present invention adopts α=3. The preferred threshold value can achieve a balance between signal consistency and separation effectiveness.

[0122] S302: Calculate the gradient vector of the optimization function:

[0123]

[0124]

[0125] Since the optimization function in step S301 has no analytical solution, step S302 is used to calculate the gradient vector, and then the optimization function can be solved step by step, and the separation effect is more in line with the sparsity constraint. Wherein, G(0, k) represents the initialization of G(k).

[0126] S303: Update the separation matrix according to the separation matrix of the previous frame and the gradient descent method:

[0127] G1(l,k)=G1(l-1,k)-εΔ(G1(k))

[0128] G2(l,k)=G2(l-1,k)-εΔ(G2(k))

[0129] Among them, G1(l, k) and G2(l, k) are separation matrices used to separate the target speech and the residual signal respectively; ∈ represents the step size of each update, and the reference value range is between 0.05 and 0.2.

[0130] Preferably, in the present invention, the step size is ∈=0.1. If the step size is too large, the update will be unstable (poor robustness), and if the step size is too small, the convergence speed will be slow; the present invention preferably uses 0.1 as the step size, which can achieve a better balance between convergence speed and robustness.

[0131] In the formula, the target speech separation matrix is ​​initialized to G1(0, k), and the separation matrix of the target speech of the first frame is G1(1, k), that is, it can be updated from G1(0, k), and the second frame is updated from the first frame, and so on, to obtain the separation matrix of the target speech G1(l, k). The calculation process of the separation matrix of the remaining signal is the same as above.

[0132] Through the above steps, the separation matrix can be optimized and solved, which can be directly used in the next step to extract the target speech spectrum and time domain signal.

[0133] S4: According to the calculated separation matrix, the frequency domain signal of the target speech is obtained, and then the time domain signal of the target speech is obtained.

[0134] The specific steps include:

[0135] S401: Obtain frequency domain estimation of the target speech according to the obtained separation matrix:

[0136]

[0137] S402: Perform inverse Fourier transform on the frequency domain estimate to obtain the time domain signal of the target speech:

[0138]

[0139] Through this step S4, the time domain signal of the target speech can be acquired.

[0140] Through the above steps S1-S4 of the present invention, the initialization of the microphone matrix signal, signal decomposition, separation matrix update and target speech estimation can be achieved, and finally the target speech can be extracted.

[0141] like Figure 3 As shown, one embodiment of the present invention is an online speech separation device based on speech spectrum domain sparsity constraint, which is applied to a microphone array-based system, and includes an initialization module 1, a signal decomposition module 2, a separation filter calculation module 3 and a target speech estimation module 4.

[0142] Initialization module 1 is used to perform short-time Fourier transform on the time domain signal of each microphone to obtain the time-frequency domain expression and construct the original frequency domain vector.

[0143] The initialization module 1 can also be used to obtain the speech signal of each microphone. The obtained speech signal is as follows: Assume xm (n) represents the original time domain signal picked up by M microphone array elements in real time, where m represents the microphone serial number label, whose value ranges from 1 to M, and n represents the time label.

[0144] Specifically, the method for performing short-time Fourier transform is as follows:

[0145] For the time domain signal x m (n) Perform short-time Fourier transform to obtain the time-frequency domain expression:

[0146]

[0147] Where N is the frame length, N = 512; w(n) is a Hamming window of length 512, where n represents the time sequence number, so w(n) represents the value at each corresponding time sequence number n; l is the time frame sequence number, in frames; k is the frequency band sequence number, where the frequency band refers to the signal component corresponding to a certain frequency; j represents the imaginary unit X m (l, k) is the spectrum of the kth frequency band in the lth frame and the mth microphone signal. In the present invention, the Hamming window function used is as follows: Figure 2 shown.

[0148] The method to construct the original vector in the frequency domain is as follows:

[0149] For each frequency band k, construct a frequency domain original vector X(l, k):

[0150] X(l,k)=[X1(l,k),X2(l,k),…,X M (l, k)] T

[0151] The superscript T represents the transposition operator, and it can be seen that the original vector is an M-dimensional column vector.

[0152] By initializing module 1, the transformation of the time domain signal into the time-frequency domain can be completed.

[0153] The signal decomposition module 2 is used to construct an initialization separation matrix with a dimension of 2×M according to the direction of the target speech relative to the microphone array.

[0154] Among them, the direction of the target speech relative to the microphone array is θ. The target speech refers to the speech signal corresponding to the target direction. For the speech separation task, the target direction is usually used as prior information based on the actual application and is known in advance based on the extracted signal. For example, for large-screen voice communication equipment, it is hoped that the speech signal separated is at a 90-degree direction.

[0155] Therefore, the initial separation matrix is:

[0156]

[0157] Among them, in order to extract the speech in the target direction, G1 is initialized as:

[0158]

[0159]

[0160] q(θ)=[cos(θ),sin(θ)]

[0161] Among them, f k is the frequency of the kth frequency band, k = 1, 2, ... K, where the value of K is determined according to the subsequent Fourier transform. If the frame length is 512, then the value of K is half of the frame length; c is the speed of sound, c = 340 m / s; d m is the two-dimensional coordinate value of the mth microphone; q(θ) is the direction vector, ω k is the band circle frequency.

[0162] To extract the residual signal, G2 is initialized to a zero vector:

[0163] G2(0,k)=[0,0,....,0]

[0164] The signal decomposition module 2 can be used to initialize the separation matrix.

[0165] The separation matrix calculation module 3 is used to construct an optimization function and calculate a separation matrix for separating the target speech and the residual signal.

[0166] Specifically, the operation steps of the separation filter calculation module 3 are as follows:

[0167] First, construct a new optimization function J(G(k)):

[0168]

[0169] y1(l,k)=G1(k)X(l,k), y2(l,k)=G2(k)X(l,k)

[0170] Where α is a weight factor, |.| represents taking the modulus of a complex number, and the superscript * represents taking the conjugate of a complex number. y1(l, k) and y2(l, k) represent the frequency domain estimates of the target speech and the residual signal based on the separation matrix, respectively. Where G1(0, k) and G2(0, k) represent the initial values ​​of G1(k) and G2(k), respectively.

[0171] The first term in the optimization function J(G(k)) ensures that the two separated signals are as equal as possible to the microphone recording signals, and the second term constrains the output two signals to be as sparse as possible.

[0172] The larger the value of the weight factor α, the stronger the sparsity constraint, making the separated speech sparser, but it will cause the inconsistency between the two separated signals and the original microphone signal to become higher. In the present invention, the recommended reference value is between 1 and 10. Preferably, the present invention adopts α=3. The preferred threshold value can achieve a balance between signal consistency and separation effectiveness.

[0173] Second, calculate the gradient vector of the optimization function:

[0174]

[0175]

[0176] Since the constructed optimization function has no analytical solution, the above method is used to calculate the gradient vector, and then the optimization function can be solved step by step, and the separation effect is more in line with the sparsity constraint.

[0177] Finally, the separation matrix is ​​updated according to the separation matrix of the previous frame and the gradient descent method:

[0178] G1(l,k)=G1(l-1,k)-εΔ(G1(k))

[0179] G2(l,k)=G2(l-1,k)-εΔ(G2(k))

[0180] Among them, G1(l, k) and G2(l, k) are separation matrices used to separate the target speech and the residual signal respectively; ∈ represents the step size of each update, and the reference value range is between 0.05 and 0.2.

[0181] Preferably, in the present invention, the step size is ∈=0.1. If the step size is too large, the update will be unstable (poor robustness), and if the step size is too small, the convergence speed will be slow; the present invention preferably uses 0.1 as the step size, which can achieve a better balance between convergence speed and robustness.

[0182] In the formula, the target speech separation matrix is ​​initialized to G1(0, k), and the separation matrix of the target speech of the first frame is G1(1, k), that is, it can be updated from G1(0, k), and the second frame is updated from the first frame, and so on, to obtain the separation matrix of the target speech G1(l, k). The calculation process of the separation matrix of the remaining signal is the same as above.

[0183] Through the separation matrix calculation module 3, the separation matrix can be optimized and solved, which can be directly used in the next step to extract the target speech spectrum and time domain signal.

[0184] The target speech estimation module 4 is used to obtain the frequency domain signal of the target speech according to the calculated separation matrix, and then obtain the time domain signal of the target speech.

[0185] Specifically, the operation steps of the target speech estimation module 4 are as follows:

[0186] First, according to the solved separation matrix, the frequency domain estimation of the target speech is obtained:

[0187]

[0188] Secondly, the frequency domain estimate is inverse Fourier transformed to obtain the time domain signal of the target speech:

[0189]

[0190] The target speech estimation module 4 can be used to obtain the time domain signal of the target speech.

[0191] In the above implementation, the four modules including the initialization module 1, the signal decomposition module 2, the separation filter calculation module 3 and the target speech estimation module 4 are indispensable. The absence of any module will result in the inability to extract the target speech.

[0192] The invention concept is described in detail using specific examples herein, and the description of the above embodiments is only used to help understand the core idea of ​​the invention. It should be pointed out that any obvious modification, equivalent substitution or other improvement made by a person of ordinary skill in the art without departing from the invention concept should be included in the protection scope of the present invention.

Claims

1. An online speech separation method based on speech spectrum domain sparsity constraint, applied to a microphone array-based system, characterized in that: The following steps are involved: S1: Perform short-time Fourier transform on the time domain signal of each microphone to obtain the time-frequency domain expression and construct the original frequency domain vector; S2: construct an initialization separation matrix with a dimension of 2×M according to the direction of the target speech relative to the microphone array; in step S2, the initialized separation matrix is: Among them, the target speech separation matrix is ​​initialized as: q(θ)=[cos(θ),sin(θ)]; Among them, f k is the frequency of the kth frequency band, k = 1, 2, ... K, where the value of K is determined according to the subsequent Fourier transform. If the frame length is 512, then the value of K is half of the frame length; c is the speed of sound; d M is the two-dimensional coordinate value of the Mth microphone; θ is the direction of the target speech relative to the microphone array, q(θ) is the direction vector, ω k is the band circular frequency; The residual signal separation matrix is ​​initialized to a zero vector: G2(0,k)=[0,0,....,0]; S3: constructing an optimization function and calculating a separation matrix for separating the target speech and the residual signal; the step S3 includes: S301: Construct a new optimization function J(G(k)): y1(l,k)=G1(k)X(l,k), y2(l,k)=G2(k)X(l,k) Where α is the weight factor; y1(l, k) and y2(l, k) represent the frequency domain estimation of the target speech and the residual signal based on the separation matrix respectively; X m (l, k) is the mth microphone signal, the spectrum of the kth frequency band in the lth frame, and X(l, k) is the original vector in the frequency domain; S302: Calculate the gradient vector of the optimization function: S303: Update the separation matrix according to the separation matrix of the previous frame and the gradient descent method: G1(l,k)=G1(l-1,k)-εΔ(G1(k)) G2(l,k)=G2(l-1,k)-εΔ(G2(k)) Where G1(l, k) and G2(l, k) are separation matrices used to separate the target speech and the residual signal respectively; ∈ represents the step size of each update; S4: According to the calculated separation matrix, the frequency domain signal of the target speech is obtained, and then the time domain signal of the target speech is obtained.

2. The online speech separation method based on speech spectrum domain sparsity constraint according to claim 1 is characterized in that: Before step S1, the step further includes: obtaining a time domain signal x of each microphone m (n); The step S1 comprises: For the time domain signal x m (n) Perform short-time Fourier transform to obtain the time-frequency domain expression: Wherein, N is the frame length, N=512; w(n) is the Hamming window of length 512, where n represents the time sequence number; 1 is the time frame sequence number, in frames; k is the frequency band sequence number; j represents the imaginary unit; X m (l, k) is the spectrum of the kth frequency band of the mth microphone signal in the first frame; For each frequency band k, construct a frequency domain original vector X(l, k): X(l,k)=[X1(l,k),X2(l,k),…,X M (l,k)] T 。 3. The online speech separation method based on speech spectrum domain sparsity constraint according to claim 1 is characterized in that: The step S4 comprises: S401: Obtain frequency domain estimation of the target speech according to the obtained separation matrix: S402: Perform inverse Fourier transform on the frequency domain estimate to obtain the time domain signal of the target speech:

4. An online speech separation device based on speech spectrum domain sparsity constraint, applied to a microphone array-based system, characterized in that: It includes initialization module, signal decomposition module, separation filter calculation module and target speech estimation module: The initialization module is used to perform short-time Fourier transform on the time domain signal of each microphone to obtain the time-frequency domain expression and construct the original vector in the frequency domain; The signal decomposition module is used to construct an initialization separation matrix with a dimension of 2×M according to the direction of the target speech relative to the microphone array; in the signal decomposition module, the initialization separation matrix is: Among them, the target speech separation matrix is ​​initialized as: q(θ)=[cos(θ),sin(θ)]; Among them, f k is the frequency of the kth frequency band, k = 1, 2, ... K, where the value of K is determined according to the subsequent Fourier transform. If the frame length is 512, then the value of K is half of the frame length; c is the speed of sound; d M is the two-dimensional coordinate value of the Mth microphone; θ is the direction of the target speech relative to the microphone array, q(θ) is the direction vector, ω k is the band circular frequency; The residual signal separation matrix is ​​initialized to a zero vector: G2(0,k)=[0,0,....,0]; The separation filter calculation module is used to construct an optimization function and calculate a separation matrix for separating the target speech and the remaining signal; the operation steps of the separation filter calculation module are as follows: First, construct a new optimization function J(G(k)): y1(l,k)=G1(k)X(l,k), y2(l,k)=G2(k)X(l,k) Wherein, y1(l, k) and y2(l, k) represent the frequency domain estimation of the target speech and the residual signal based on the separation matrix respectively; X m (l, k) is the mth microphone signal, the spectrum of the kth frequency band in the lth frame, and X(l, k) is the original vector in the frequency domain; Second, calculate the gradient vector of the optimization function: Finally, the separation matrix is ​​updated according to the separation matrix of the previous frame and the gradient descent method: G1(l,k)=G1(l-1,k)-εΔ(G1(k)) G2(l,k)=G2(l-1,k)-εΔ(G2(k)) Where G1(l, k) and G2(l, k) are separation matrices used to separate the target speech and the residual signal respectively; ∈ represents the step size of each update; The target speech estimation module is used to obtain the frequency domain signal of the target speech according to the calculated separation matrix, and then obtain the target speech time domain signal.

5. The online speech separation device based on speech spectrum domain sparsity constraint according to claim 4 is characterized in that: The initialization module is also used to obtain the time domain signal x of each microphone m (n); The signal decomposition module is used to decompose the time domain signal x m (n) Perform short-time Fourier transform to obtain the time-frequency domain expression: For each frequency band k, construct a frequency domain original vector X(l, k): X(l,k)=[X1(l,k),X2(l,k),…,X M (l,k)] T 。 6. The online speech separation device based on speech spectrum domain sparsity constraint according to claim 4 is characterized in that: The operation steps of the speech estimation module are as follows: First, according to the solved separation matrix, the frequency domain estimation of the target speech is obtained: Secondly, the frequency domain estimate is inverse Fourier transformed to obtain the time domain signal of the target speech:

Images