A sound separation method and system based on vector microphones

Abstract

The present application relates to the field of audio processing, in particular to a sound separation method and system based on a vector microphone, the method comprising: constructing a fourth-order cumulant matrix of a sound signal received by the vector microphone, solving sound source direction angle estimation and sound source pitch angle estimation of each incident signal; constructing a spatial characteristic matrix of each incident signal at different frequencies; based on the spatial characteristic matrix, calculating a spatial correlation matrix estimation of the sound signal at different frequencies and different frame numbers, and constructing a spatial correlation matrix of the sound signal at different frequencies and different frame numbers; constructing an orthogonal MNMF model; adjusting the variables of the MNMF through the multiplication update method, so that the difference between the spatial correlation matrix estimation and the spatial correlation matrix is less than a preset target, to obtain an optimal spatial correlation matrix estimation; and separating the sound signal based on the optimal spatial correlation matrix. The present application effectively solves the sound separation problem when the distance between sound sources is close, and improves the accuracy of sound separation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio processing, and in particular to a sound separation method and system based on a vector microphone. BACKGROUND

[0002] Sound separation is a signal processing technique that aims to separate individual sound sources from a mixed sound signal. This technology plays an important role in daily life and work, not only improving the performance of hearing aids and noise-cancelling headphones, but also helping to improve the experience and effect of telephone conferences, speech recognition, music production and many other fields. In simple terms, sound separation technology allows people to clearly hear the sound they want to focus on in a noisy environment, which is crucial for communication and understanding.

[0003] In the prior art, technicians usually perform sound separation based on the independent component analysis (ICA) method, which is based on the statistical independence of signals and maximizes the statistical independence of output signals to achieve blind source separation. However, when the environment becomes complex, such as when there is Gaussian noise or the source signal spacing is close, the performance of ICA will be affected. Gaussian noise can interfere with signal reception, reducing signal quality, while close source signal spacing can cause signal overlap and confusion, which can all cause problems with the accuracy of sound separation using ICA. SUMMARY

[0004] To solve the above problems, the present application provides a sound separation method and system based on a vector microphone.

[0005] The first aspect of the present application discloses a sound separation method based on a vector microphone, comprising:

[0006] receiving a sound signal comprising a plurality of incident signals using a vector microphone;

[0007] constructing a fourth-order cumulant matrix of the sound signal, and solving the sound source direction angle estimate and the sound source pitch angle estimate of each incident signal based on the fourth-order cumulant matrix;

[0008] based on the sound source direction angle estimate and the sound source pitch angle estimate, constructing a spatial characteristic matrix of each incident signal at different frequencies;

[0009] based on the spatial characteristic matrix, calculating a spatial correlation matrix estimate of the sound signal at different frequencies and different frame numbers, and constructing a spatial correlation matrix of the sound signal at different frequencies and different frame numbers;

[0010] based on the spatial characteristic matrix, the spatial correlation matrix estimate and the spatial correlation matrix, constructing an orthogonalized MNMF model;

[0011] The variable of the MNMF is adjusted by a multiplication update method, and the spatial correlation matrix is gradually updated until the difference between the spatial correlation matrix estimation and the spatial correlation matrix is less than a preset target, so as to obtain an optimal spatial correlation matrix estimation.

[0012] The sound signal is separated based on the optimal spatial correlation matrix.

[0013] Further, the sound signal is a sum of a target signal and noise, the target signal is a sum of products of each incident signal and a steering vector thereof, and the noise includes sound pressure noise and vibration noise on x, y and z axes.

[0014] Further, the step of solving the sound source direction angle estimation and the sound source pitch angle estimation of each incident signal based on the fourth-order cumulant matrix comprises:

[0015] The fourth-order cumulant matrix is subjected to eigenvalue decomposition to obtain a signal subspace and a noise subspace of the sound signal.

[0016] The sound source direction angles and the sound source pitch angles of each incident signal are calculated based on the noise subspace.

[0017] The sound source direction angle estimation and the sound source pitch angle estimation of the corresponding incident signal are selected from the sound source direction angles and the sound source pitch angles according to the correlation degree between the signal subspace and the noise subspace.

[0018] Further, a spatial characteristic matrix of each incident signal at different frequencies is constructed, and the spatial characteristic matrix H ip of the pth incident signal at the i th frequency is:

[0019]

[0020] wherein, is the sound source direction angle estimation of the pth incident signal, is the sound source pitch angle estimation of the pth incident signal.

[0021] Further, based on the spatial characteristic matrix, the step of calculating the spatial correlation matrix estimation of the sound signal at different frequencies and different frame numbers comprises:

[0022] The spatial correlation matrix estimation at the i th frequency and the frame number l is:

[0023]

[0024] wherein, t ik is the amplitude of the k th incident signal at the i th frequency, v kl is the gain of the short-time Fourier transform of the k th incident signal at the frame number l, K represents the number of incident signals, H ikXi, k represents the spatial characteristic matrix of the kth incident signal at the i th frequency.

[0025] Further, the step of constructing the spatial correlation matrix of the sound signal at different frequencies and different frame numbers comprises:

[0026] the spatial correlation matrix Xil at the i th frequency and the l th frame number il :

[0027]

[0028] x il represents the snapshot signal at the i th frequency and the l th frame number:

[0029]

[0030] v x (t) = Bs(t) + n x (t) ;

[0031] E[n p (t)n x (t)] = 0 M×M ;

[0032] wherein p(t) represents the sound pressure at the position of the vector microphone at time t, n p (t) represents the sound pressure noise at the position of the vector microphone at time t, n x (t) represents the noise at the position of the vector microphone at time t in the x-axis, 0 M×M is an MxM zero matrix, E represents the expectation operation, B is a steering vector, s(t) represents an incident signal vector, H represents a conjugate transpose, v x (t) represents the vibration velocity at the position of the vector microphone at time t in the x-axis.

[0033] Further, the objective function f of the MNMF is:

[0034]

[0035] wherein the elements in the basis matrix T represent the amplitudes of the sound sources at different frequencies, the elements in the coefficient matrix V represent the gains of the short-time Fourier transform of the sound sources at different frame numbers, H is a spatial characteristic matrix, i represents the index of the frequency, l represents the index of the frame, tr represents the trace of the matrix, X il represents the spatial correlation matrix, det is a determinant, λ1 and λ2 are preset regularization parameters, represents the square of the Frobenius norm, I represents a unit matrix, and the superscript T in the formula represents the transpose of the matrix.

[0036] Further, the step of adjusting the variables of the MNMF by the multiplicative update method, gradually updating the spatial correlation matrix, and making the difference between the spatial correlation matrix estimation and the spatial correlation matrix less than a preset target to obtain the optimal spatial correlation matrix estimation comprises:

[0037] An auxiliary function of the objective function f is constructed based on the majorization-minimization algorithm;

[0038] The basis matrix T and the coefficient matrix V are adjusted by the multiplicative update method, and the spatial characteristic matrix H is updated based on the adjusted basis matrix T and the adjusted coefficient matrix V until the IS divergence converges, to obtain the optimized basis matrix, the coefficient matrix and the spatial characteristic matrix;

[0039] The optimal spatial correlation matrix estimation is calculated according to the optimized basis matrix, the coefficient matrix and the spatial characteristic matrix.

[0040] Further, the step of separating the sound signal based on the optimal spatial correlation matrix comprises:

[0041] After the optimal spatial correlation matrix is multiplied by the phase spectrum of the sound signal, inverse short-time Fourier transform is performed to obtain the separated sound source signals.

[0042] The second aspect of the present application discloses a sound separation system based on a vector microphone, comprising:

[0043] A receiving module is configured to receive a sound signal comprising multiple incident signals using a vector microphone;

[0044] An angle estimation module is configured to construct a fourth-order cumulant matrix of the sound signal, and solve the sound source direction angle estimation and the sound source pitch angle estimation of each incident signal based on the fourth-order cumulant matrix;

[0045] A matrix construction module is configured to construct the spatial characteristic matrix of each incident signal at different frequencies based on the sound source direction angle estimation and the sound source pitch angle estimation;

[0046] A matrix calculation module is configured to calculate the spatial correlation matrix estimation of the sound signal at different frequencies and different frame numbers based on the spatial characteristic matrix, and construct the spatial correlation matrix of the sound signal at different frequencies and different frame numbers;

[0047] A model construction module is configured to construct an orthogonalized MNMF model based on the spatial characteristic matrix, the spatial correlation matrix estimation and the spatial correlation matrix;

[0048] An optimization module is configured to adjust the variables of the MNMF by the multiplicative update method, gradually update the spatial correlation matrix, and make the difference between the spatial correlation matrix estimation and the spatial correlation matrix less than a preset target to obtain the optimal spatial correlation matrix estimation;

[0049] A separation module is configured to separate the sound signals based on the optimal spatial correlation matrix.

[0050] The sound separation method based on the vector microphone provided by the application improves the signal-to-noise ratio by using the data space discrimination and noise suppression characteristics of the high-order covariance matrix, effectively solves the problem of Gaussian noise; the spatial characteristic matrix containing angle information is constructed, which can improve the discrimination of the vector spatial information characteristics in the case of close source signal spacing, the multi-channel non-negative matrix orthogonal constraint cost function is designed, and the iterative update rule is optimized, thereby effectively solving the sound separation problem in the case of close source spacing and improving the accuracy of sound separation. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0052] Figure 1 is a flow chart of a sound separation method based on a vector microphone disclosed by the embodiment of the application;

[0053] Figure 2 is a structural diagram of a sound separation system based on a vector microphone disclosed by the embodiment of the application. DETAILED DESCRIPTION

[0054] In order to make the technical personnel in the art better understand the application scheme, the following will combine the drawings in the embodiments of the application to clearly and completely describe the technical solutions in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0055] The terms "first", "second", and the like in the specification and claims of the application and the above drawings are used to distinguish different objects, not to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, or product including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or end.

[0056] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. As will be apparent to those of ordinary skill in the art, embodiments described herein can be combined with other embodiments.

[0057] Figure 1 A flow chart of a sound separation method based on a vector microphone according to the present disclosure is shown. The specific steps of the present disclosure will be described below in conjunction with a specific embodiment:

[0058] 1. Sound acquisition

[0059] A vector microphone is a device that can acquire spatial sound information. It receives sound from different directions and generates multi-channel audio signals. These different audio signals can represent the distribution of sound in different directions, forming a vector field. These signals often need to be pre-processed (e.g., filtering, noise reduction, etc.) to reduce the effects of noise and other irrelevant signals.

[0060] The aperture of the vector microphone is small, which can increase the directional resolution of the system, thereby improving the accuracy of sound source localization. The vector microphone used in this embodiment is composed of a sound pressure microphone and three co-point orthogonal particle velocity sensors, which can realize the measurement of sound pressure and three-dimensional orthogonal particle velocity at any spatial point. The signal receiving model of the vector microphone is:

[0061]

[0062] wherein x(t) = [p(t), v z (t), v x (t), v y (t)] is the sound pressure and three-axis orthogonal velocity signal, t represents time, p(t) represents the sound pressure at the position of the vector microphone at time t, v x (t), v y (t), v z (t) respectively represent the vibration velocity of the vector microphone at the position of the vector microphone at time t in the x-axis, y-axis, and z-axis, P represents the number of incident signals, the incident signal vector s(t) = [s1(t), …, s p (t), …, s P (t)] T , s p (t) is the pth incident signal, B = [b1, b2, …, b p ,…, b P ] is the steering vector, n(t) = [n p (t), nz (t),n x (t),n y (t)] T is 4xL k noise vector, n p (t) represents the sound pressure noise at the position of the vector microphone at time t, n x (t),n y (t),n z (t) respectively represent the noise at the position of the vector microphone at time t in the x-axis, y-axis and z-axis, L k is the number of snapshots.

[0063] the steering vector b of the pth incident signal p is:

[0064]

[0065] where the azimuth angle θ p ∈ [0, 2π], the elevation angle where are respectively the x-axis, y-axis and z-axis direction cosines.

[0066] define the 4xL k dimensional data received by the microphone as microphone received data x:

[0067]

[0068] where,

[0069] 2. High-order vector space azimuth angle estimation

[0070] The fourth-order cumulant of the obtained time-domain observation multi-channel data x is solved, the estimation factor decomposition formula of the spatial correlation matrix is obtained through the high-order introduction of the spatial characteristic matrix, and the accurate azimuth information of the sound source signal can be obtained through spectral search. The azimuth information of the sound source signal is introduced into the multi-channel non-negative matrix model, which provides prior information for the spatial covariance matrix and improves the sound separation effect.

[0071] The fourth-order cumulant matrix C of the signal reception model x(t) of the vector microphone x is:

[0072]

[0073] where cum() represents the cumulant calculation, x(a, l k ) and x(:, l k ) are the (a, l k )th element and lk List, It is the a-th element of the guide vector B, s4(l k ) is the lth incident signal of the fourth channel of the vector microphone. k n(a,l) elements, * represents complex conjugate, H represents conjugate transpose, n(a,l) k ) represents the a-th element of the fourth channel in the noise vector. Each channel corresponds to one sensor in the vector microphone. In this embodiment, the vector microphone has four sensors. Smooth the cumulative result to reduce error. Based on the Gaussian assumption, we have cum(n(4,l)). k ),n*(a,l k ),n(:,l k ),n H (:,l k If ))=0, then the fourth-order cumulant matrix C x Simplified to:

[0074]

[0075] in, Let s(t) be the fourth-order cumulant. det(Ψ)≠0, Ψ is full rank, and the Gaussian measurement noise in each channel of AVS can be suppressed based on the high-order cumulant.

[0076] For the fourth-order cumulant matrix C x Perform eigenvalue decomposition to obtain the signal subspace U s,p and noise subspace U v,p And based on the noise subspace U v,p Solution to sound source direction angle estimation Estimation of sound source pitch angle

[0077]

[0078] 3. Construct a spatial correlation matrix

[0079] Perform a short-time Fourier transform on the acquired time-domain observation signal.

[0080] Sound source direction angle estimation based on fourth-order cumulant estimation Estimation of sound source pitch angle Constructing the spatial characteristic matrix:

[0081] H ip =BB H ;

[0082] B = [b1, b2, ..., b p ,…,b P ]

[0083]

[0084] where H ip is the spatial characteristic matrix of the pth incident signal at frequency i, where i = 1, 2,..., I, and I is the total number of frequency bins, is the source direction angle estimate of the pth incident signal, is the source elevation angle estimate of the pth incident signal.

[0085] The vector microphone output snapshot signal is expressed as:

[0086] p(t) = Bs(t) + n p (t) ;

[0087] v x (t) = Bs(t) + n x (t) ;

[0088] v y (t) = Bs(t) + n y (t) ;

[0089] v z (t) = Bs(t) + n z (t) ;

[0090] v x (t), v y (t), v z (t) represent the vibration velocity of the vector microphone at the position in the x, y, z axis at time t, respectively

[0091] The covariance matrix containing the sound pressure and three-axis particle velocity signal is constructed, and the spatial covariance matrix pv x , pv y , pv z is constructed by sound pressure and x, y, z axis particle velocity signal. Assuming that the vector sound pressure measurement and the particle velocity channel measurement noise are independent, the sound pressure velocity covariance matrix pv x is combined, then:

[0092]

[0093] E[n p (t)n x (t)] = 0 M×M ;

[0094] where p(t) represents the sound pressure at the position of the vector microphone at time t, n p (t) represents the sound pressure noise at the position of the vector microphone at time t, n x (t) represents the noise in the x axis at the position of the vector microphone at time t, x ilRepresents the snapshot signal of the i-th frequency at the l-th frame number, 0 M×M Let x be an M×M dimensional zero matrix, and E denote the desired operation, based on x. il Obtain the spatial correlation matrix X il This can be expressed as a vector product:

[0095]

[0096] Where i = 1, ..., I, l = 1, ..., L represent the frequency number and frame number, respectively, and H represents the conjugate transpose. Spatial correlation matrix X il The matrix form is as follows:

[0097]

[0098] Where the diagonal elements represent the amplitude of the observed signal |x il |=[|x il1 |,……,|x ilM |] T The off-diagonal elements represent the phase difference between the r-th microphone and the m-th microphone.

[0099] Assuming the sound source location remains unchanged, introduce the spatial characteristic matrix H. ik , used to model the spatial characteristics of the k-th incident signal at frequency i, where H ik The spatial correlation matrix is ​​a Hermitian positive semi-definite matrix and is non-negative. Modeling the spatial correlation matrix using a product sum form yields the spatial correlation matrix estimate.

[0100]

[0101] Among them, t ik Let v be the amplitude of the k-th incident signal at frequency i. kl H represents the gain of the short-time Fourier transform of the k-th incident signal at frame number l, where K represents the number of incident signals. ik This represents the spatial characteristic matrix of the k-th incident signal at frequency i.

[0102] 4. Model Optimization

[0103] Design a cost function for orthogonally constrained multi-channel non-negative matrices, apply orthogonal constraints to the basis matrix T and coefficient matrix V, and construct an auxiliary function to solve the optimization iterative update rule to obtain the non-negative basis matrix and coefficient matrix of the sound separation algorithm based on vector microphones.

[0104] The amplitude t of the fundamental number k of the sound source at frequency i ik Construct a basis matrix T, with the gain v of the short-time Fourier transform of the sound source at frame number l and base k. klThe coefficient matrix V is constructed, and the Lagrange multiplier method is used to construct the MNMF (Multichannel Nonnegative Matrix Factorization) model, that is, the optimization model of the basis matrix T and the coefficient matrix V in the Multichannel Nonnegative Matrix Factorization (MNMF) model, and the objective function f is defined as:

[0105]

[0106] Wherein, d * The IS divergence measurement criterion is taken to evaluate the approximation degree between X il and det is the determinant, H is the spatial characteristic matrix, λ1 and λ2 are preset regularization parameters, representing the square of the Frobenius norm, and I represents the unit matrix.

[0107] Based on the Majorization-Minimization (MM) algorithm, the auxiliary function f of the objective function f is constructed + :

[0108]

[0109] Wherein, R is the first auxiliary variable matrix, and the element of R is R il , U is the second auxiliary variable matrix, and the element of U is U il , R il and U il are positive auxiliary variables, and

[0110] The auxiliary function f + satisfies:

[0111] 1) f (T, V, H) ≤ f + (T, V, H, R, U);

[0112] 2) f (T, V, H) = min R,U f + (T, V, H, R, U);

[0113] When R il and U il satisfy the following formula, the objective function f is equivalent to the auxiliary function f + :

[0114]

[0115] The basis matrix T and coefficient matrix V are adjusted using the multiplicative update method to minimize the objective function f, where the basis matrix T is updated as follows:

[0116]

[0117] The coefficient matrix V is updated as follows:

[0118]

[0119] Based on the adjusted basis matrix T and coefficient matrix V, the spatial characteristic matrix H is updated by solving the Riccati equation:

[0120] H ik AH ik =Q;

[0121]

[0122] Among them, H i ′ k For H before this adjustment ik The corresponding value.

[0123] Adjust the basis matrix T, coefficient matrix V, and spatial property matrix H until the IS (Itakura-Saito) divergence converges to obtain the optimized basis matrix T. final Optimized coefficient matrix V final and the optimized spatial characteristic matrix H final .

[0124] 5. Constructing a sound source separation signal

[0125] Based on the optimized basis matrix T final Optimized coefficient matrix V final and the optimized spatial characteristic matrix H final ,according to The spatial correlation matrix of each separated acoustic signal was estimated. After multiplying by the phase spectrum of the original signal, an inverse short-time Fourier transform is performed to obtain the separated acoustic signals.

[0126] Figure 2 A structural diagram of a sound separation system based on a vector microphone disclosed in this invention is shown. The sound separation system includes:

[0127] The receiving module is used to receive sound signals including multiple incident signals using a vector microphone;

[0128] An angle estimation module is configured to construct a fourth-order cumulant matrix of the sound signal, and solve a sound source direction angle estimation and a sound source pitch angle estimation of each incident signal based on the fourth-order cumulant matrix;

[0129] A matrix construction module is configured to construct a spatial characteristic matrix of each incident signal at different frequencies based on the sound source direction angle estimation and the sound source pitch angle estimation;

[0130] A matrix calculation module is configured to calculate a spatial correlation matrix estimation of the sound signal at different frequencies and different frame numbers based on the spatial characteristic matrix, and construct a spatial correlation matrix of the sound signal at different frequencies and different frame numbers;

[0131] A model construction module is configured to construct an orthogonalized MNMF model based on the spatial characteristic matrix, the spatial correlation matrix estimation and the spatial correlation matrix;

[0132] An optimization module is configured to adjust variables of the MNMF by a multiplicative update method, gradually update the spatial characteristic matrix, and make a difference between the spatial correlation matrix estimation and the spatial correlation matrix less than a preset target, so as to obtain an optimal spatial correlation matrix estimation;

[0133] A separation module is configured to separate the sound signal based on the optimal spatial correlation matrix.

[0134] In summary, the sound separation method and system based on the vector microphone disclosed by the application can improve the signal-to-noise ratio by using the data space discrimination degree and noise suppression characteristics of the high-order covariance matrix, effectively solve the problem of Gaussian noise, construct a spatial characteristic matrix containing angle information, improve the discrimination degree of the vector spatial information characteristics in the case of close source signal spacing, design a multi-channel non-negative matrix orthogonal constraint cost function, and optimize the iterative update rule, thereby effectively solving the sound separation problem in the case of close source spacing and improving the accuracy of sound separation. Therefore, the application effectively overcomes the shortcomings of the prior art and has high industrial utilization value.

[0135] The above embodiments only exemplarily illustrate the principles and effects of the application, and are not used to limit the application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical idea disclosed by the application should be covered by the claims of the application.

Claims

1. A sound separation method based on a vector microphone, characterized in that, include: Use a vector microphone to receive sound signals that include multiple incident signals; Construct a fourth-order cumulant matrix for the sound signal, and solve for the source direction angle and source elevation angle estimation for each incident signal based on the fourth-order cumulant matrix. Based on the source direction angle estimation and source elevation angle estimation, a spatial characteristic matrix of each incident signal at different frequencies is constructed. Based on the spatial characteristic matrix, the spatial correlation matrix of the sound signal at different frequencies and different frame numbers is estimated, and the spatial correlation matrix of the sound signal at different frequencies and different frame numbers is constructed. An orthogonalized MNMF model is constructed based on the spatial characteristic matrix, the estimated spatial correlation matrix, and the spatial correlation matrix. By adjusting the variables of MNMF through the multiplicative update method, the spatial characteristic matrix is ​​updated step by step, so that the difference between the estimated spatial correlation matrix and the spatial correlation matrix is ​​less than the preset target, thus obtaining the optimal spatial correlation matrix estimate. The sound signal is separated based on the optimal spatial correlation matrix.

2. The sound separation method based on a vector microphone according to claim 1, characterized in that, The sound signal is the sum of the target signal and the noise, and the target signal is the sum of the products of each incident signal and its guide vector; the noise includes sound pressure noise and vibration noise on the x, y, and z axes.

3. The sound separation method based on a vector microphone according to claim 1, characterized in that, The steps for estimating the source direction angle and elevation angle of each incident signal based on the fourth-order cumulant matrix include: Eigenvalue decomposition of the fourth-order cumulant matrix yields the signal subspace and noise subspace of the sound signal; The direction angle and elevation angle of multiple sound sources for each incident signal are calculated based on the noise subspace. Based on the correlation between the signal subspace and the noise subspace, the source direction angle and source elevation angle of the corresponding incident signal are estimated from multiple source direction angles and elevation angles.

4. The sound separation method based on a vector microphone according to claim 1, characterized in that, Construct the spatial characteristic matrix of each incident signal at different frequencies. For the p-th incident signal at frequency i, the spatial characteristic matrix H is... iP for: in, For the estimation of the sound source direction angle of the p-th incident signal, This is the source elevation angle estimate for the p-th incident signal.

5. The sound separation method based on a vector microphone according to claim 4, characterized in that, Based on the spatial characteristic matrix, the steps for estimating the spatial correlation matrix of a sound signal at different frequencies and frame numbers include: Spatial correlation matrix estimation at the i-th frequency and frame number l Among them, t ik Let v be the amplitude of the k-th incident signal at frequency i. kl H represents the gain of the short-time Fourier transform of the k-th incident signal at frame number l, where K represents the number of incident signals. ik This represents the spatial characteristic matrix of the k-th incident signal at frequency i.

6. The sound separation method based on a vector microphone according to claim 5, characterized in that, The steps for constructing the spatial correlation matrix of an audio signal at different frequencies and frame numbers include: The spatial correlation matrix X at the i-th frequency and frame number l il : x il This represents the snapshot signal of the i-th frequency at the l-th frame number: v x (t)=Bs(t)+n x (t); E[n p (t)n x (t)]=0 M×M ; Where p(t) represents the sound pressure at the location of the vector microphone at time t, and n p (t) represents the sound pressure noise at the location of the vector microphone at time t, n x (t) represents the noise along the x-axis at the location of the vector microphone at time t, 0 M×M Let E be an M×M zero matrix, E denote the desired operation, B be the steering vector, s(t) represent the incident signal vector, H represent the conjugate transpose, and v x (t) represents the vibration velocity of the vector microphone at time t along the x-axis.

7. The sound separation method based on a vector microphone according to claim 6, characterized in that, The objective function f of the MNMF is: In this matrix, the elements of the basis matrix T represent the amplitude of the sound source at different frequencies, the elements of the coefficient matrix V represent the gain of the short-time Fourier transform of the sound source at different frame numbers, H is the spatial characteristic matrix, i represents the frequency index, l represents the frame index, tr represents the trace of the matrix, and X... il The matrix represents the spatial correlation matrix, where det is the determinant, and λ1 and λ2 are preset regularization parameters. denoted by , I represents the square of the Frobenius norm, and represents the identity matrix. The superscript T in the formula represents the transpose of the matrix.

8. The sound separation method based on a vector microphone according to claim 7, characterized in that, The steps to adjust the variables of the MNMF using the multiplicative update method, gradually updating the spatial correlation matrix to ensure that the difference between the estimated spatial correlation matrix and the actual spatial correlation matrix is ​​less than a preset target, and obtaining the optimal spatial correlation matrix estimate include: Construct an auxiliary function for the objective function f based on the mastery-minimization algorithm; The basis matrix T and coefficient matrix V are adjusted by multiplication update method, and the spatial characteristic matrix H is updated based on the adjusted basis matrix T and coefficient matrix V until the IS divergence converges, thus obtaining the optimized basis matrix, coefficient matrix and spatial characteristic matrix. The optimal spatial correlation matrix estimate is calculated based on the optimized basis matrix, coefficient matrix, and spatial characteristic matrix.

9. The sound separation method based on a vector microphone according to claim 1, characterized in that, The steps for separating sound signals based on the optimal spatial correlation matrix include: After multiplying the optimal spatial correlation matrix by the phase spectrum of the sound signal, an inverse short-time Fourier transform is performed to obtain the separated sound source signals.

10. A sound separation system based on a vector microphone, characterized in that, include: The receiving module is used to receive sound signals including multiple incident signals using a vector microphone; The angle estimation module is used to construct the fourth-order cumulant matrix of the sound signal and solve the source direction angle estimation and source elevation angle estimation of each incident signal based on the fourth-order cumulant matrix. The matrix construction module is used to construct the spatial characteristic matrix of each incident signal at different frequencies based on the source direction angle estimation and the source elevation angle estimation. The matrix calculation module is used to calculate the spatial correlation matrix estimate of the sound signal at different frequencies and different frame numbers based on the spatial characteristic matrix, and to construct the spatial correlation matrix of the sound signal at different frequencies and different frame numbers. The model building module is used to construct an orthogonalized MNMF model based on the spatial characteristic matrix, the spatial correlation matrix estimation, and the spatial correlation matrix. The optimization module is used to adjust the variables of MNMF through the multiplicative update method, gradually update the spatial correlation matrix, and make the difference between the spatial correlation matrix estimate and the spatial correlation matrix less than the preset target, so as to obtain the optimal spatial correlation matrix estimate. The separation module is used to separate the sound signal based on the optimal spatial correlation matrix.

Images