Sound source separating device

By generating the objective function and maximizing the log-likelihood function, using the estimation source vector and the noise vector estimation hierarchical matrix, the problem of multi-sound source signal separation is solved and the sound recognition performance is improved.

CN118942479APending Publication Date: 2024-11-12MPWAV INC +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311409546.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-04-28
Filing Date
2023-10-27
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The prior art is difficult to effectively separate the sound signals transmitted by multiple sound sources, especially when noise exists, which affects the sound recognition performance.

Method used

The objective function is generated by the estimated source vector and the estimated noise vector estimated based on the input signals of multiple microphones, and the first and second terms in the objective function are replaced by the log-likelihood function, and the third term is a non-negative limit condition. The log-likelihood function is maximized to estimate the hierarchical matrix, thereby achieving accurate separation of sound signals.

Benefits of technology

Accurate separation of multiple sound source signals is achieved, sound recognition performance is improved, and target sound can be effectively extracted in the presence of noise.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942479A_ABST
    Figure CN118942479A_ABST
Patent Text Reader

Abstract

A sound source separation device according to an embodiment of the present invention may include a plurality of microphones, a matrix unit, and an output unit. A plurality of microphones may receive a plurality of microphone input signals transmitted from a plurality of sound sources. The matrix section may generate an objective function from a derived source vector and an estimated noise vector estimated based on a plurality of microphone input signals, and may estimate a hierarchical matrix by replacing a first term and a second term included in the objective function with a log-likelihood function. The output may provide an output vector calculated based on the microphone input signal and the hierarchical matrix. The sound source separation device according to the present invention generates an objective function from an estimated source vector and an estimated noise vector estimated on the basis of a plurality of microphone input signals, and estimates a hierarchical matrix by replacing a first term and a second term included in the objective function with a log-likelihood function. Sound signals respectively transmitted from a plurality of sound sources can be further accurately separated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a sound source separation device. Background Art

[0002] The sound input signal input through the microphone includes not only the target sound required to be recognized, but also noise that hinders the sound recognition. Therefore, various studies are currently being conducted to improve the sound recognition performance by removing noise from the sound input signal and extracting only the required target sound.

[0003] Prior art literature

[0004] Patent Literature

[0005] Korean Patent No. 10-1133308 (grant date: March 28, 2012) Summary of the invention

[0006] Technical issues

[0007] The technical purpose to be achieved by the present invention is to provide a sound source separation device, that is, to generate an objective function by estimating a source vector and an estimated noise vector based on multiple microphone input signals, to replace the first term and the second term included in the objective function with a log-likelihood function, and to estimate a hierarchical matrix by maximizing the log-likelihood function under the constraint that the third term included in the objective function is a non-negative number, thereby further accurately separating the sound signals transmitted from multiple sound sources.

[0008] Technical Solution

[0009] In order to achieve the above-mentioned purpose, the sound source separation device of the embodiment of the present invention may include multiple microphones, a matrix unit and an output unit. The multiple microphones may receive multiple microphone input signals transmitted from multiple sound sources. The matrix unit may generate an objective function by inferring a source vector and an estimated noise vector estimated based on the multiple microphone input signals, and may use a log-likelihood function to replace the first term and the second term included in the objective function to estimate a hierarchical matrix. The output unit may provide an output vector calculated based on the microphone input signal and the hierarchical matrix.

[0010] In an embodiment of the present invention, the third term included in the objective function may be greater than or equal to zero.

[0011] In one embodiment of the present invention, under the restriction that the third term is a non-negative number, the maximization of the log-likelihood function can be achieved by maximizing the Lagrangian function.

[0012] In one embodiment of the present invention, the Lagrangian function can be separated into Lagrangian functions of each frequency, and the Lagrangian function can be maximized by independently maximizing the Lagrangian functions of each frequency for all frequencies.

[0013] In one embodiment of the present invention, the variance of the estimated source vector may be calculated by performing partial differentiation of the variance of the estimated source vector by the Lagrangian function.

[0014] In one embodiment of the present invention, the sound source separation device may further include a first variance estimator. The first variance estimator may estimate the variance of the estimated source vector based on the microphone input signal.

[0015] In one embodiment of the present invention, the sound source separation device may further include a first mask unit. The first mask unit may provide a variance of the estimated source vector using a first mask applied to the microphone input signal.

[0016] In one embodiment of the present invention, the variance of the estimated noise vector may be calculated by performing partial differentiation of the variance of the estimated noise vector using the Lagrangian function.

[0017] In an embodiment of the present invention, the variance of the estimated noise vector may be a constant greater than zero.

[0018] In one embodiment of the present invention, the sound source separation device may further include a second variance estimator. The second variance estimator may estimate the variance of the estimated noise vector based on the microphone input signal.

[0019] In one embodiment of the present invention, the sound source separation device may further include a second mask unit. The second mask unit may provide a variance of the estimated source vector using a second mask applied to the microphone input signal.

[0020] In one embodiment of the present invention, the sound source separation device may further include a matrix calculation unit. The matrix calculation unit may calculate the derived source layered matrix and the estimated noise layered matrix included in the layered matrix based on the estimated source vector and the estimated noise vector.

[0021] In one embodiment of the present invention, the estimated source layer matrix may be calculated by sequentially calculating estimated source layer vectors.

[0022] In one embodiment of the present invention, the estimated noise layer matrix may be calculated by sequentially calculating estimated noise layer vectors.

[0023] In one embodiment of the present invention, the estimated noise layer matrix may be calculated by the estimated source layer matrix.

[0024] In one embodiment of the present invention, the determinant of the layered matrix updated by the estimated source layered vector may be the inverse of the conjugate determinant of the layered matrix before the estimated source layered vector is updated.

[0025] In one embodiment of the present invention, the determinant of the layered matrix updated by the estimated noise layered vector may be the reciprocal of the conjugate determinant of the layered matrix before the estimated noise layered vector is updated.

[0026] In one embodiment of the present invention, the layered matrix is ​​initialized by an identity matrix. After initialization, the determinant of the layered matrix can always be maintained as 1.

[0027] In one embodiment of the present invention, the estimated source layer vector may be calculated based on the estimated source inverse spatial covariance matrix and the layer inverse matrix.

[0028] In one embodiment of the present invention, the estimated noise layer vector may be calculated based on the estimated noise spatial inverse covariance matrix and the layer inverse matrix.

[0029] In one embodiment of the present invention, the estimated source spatial inverse covariance matrix may be recursively calculated using the variance of the estimated source vector and the spatial inverse covariance matrix of the estimated source at a previous time.

[0030] In one embodiment of the present invention, the initialization of the estimated source space inverse covariance matrix may be implemented using an identity matrix.

[0031] In one embodiment of the present invention, the estimated noise spatial covariance inverse matrix can be recursively calculated using the variance of the estimated noise vector and the estimated noise spatial covariance inverse matrix at a previous time.

[0032] In one embodiment of the present invention, the initialization of the estimated noise spatial covariance inverse matrix may be implemented using an identity matrix.

[0033] In one embodiment of the present invention, the layered inverse matrix may be calculated using the estimated source layered vector.

[0034] In one embodiment of the present invention, the layered inverse matrix may be calculated using the estimated noise layered vector.

[0035] In addition to the technical solutions of the present invention as described above, ordinary technicians in the technical field to which the present invention belongs can clearly understand other features and advantages of the present invention through the following descriptions.

[0036] Effects of the Invention

[0037] The present invention as described above has the following effects.

[0038] The sound source separation device of the present invention can generate an objective function by estimating a source vector and an estimated noise vector based on multiple microphone input signals, and use a log-likelihood function to replace the first term and the second term included in the above objective function to estimate a hierarchical matrix, thereby further accurately separating the sound signals transmitted from multiple sound sources.

[0039] In addition, other features and advantages of the present invention can also be clearly understood through the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A diagram for explaining a sound source separation device according to an embodiment of the present invention.

[0041] Figure 2 A diagram showing a sound source separation device according to an embodiment of the present invention.

[0042] Figure 3 For illustration Figure 2 A diagram of a working example of a sound source separation device.

[0043] Figure 4 For illustration Figure 2 A diagram of another working example of the sound source separation device.

[0044] Figure 5 for Figure 2 An illustrative diagram of a matrix portion included in a sound source separation device. DETAILED DESCRIPTION

[0045] In this specification, it should be noted that in the process of assigning reference numerals to components of each drawing, even if the same components are shown in different drawings, the same reference numerals are assigned as much as possible.

[0046] On the other hand, the terms in this specification should be understood according to the following meanings.

[0047] Unless clearly defined in the context, an expression in the singular includes an expression in the plural, and such terminology does not limit the scope of protection of the invention.

[0048] The terms “include” or “have” etc. should be understood as not excluding the existence or additional possibility of one or more other features, numbers, steps, tasks, structural elements, components or their combinations.

[0049] In order to solve the above-mentioned problems, preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] Figure 1 1 is a diagram for explaining a sound source separation device according to an embodiment of the present invention. Figure 2 A diagram showing a sound source separation device according to an embodiment of the present invention.

[0051] Reference Figure 1 to Figure 2 , the sound source separation device 10 of the embodiment of the present invention may include a plurality of microphones 100, a matrix unit 200 and an output unit 300. The plurality of microphones may receive a plurality of microphone input signals X transmitted from a plurality of sound sources S. For example, the plurality of sound sources S may include a first sound source S1 to a Kth sound source SK, and the plurality of microphones 100 may include a first microphone MC1 to an Mth microphone MCM. Wherein, M and K may be natural numbers, and K may be less than or equal to M. The sound signals generated from the first sound source S1 to the Kth sound source SK may be transmitted to the first microphone MC1 to the Mth microphone MCM through the space between the first sound source S1 to the Kth sound source SK and the first microphone MC1 to the Mth microphone MCM. The transfer function corresponding to the space between the first sound source S1 to the Kth sound source SK and the first microphone MC1 to the Mth microphone MCM may be represented by A. Wherein, A may be a mixing matrix MM. In addition, the sound source separation device 10 of the present invention may also be applied to the case where the number of sound sources is K and the number of microphones is M.

[0052] The matrix unit 200 may generate an objective function by using an estimated source vector and an estimated noise vector estimated based on a plurality of microphone input signals X. For example, the estimated source vector and the estimated noise vector may be calculated by the following equations 1 to 6.

[0053] Mathematical formula 1

[0054]

[0055] Among them, x t,f can be the microphone input signal, t is time, f is frequency, A t,f is the mixing matrix, s t,f is the source vector, n t,f is the noise vector.

[0056] Mathematical formula 2

[0057]

[0058] Among them, y t,f can be the estimated source vector, z t,f is the estimated noise vector, W t is a hierarchical matrix, x t,f is the microphone input signal, t is time, and f is frequency.

[0059] Mathematical formula 3

[0060]

[0061] Among them, y t,f can be the inferred source vector, is the estimated source vector layer matrix, x t,f is the microphone input signal, t is time, and f is frequency.

[0062] Mathematical formula 4

[0063]

[0064] Among them, w 1,t,f ,…,w K,t,f is the first estimated source layer vector to the Kth estimated source layer vector, and H is the Hermitian transpose.

[0065] Mathematical formula 5

[0066]

[0067] Among them, z t,f can be the estimated noise vector, is the estimated noise vector layer matrix, x t,f is the microphone input signal, t is time, and f is frequency.

[0068] Mathematical formula 6

[0069]

[0070] Among them, w K+1,t,f ,…,w M,t,f are the K+1th estimated noise layer vector to the Mth estimated noise layer vector, and H is the Hermitian transpose.

[0071] In this case, the objective function may be represented by the mutual information amount between the output vectors for all frequencies. The multiple sound sources S may be separated independently by querying the hierarchical matrix that minimizes the objective function. For example, the objective function may be represented by the following mathematical formula 7.

[0072] Mathematical formula 7

[0073]

[0074] Among them, J(W t ) can be the objective function, is the mutual information, is entropy, is the Kth estimated source vector of the overall frequency in time t, is the mth estimated noise vector of the overall frequency in time t, G(r)=-logq(r) is the contrast function, q(r) is the probability density function of the assumed model of the random variable r, E[·] is the expected value, and C is a constant.

[0075] The matrix unit 200 may estimate the hierarchical matrix DDM by replacing the first term and the second term included in the objective function with a log-likelihood function. For example, the log-likelihood function used in the present invention may be represented by the following mathematical formula 8.

[0076] Mathematical formula 8

[0077]

[0078] in, can be the log-likelihood function, λ k,t is the variance of the kth estimated source vector at time t, is the estimated source vector, Z m,t,f is the mth estimated noise constituting the estimated noise vector, σ m,t,f is the variance of the mth estimated noise vector at time t and frequency f.

[0079] Furthermore, the probability density function of the estimated source vector may be a spherical Gaussian distribution with a time-varying variance, as shown in the following mathematical formula 9.

[0080] Mathematical formula 9

[0081]

[0082] The probability density function of the estimated noise may be a Gaussian distribution with a time-varying variance, as shown in the following mathematical formula 10.

[0083] Mathematical formula 10

[0084]

[0085] When it is assumed that the probability density function of the assumed model of the first and second terms included in the objective function follows the probability density functions of Mathematical Formulas 9 and 10, the relationship between the first and second terms included in the objective function and the log-likelihood function can be expressed by the following Mathematical Formula 11. When the log-likelihood function is used to replace the first and second terms included in the objective function, the log-likelihood function needs to be maximized in order to minimize the objective function.

[0086] Mathematical formula 11

[0087]

[0088] in, For the first item, For the second item, is the log-likelihood function, and T can be a constant.

[0089] In one embodiment of the present invention, the sum of the first term and the second term included in the objective function is the product of (1 / T) and the log-likelihood function, and T may be a constant greater than 0.

[0090] In one embodiment of the present invention, the third term included in the objective function may be greater than or equal to 0. For example, the third term included in the objective function may be represented by the following mathematical formula 12. When maximizing the log-likelihood function in order to minimize the objective function, the third term may be set as a non-negative number to control the mutual information amount to further effectively minimize it.

[0091] Mathematical formula 12

[0092]

[0093] In one embodiment of the present invention, under the constraint that the third term is a non-negative number, the maximization of the log-likelihood function can be achieved by maximizing the Lagrangian function. The Lagrangian function can be applied with the recursive least squares method, as shown in the following mathematical formula 13.

[0094] Mathematical formula 13

[0095]

[0096] Among them, J t can be the Lagrangian function at time t, β t,f is the Lagrange multiplier in time t and frequency f, γ is the forgetting factor, and under the restriction that the third term is non-negative, the effective minimization of mutual information can be expressed as maximizing the Lagrangian function J at each t in time t. t .

[0097] In one embodiment of the present invention, the Lagrangian function can be separated into Lagrangian functions of each frequency, and the Lagrangian function can be maximized by maximizing the Lagrangian functions of each frequency independently for all frequencies. The Lagrangian function in time t and frequency f can be expressed by the following mathematical formula 14.

[0098] Mathematical formula 14

[0099]

[0100] Among them, J t,f can be a Lagrangian function in time t and frequency f, w k,t,f is the k-th estimated source layer vector, w m,t,f is the mth estimated noise layer vector, R k,t,f is the k-th estimated source space covariance matrix, Q m,t,f is the m-th estimated noise spatial covariance matrix. The k-th estimated source spatial covariance matrix and the m-th estimated noise spatial covariance matrix can be represented by the following equations 15 and 16, respectively.

[0101] Mathematical formula 15

[0102]

[0103] Mathematical formula 16

[0104]

[0105] Figure 3 For illustration Figure 2 A diagram of a working example of a sound source separation device.

[0106] Reference Figures 1 to 3 In one embodiment of the present invention, the maximization condition of the Lagrangian function may depend on the variance of the estimated source vector and the variance of the estimated noise vector. For example, the variance of the estimated source vector may be calculated by performing partial differentiation of the variance of the estimated source vector by the Lagrangian function. In this case, the variance of the estimated source vector may be expressed by the following mathematical formula 17.

[0107] Mathematical formula 17

[0108]

[0109] in, The kth estimated source vector for the overall frequency in time t, F is the number of frequencies.

[0110] In one embodiment of the present invention, the sound source separation device 10 may further include a first variance estimator 410. The first variance estimator 410 may estimate the variance SV of the estimated source vector based on the microphone input signal X. For example, the first variance estimator 410 may deeply learn the variance of the input signal and the estimated source vector corresponding to the input signal, and the learned first variance estimator 410 may provide the variance SV of the estimated source vector corresponding to the microphone input signal.

[0111] In an embodiment of the present invention, the sound source separation device 10 may further include a first masking unit 420. The first masking unit 420 may provide a variance of an estimated source vector using the first microphone MS1 applied to the microphone input signal X. The variance of the estimated source vector may be represented by the following mathematical formula 18.

[0112] Mathematical formula 18

[0113]

[0114] in, can be the first mask, λ k,t,f is the variance of the kth estimated source vector at time t and frequency f.

[0115] Figure 4 For illustration Figure 2 A diagram of another working example of the sound source separation device.

[0116] Reference Figures 1 to 4 In one embodiment of the present invention, the variance of the estimated noise vector can be calculated by performing partial differentiation of the variance of the estimated noise vector by the Lagrangian function. In this case, the variance of the estimated noise vector can be expressed by the following mathematical formula 19.

[0117] Mathematical formula 19

[0118]

[0119] Among them, Z m,t,f can be the mth estimated noise constituting the estimated noise vector, σ m,t,f is the variance of the mth estimated noise vector.

[0120] In an embodiment of the present invention, the variance of the estimated noise vector may be a constant greater than 0. For example, the variance of the estimated noise vector may be 1.

[0121] In one embodiment of the present invention, the sound source separation device 10 may further include a second variance estimator 510. The second variance estimator 510 may estimate the variance NV of the estimated noise vector based on the microphone input signal X. For example, the second variance estimator 510 may deeply learn the variance of the input signal and the estimated noise vector corresponding to the input signal, and the learned second variance estimator 510 may provide the variance NV of the estimated noise vector corresponding to the microphone input signal.

[0122] In an embodiment of the present invention, the sound source separation device 10 may further include a second mask unit 520. The second mask unit 520 may use a second mask MS2 applied to the microphone input signal X to provide a variance of the estimated source vector.

[0123] Here, the variance of the estimated noise vector can be expressed by the following mathematical formula 20.

[0124] Mathematical formula 20

[0125]

[0126] Figure 5 for Figure 2 An illustrative diagram of a matrix portion included in a sound source separation device.

[0127] Reference Figures 1 to 5 In one embodiment of the present invention, the sound source separation device 10 may further include a matrix calculation unit 210. The matrix calculation unit 210 may calculate the derived source layered matrix SDM and the estimated noise layered matrix NDM included in the layered matrix DDM based on the estimated source vector and the estimated noise vector.

[0128] In one embodiment of the present invention, the estimated source layer matrix SDM can be obtained by sequentially calculating the estimated source layer vectors. For example, the optimal estimated source layer vector can be calculated by performing partial differentiation of the estimated source layer vectors in time t and frequency f using the Lagrangian function, which can be expressed by the following mathematical formula 21.

[0129] Mathematical formula 21

[0130]

[0131]

[0132]

[0133] Among them, w k,t,f is the k-th estimated source layer vector, β t,f is the Lagrange multiplier, det W t,f is the determinant of the hierarchical matrix, e k is a unit vector whose kth element is 1 and the remaining elements are 0, R k,t,f is the k-th estimated source space covariance matrix, The inverse source spatial covariance matrix may be estimated for the kth source. For k having a value greater than 1 and less than K, the estimated source layer matrix SDM may be calculated by sequentially calculating the kth estimated source layer vectors.

[0134] In one embodiment of the present invention, the estimated noise layering matrix NDM can be calculated by sequentially calculating the estimated noise layering vectors. For example, the optimal estimated noise layering vector can be calculated by performing partial differentiation of the estimated noise layering vector in time t and frequency f using the Lagrangian function, which can be expressed by the following mathematical formula 22.

[0135] Mathematical formula 22

[0136]

[0137]

[0138]

[0139] Among them, w m,t,f is the mth estimated noise layer vector, Q m,t,f is the mth estimated noise spatial covariance matrix, is the inverse covariance matrix of the mth estimated noise space, β t,f is the Lagrange multiplier, det w t,f is the determinant of the hierarchical matrix, e m It may be a unit vector whose m-th element is 1 and whose remaining elements are 0. For k having a value greater than or equal to K+1 and less than or equal to M, the m-th estimated source layer vector may be sequentially calculated to calculate the estimated noise layer matrix NDM.

[0140] In one embodiment of the present invention, the estimated noise layer matrix NDM can be calculated from the estimated source layer matrix SDM. For example, when it is assumed that the estimated source vector and the estimated noise vector are uncorrelated at the statistical level, the estimated noise layer matrix can be expressed by the following mathematical formula 23.

[0141] Mathematical formula 23

[0142] The estimated noise layered partial matrix in time t and frequency f, I M-K It can be an identity matrix of size (MK)×(MK). Since the estimated source vector and the estimated noise vector are not correlated at the statistical level, the correlation function between the estimated source vector and the estimated noise vector is 0, which can be expressed by the following mathematical formula (24).

[0143] Mathematical formula 24

[0144]

[0145] Among them, E[·] is the expected value, It can be the estimated source layer matrix in time t and frequency f. Based on mathematical formula 24, the estimated noise layer part matrix can be expressed by the following mathematical formula 25.

[0146] Mathematical formula 25

[0147]

[0148] in, The spatial covariance matrix of the microphone input signal in time t and frequency f can be expressed by the following mathematical formula 26 by applying the recursive least square method.

[0149] Mathematical formula 26

[0150]

[0151] The estimated noise stratification partial matrix can be calculated by using Equations 25 and 26, thereby the estimated noise stratification matrix of Equation 23 can be calculated.

[0152] In one embodiment of the present invention, the determinant of the layer matrix of the estimated source layer vector update may be the inverse of the conjugate determinant of the layer matrix before the estimated source layer vector update. For example, the kth estimated source layer vector may be represented by the following mathematical formula 27.

[0153] Mathematical formula 27

[0154]

[0155] Among them, b k,t,f Can be a hierarchical matrix W t,f The k-th column vector in the adjugate matrix of . The determinant of the hierarchical matrix for updating the k-th estimated source hierarchical vector based on equation 27 can be expressed by equation 28 below.

[0156] Mathematical formula 28

[0157]

[0158] in, The updated layer matrix for the k-th imputed source layer vector, is the conjugate determinant of the layer matrix before the k-th estimated source layer vector update, The k-th column vector included in the adjugate matrix of the layered matrix updated for the k-th estimated source layered vector is independent of the k-th estimated source layered vector corresponding to the k-th row vector of the layered matrix, and therefore,

[0159] In one embodiment of the present invention, the determinant of the layered matrix of the estimated noise layered vector update may be the inverse of the conjugate determinant of the layered matrix before the estimated noise layered vector update. For example, the mth estimated noise layered vector may be represented by the following mathematical formula 29.

[0160] Mathematical formula 29

[0161]

[0162] Among them, b m,t,f is the hierarchical matrix W t,f The determinant of the hierarchical matrix for updating the mth estimated noise hierarchical vector based on equation 29 can be expressed by equation 30 below.

[0163] Mathematical formula 30

[0164]

[0165] in, is the updated hierarchical matrix of the mth estimated noise hierarchical vector, is the conjugate determinant of the layer matrix before the m-th estimated noise layer vector update, The mth column vector included in the adjugate matrix of the layered matrix updated for the mth estimated noise layered vector is independent of the mth estimated noise layered vector corresponding to the mth row vector of the layered matrix, and therefore,

[0166] In one embodiment of the present invention, the hierarchical matrix is ​​initialized by an identity matrix, and after initialization, the determinant of the hierarchical matrix can always be maintained at 1. In order to make the estimated source hierarchical vector calculated based on mathematical formula 21 and the estimated noise hierarchical vector of mathematical formula 22 meet the optimal solution, the Karush-Kuhn-Tucker condition (KKT) must be satisfied, and the Karush-Kuhn-Tucker condition can be expressed by the following mathematical formula 31.

[0167] Mathematical formula 31

[0168]

[0169] β t,f ≥0 (2)

[0170] log|det w t,f |≥0 (3)

[0171] β t,f log|det w t,f |≥0 (4)

[0172] The Carlo-Kuhn-Tucker condition (1) can be satisfied by equations 21 and 22. In order to satisfy the Carlo-Kuhn-Tucker condition (2), the k-th estimated source space covariance matrix R k,t,f and the mth estimated noise spatial covariance matrix Q m,t,f Assuming that it is a positive definite matrix, if the time t in equations 15 and 16 is greater than the number of microphones M, the assumption of a positive definite matrix can be satisfied. In order to satisfy the Carlo-Kuhn-Tucker condition (3) and the Carlo-Kuhn-Tucker condition (4), it can be assumed that the hierarchical matrix W at time 0 and frequency f is 0,f Initialized by the identity matrix, and based on Equation 27 and Equation 29, the layered matrix determinant det W t,f Can always be maintained at 1.

[0173] In one embodiment of the present invention, the estimated source layer vector may be calculated based on the estimated source inverse spatial covariance matrix and the layer inverse matrix. For example, the update of the estimated source layer vector may be represented by the following mathematical formula 32.

[0174] Mathematical formula 32

[0175]

[0176] Among them, w k,t,f is the k-th imputed source layer vector, is the inverse covariance matrix of the k-th estimated source space, is the layered inverse matrix, e k It may be a unit vector whose k-th element is 1 and whose remaining elements are 0. The k-th estimated source layer vector may be sequentially updated with respect to k having a value greater than 1 and less than K, thereby updating the estimated source layer matrix.

[0177] In one embodiment of the present invention, the estimated noise layering vector may be calculated based on the estimated noise spatial covariance inverse matrix and the layering inverse matrix. For example, the update of the estimated noise layering vector may be expressed by the following mathematical formula 33.

[0178] Mathematical formula 33

[0179]

[0180] Among them, w m,t,f is the mth estimated noise layer vector, is the inverse covariance matrix of the mth estimated noise space, is a hierarchical inverse matrix, and em may be a unit vector whose mth element is 1 and the remaining elements are 0. Now, for m having a value greater than K+1 and less than M, the mth estimated noise hierarchical vector may be updated sequentially, thereby updating the estimated noise hierarchical matrix.

[0181] In one embodiment of the present invention, the estimated source spatial covariance inverse matrix can be recursively calculated using the variance of the estimated source vector and the estimated source spatial covariance inverse matrix at the previous time. For example, the estimated source spatial covariance inverse matrix can be expressed by the following mathematical formula 34 using the matrix inversion lemma.

[0182] Mathematical formula 34

[0183]

[0184] in, is the inverse spatial covariance matrix of the kth estimated source in time t and frequency f, λ k,t is the variance of the kth estimated source vector at time t, The inverse source spatial covariance matrix may be estimated for the kth in time t-1 and frequency f.

[0185] In one embodiment of the present invention, the initialization of the estimated source space covariance inverse matrix can be implemented using an identity matrix. For example, the initialization of the estimated source space covariance can be expressed by the following mathematical formula 35.

[0186] Mathematical formula 35

[0187]

[0188] I M is the identity matrix of size M×M, ρ (s) Can be a constant greater than 0. For example, ρ (s) Can be 1 or 1e-6.

[0189] In one embodiment of the present invention, the estimated noise spatial covariance inverse matrix can be recursively calculated using the variance of the estimated noise vector and the previous estimated noise spatial covariance inverse matrix. For example, the estimated noise spatial covariance inverse matrix can be expressed by the following mathematical formula 36 using the matrix inversion lemma.

[0190] Mathematical formula 36

[0191]

[0192] in, is the inverse spatial covariance matrix of the mth estimated noise in time t and frequency f, σ m,t,f is the variance of the mth estimated noise vector at time t and frequency f, The inverse noise spatial covariance matrix may be estimated for the mth in time t-1 and frequency f.

[0193] In one embodiment of the present invention, the initialization of the estimated noise spatial covariance inverse matrix can be implemented using an identity matrix. For example, the initialization of the estimated noise spatial covariance can be expressed by the following mathematical formula 37.

[0194] Mathematical formula 37

[0195] Matrix, I M is the identity matrix of size M×M, ρ (n) Can be a constant greater than 0. For example, ρ (n) Can be 1 or 1e-6.

[0197] In one embodiment of the present invention, the layered inverse matrix can be calculated using the estimated source layered vector. For example, the layered inverse matrix can be expressed by the following mathematical formula 38.

[0198] Mathematical formula 38

[0199]

[0200] in, is the hierarchical inverse matrix in time t and frequency f, The change in the k-th estimated source layer vector in time t and frequency f can be

[0201] In one embodiment of the present invention, the layered inverse matrix can be calculated using the estimated noise layered vector. For example, the layered inverse matrix can be expressed by the following mathematical formula 39.

[0202] Mathematical formula 39

[0203]

[0204] in, is the hierarchical inverse matrix in time t and frequency f, The change in the k-th estimated noise layer vector in time t and frequency f can be

[0205] The output unit 300 may provide an output vector Y calculated based on the microphone input signal X and the hierarchical matrix DDM. For example, the plurality of output vectors Y may include a first output vector Y1 to an Mth output vector YM. The first output vector Y1 to the Mth output vector YM may include signals separated according to respective sound sources. The sound source separation device 10 of the present invention may generate an objective function by using an estimated source vector and an estimated noise vector estimated based on a plurality of microphone input signals, and may use a log-likelihood function to replace the first term and the second term included in the objective function, and maximize the log-likelihood function under the constraint that the third term is a non-negative number to estimate the hierarchical matrix DDM, thereby further accurately separating the sound signals transmitted from the plurality of sound sources S, respectively.

[0206] In addition to the technical features of the present invention as described above, a person skilled in the art can clearly understand other features and advantages of the present invention through the above description.

Claims

1. A sound source separation device, characterized in that: include: a plurality of microphones for receiving a plurality of microphone input signals transmitted from a plurality of sound sources; A matrix unit generates an objective function by using an estimated source vector and an estimated noise vector estimated based on the plurality of microphone input signals, and estimates a hierarchical matrix by replacing a first term and a second term included in the objective function with a log-likelihood function; An output unit, used for providing an output vector calculated based on the microphone input signal and the layered matrix; as well as The matrix calculation unit calculates the derived source hierarchical matrix and the estimated noise hierarchical matrix included in the hierarchical matrix based on the estimated source vector and the estimated noise vector, respectively.

2. The sound source separation device according to claim 1, characterized in that: The above objective function includes a third term greater than zero.

3. The sound source separation device according to claim 2, characterized in that: Under the restriction that the third term is non-negative, the maximization of the log-likelihood function is achieved by maximizing the Lagrangian function.

4. The sound source separation device according to claim 3, characterized in that: The Lagrangian function can be separated into Lagrangian functions of each frequency, and the Lagrangian function can be maximized by maximizing the Lagrangian functions of each frequency independently for all frequencies.

5. The sound source separation device according to claim 4, characterized in that: The variance of the estimated source vector is calculated by partially differentiating the variance of the estimated source vector using the Lagrangian function.

6. The sound source separation device according to claim 5, characterized in that: The sound source separation device further includes a first variance estimator that estimates the variance of the estimated source vector based on the microphone input signal.

7. The sound source separation device according to claim 6, characterized in that: The sound source separation device further includes a first masking unit configured to provide a variance of the estimated source vector using a first mask applied to the microphone input signal.

8. The sound source separation device according to claim 7, characterized in that: The variance of the estimated noise vector is calculated by partially differentiating the variance of the estimated noise vector using the Lagrangian function.

9. The sound source separation device according to claim 8, characterized in that: The variance of the above-mentioned estimated noise vector is a constant greater than 0.

10. The sound source separation device according to claim 9, characterized in that: The sound source separation device further includes a second variance estimator configured to estimate a variance of the estimated noise vector based on the microphone input signal.

11. The sound source separation device according to claim 10, characterized in that: The sound source separation device further includes a second masking unit configured to provide a variance of the estimated source vector using a second mask applied to the microphone input signal.

12. The sound source separation device according to claim 11, characterized in that: The estimated source layer matrix is ​​composed of estimated source layer vectors, and the estimated source layer vectors can be calculated based on the estimated source space inverse covariance matrix and the layer inverse matrix.

13. The sound source separation device according to claim 12, characterized in that: The estimated source spatial inverse covariance matrix can be recursively calculated using the variance of the estimated source vector and the spatial inverse covariance matrix of the estimated source at the previous time.

14. The sound source separation device according to claim 13, characterized in that: The above-mentioned hierarchical inverse matrix can be calculated using the above-mentioned estimated source hierarchical vector.

15. The sound source separation device according to claim 11, characterized in that: The estimated noise layer matrix is ​​composed of estimated noise layer vectors. The estimated noise layer vectors can be calculated based on the estimated noise spatial inverse covariance matrix and the layer inverse matrix.

16. The sound source separation device according to claim 15, characterized in that: The estimated noise spatial covariance inverse matrix can be recursively calculated using the variance of the estimated noise vector and the estimated noise spatial covariance inverse matrix at a previous time.

17. The sound source separation device according to claim 16, characterized in that: The above-mentioned hierarchical inverse matrix can be calculated using the above-mentioned estimated noise hierarchical vector.

18. The sound source separation device according to claim 17, characterized in that: The estimated noise layering matrix can be calculated based on the estimated source layering matrix.

19. The sound source separation device according to claim 11, characterized in that: The above-mentioned layered matrix is ​​initialized by an identity matrix. After initialization, the determinant of the above-mentioned layered matrix can always be maintained as 1.