Multi-sound-source positioning method and system and electronic equipment

Abstract

The invention provides a multi-sound-source positioning method and system and electronic equipment, and relates to the technical field of sound source localization, and the multi-sound-source positioning method comprises the steps: obtaining time domain sound pressure collection data of a multi-channel microphone array; performing short-time Fourier transform on the time domain sound pressure acquisition data to obtain a frequency domain sound pressure signal model; performing spherical Fourier transform on the frequency domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model; processing the spherical harmonic domain sound pressure coefficient model to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain receiving signal; and performing singular value decomposition on the high-order cumulant matrix corresponding to the spherical harmonic domain receiving signal to obtain a noise subspace. According to the method, the system and the electronic equipment provided by the invention, the dimensionality of the steering vector matrix is expanded through the spherical harmonic domain MUSIC algorithm of the fourth-order cumulant, so that the effective rank of the steering vector matrix is improved, and the number of locatable target sound sources is increased.

Description

Technical Field

[0001] The present invention relates to the technical field of sound source localization, and in particular to a multi-sound source localization method, system and electronic equipment. Background Art

[0002] Sound source localization technology based on microphone sensor array signal processing is an important acoustic signal processing work. In recent years, it has been applied to the fields of spatial filtering, sound source separation, sound source tracking, dereverberation and speech enhancement. Among them, spherical microphone arrays are widely used in three-dimensional sound field environments such as three-dimensional sound field reconstruction, room acoustic analysis, and closed cabin sound source localization. Spherical microphone arrays can not only localize sound sources in the array element domain, but also in the modal domain. Spherical harmonic domain sound source localization technology uses spherical Fourier transform to convert the array element domain data obtained by the spherical array into the spherical harmonic domain (SHD), and then locate the sound source. Compared with the array element domain processing method of traditional microphone arrays, the spherical harmonic domain processing method can decouple the frequency and direction information of the sound source, making it easier to localize the sound source in the three-dimensional sound field. Therefore, most of the current spherical array sound source localization methods are processed through the spherical harmonic domain. Most spherical array modal beamforming needs to be converted to the frequency domain before being processed in the spherical harmonic domain. Yan et al. proposed a real-valued time-domain implementation method for broadband beamforming of spherical microphone arrays, and formulated the weight coefficients of the filter for solving multi-constraint problems to provide appropriate trade-offs between multiple conflicting performance indicators. Li et al. combined the traditional MUSIC algorithm with the spherical harmonic domain and proposed the spherical harmonic domain MUSIC algorithm (SH-MUSIC), successfully applying the subspace method to spherical arrays for sound source localization.

[0003] However, most commonly used spherical harmonic domain sound source localization algorithms are extended to spherical microphone arrays through array element domain microphone array localization algorithms. When these algorithms are used to localize sound sources in high-noise and high-reverberation environments such as indoors and closed cabins, their spatial resolution and sound source localization accuracy will be severely reduced, and they are not robust to spatial noise. Summary of the invention

[0004] In view of the shortcomings of the prior art described above, the object of the present invention is to provide a multi-sound source localization method, system and electronic device, which are used to solve the problems in the prior art that the conventional spherical harmonic domain MUSIC algorithm has degraded positioning performance in complex scenes with multiple sound sources in a closed room and is not robust to spatial noise.

[0005] To achieve the above-mentioned purpose and other related purposes, the present invention provides a multi-sound source localization method, comprising: acquiring time-domain sound pressure acquisition data of a multi-channel microphone array; performing short-time Fourier transform on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model; performing spherical Fourier transform on the frequency-domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model; processing the spherical harmonic domain sound pressure coefficient model to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal; performing singular value decomposition on the high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain a noise subspace; constructing a MUSIC space spectrum according to the orthogonality property relationship between the corresponding expanded steering vector of the high-order cumulant matrix corresponding to the spherical harmonic domain received signal and the noise subspace; performing grid scanning search on the MUSIC space spectrum to obtain the peak position in the MUSIC space spectrum as the sound source position.

[0006] In one embodiment of the present invention, time-domain sound pressure acquisition data of a multi-channel microphone array is obtained, including: obtaining time-domain sound pressure acquisition data of a plane wave on a spherical surface according to the spatial sampling distribution of corresponding microphone array elements in a multi-channel microphone spherical array and the wave arrival direction of a harmonic plane wave.

[0007] In one embodiment of the present invention, a short-time Fourier transform is performed on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model, including: obtaining a representation model of the time-domain sound pressure acquisition data in the frequency domain based on the time-domain sound pressure acquisition data; and converting the representation model into a frequency-domain sound pressure signal model based on the corresponding microphone array elements in a multi-channel microphone spherical array that carries its own noise.

[0008] In one embodiment of the present invention, a spherical Fourier transform is performed on the frequency domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model, including: obtaining a deformation matrix of the steering vector matrix based on the steering vector matrix in the frequency domain sound pressure signal model, the corresponding spherical modal intensity and the angular position of the microphone array on the spherical array; performing a spherical Fourier transform on the received sound pressure of the multi-channel microphone array to obtain a spherical harmonic domain receiving matrix; obtaining a spherical harmonic domain sound pressure coefficient model for sound source localization based on the spherical harmonic domain receiving matrix, the sampling weight conditions corresponding to the spherical harmonic domain receiving matrix and the deformation matrix.

[0009] In one embodiment of the present invention, the spherical harmonic domain sound pressure coefficient model is processed to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal, including: eliminating the influencing factors in the spherical harmonic domain sound pressure coefficient model by plane wave decomposition to obtain a spherical harmonic domain data model; wherein the influencing factors include the spherical array modal intensity; and performing steering vector matrix expansion on the spherical harmonic domain data model to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal.

[0010] In one embodiment of the present invention, the spherical harmonic domain data is expanded by a steering vector matrix to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal, including: according to the spherical harmonic domain data model, respectively obtaining a high-order cumulant matrix of the spherical harmonic domain signal source and a high-order cumulant matrix of the spherical harmonic domain noise; according to the high-order cumulant matrix of the spherical harmonic domain signal source and the high-order cumulant matrix of the spherical harmonic domain noise, the spherical harmonic domain data model is processed to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal.

[0011] In one embodiment of the present invention, the high-order cumulant matrix is ​​a fourth-order cumulant matrix.

[0012] In one embodiment of the present invention, a high-order cumulant matrix corresponding to a received signal in the spherical harmonic domain is subjected to singular value decomposition to obtain a noise subspace, including: performing eigenvalue decomposition on the high-order cumulant matrix corresponding to the received signal in the spherical harmonic domain to obtain eigenvalues ​​and corresponding eigenvectors; classifying the eigenvalues ​​to obtain large eigenvalues ​​and small eigenvalues; and using the space spanned by the eigenvectors corresponding to the small eigenvalues ​​as the noise subspace.

[0013] To achieve the above-mentioned purpose and other related purposes, the present invention also provides a multi-sound source localization system, comprising: an acquisition unit, used to acquire time-domain sound pressure acquisition data of a multi-channel microphone array; a short-time Fourier transform unit, used to perform short-time Fourier transform on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model; a spherical Fourier transform unit, used to perform spherical Fourier transform on the frequency-domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model; a processing unit, used to process the spherical harmonic domain sound pressure coefficient model to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal; a decomposition unit, used to perform singular value decomposition on the high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain a noise subspace; a spatial spectrum construction unit, used to construct a MUSIC spatial spectrum according to the orthogonality property relationship between the corresponding expanded steering vector of the high-order cumulant matrix corresponding to the spherical harmonic domain received signal and the noise subspace; and a scanning unit, used to perform grid scanning search on the MUSIC spatial spectrum to obtain the peak position in the MUSIC spatial spectrum as the sound source position.

[0014] To achieve the above-mentioned purpose and other related purposes, the present invention further provides an electronic device, which includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by one or more processors, the electronic device implements the aforementioned multi-sound source localization method.

[0015] As described above, a method, system and electronic device for localizing multiple sound sources of the present invention have the following beneficial effects: by using high-order cumulants to preprocess spherical harmonic domain signals, high-order cumulants can well suppress Gaussian measurement noise and improve the robustness of the algorithm to noise. At the same time, the spherical harmonic domain MUSIC algorithm of high-order cumulants proposed by the present invention also increases the effective rank of the steering vector matrix by expanding the dimension of the steering vector matrix, thereby increasing the number of target sound sources that can be located. In complex environments, this method effectively enhances the accuracy of sound source position estimation, overcomes the limitations of traditional algorithms when the number of sound sources increases, and improves the overall effect of multiple sound source localization. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a flow chart of a method for localizing multiple sound sources provided by an embodiment of the present invention.

[0017] Figure 2 FIG. 1 shows a diagram of the rank and effective rank of a steering vector matrix at different highest modal orders provided by an embodiment of the present invention.

[0018] Figure 3 A diagram showing the influence of the modal order and the sound source spacing angle on the effective rank provided by an embodiment of the present invention.

[0019] Figures 4 to 5 Shown are sound source localization diagrams using different algorithms when the number of sound sources provided by an embodiment of the present invention is less than the spherical harmonic coefficients.

[0020] Figures 6 to 9 Shown is a sound source localization diagram of different algorithms when the number of sound sources provided by an embodiment of the present invention is greater than or equal to the spherical harmonic coefficients.

[0021] Fig.10 Shown is a structural block diagram of a multi-sound source localization system provided by an embodiment of the present invention.

[0022] Fig.11 Shown is a structural schematic diagram of an electronic device according to an embodiment of the present invention.

[0023] Component number description

[0024] Electronic device 1; multiple sound source localization system 11; memory 12; processor 13; acquisition unit 111; short-time Fourier transform unit 112; spherical Fourier transform unit 113; processing unit 114; decomposition unit 115; spatial spectrum construction unit 116; scanning unit 117. DETAILED DESCRIPTION

[0025] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0026] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0027] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0028] See also Figure 1 The present invention provides a multi-sound source localization method, comprising:

[0029] Step S10: Acquire time-domain sound pressure data collected by the multi-channel microphone array;

[0030] Step S20: performing short-time Fourier transform on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model;

[0031] Step S30: performing spherical Fourier transform on the frequency domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model;

[0032] Step S40: Processing the spherical harmonic domain sound pressure coefficient model to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal;

[0033] Step S50: performing singular value decomposition on the high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain a noise subspace;

[0034] Step S60: constructing a MUSIC spatial spectrum according to the orthogonality property relationship between the corresponding expanded steering vector and the noise subspace of the high-order cumulant matrix corresponding to the spherical harmonic domain received signal;

[0035] Step S70: performing a grid scanning search on the MUSIC spatial spectrum to obtain a peak position in the MUSIC spatial spectrum as the sound source position.

[0036] Through the above steps, it is not difficult to find that in the process of using the spherical harmonic domain MUSIC algorithm for multi-sound source localization, the time domain sound pressure acquisition data of the multi-channel microphone array is first obtained, and then after the short-time Fourier transform, the spherical Fourier transform is performed to obtain the spherical harmonic domain sound pressure coefficient model. Then, the spherical harmonic domain sound pressure coefficient model is processed to decompose and eliminate the influence of the spherical array modal intensity on the spherical harmonic domain sound pressure coefficient model, and the high-order cumulant matrix corresponding to the spherical harmonic domain received signal is calculated by expanding the steering vector matrix. And then the high-order cumulant matrix corresponding to the spherical harmonic domain received signal is subjected to singular value decomposition to obtain the signal subspace and noise subspace. And the orthogonality property relationship between the extended steering vector and the noise subspace is used to construct the MUSIC spatial spectrum, and the position information corresponding to the peak value obtained by the scan is the sound source position by grid scanning and searching the MUSIC spatial spectrum. The steering vector matrix after adding the high-order cumulant as mentioned above has a higher effective rank, which can realize the sound source positioning of more target sound sources; effectively enhance the accuracy of sound source position estimation, overcome the limitations of traditional algorithms when the number of sound sources increases, and improve the overall effect of multi-sound source positioning.

[0037] Figure 1 A flowchart of a method for localizing multiple sound sources in an exemplary embodiment of the present application is shown, including steps S10 to S70. Figure 1 The technical solution of this application will be described in detail.

[0038] First, step S10 is executed to obtain time-domain sound pressure collection data of the multi-channel microphone array.

[0039] It is worth noting that after obtaining the time-domain sound pressure acquisition data of the multi-channel microphone array, the time-domain sound pressure acquisition data needs to be further preprocessed, and then the preprocessed time-domain sound pressure acquisition data needs to be short-time Fourier transformed.

[0040] In step S10, the time domain sound pressure acquisition data of the multi-channel microphone array is obtained, including:

[0041] According to the spatial sampling distribution of the corresponding microphone array elements in the multi-channel microphone spherical array and the wave arrival direction of the harmonic plane wave, the time domain sound pressure collection data of the plane wave on the spherical surface is obtained.

[0042] In this embodiment, it is assumed that there are S far-field sound sources in a three-dimensional free sound field, and the S far-field sound sources generate harmonic plane waves with a wave number of k, which are radiated onto a spherical array. Among them, the spherical microphone consists of I microphone elements distributed according to a certain spatial sampling. The position of the i-th microphone element on the spherical array is r i =(r i cosφ i sinθ i ,r i sinφ i sinθ i ,r i cosθ i ), T ,φ i is the azimuth angle, θ i is the elevation angle, and the direction of arrival of L harmonic plane waves is (θ l ,φ l )(l = 1,…,L). The sound pressure expression of the plane wave at the spherical surface r=(r,θ,φ) is expressed by the sum of spherical harmonic functions and spherical Bessel functions as follows:

[0043]

[0044] In actual situations, a maximum spherical harmonic order N is selected to approximate the infinite summation in Equation (1) by a finite summation, and the approximate representation of the time-domain sound pressure acquisition data is obtained as:

[0045]

[0046] where N is the maximum modal order, and the product of the beam k and the radius r satisfies kr < N, j m (kr) is the spherical Bessel function of the first kind, represents the spherical harmonic function of the n-th order and m-th degree, and its expression is as follows:

[0047]

[0048] In the formula represents the associated Legendre function; i represents the imaginary unit; e imφ represents the complex exponential function, which provides the periodicity of the azimuth angle φ; represents the normalization factor.

[0049] In this embodiment, selecting a finite summation of the maximum modal order to approximate the sound pressure also introduces a truncation error. It can be seen that when the product kr of the wave number k and the radius r satisfies kr < N, the sound field within the circle with a radius of r can be accurately reconstructed.

[0050] Next, step S20 is executed to perform a short-time Fourier transform on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model.

[0051] In step S20, a short-time Fourier transform is performed on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model, including:

[0052] Step S201: obtaining a representation model of the time-domain sound pressure collection data in the frequency domain according to the time-domain sound pressure collection data;

[0053] Step S202: converting the representation model into a frequency domain sound pressure signal model according to the corresponding microphone array element in the multi-channel microphone spherical array that carries its own noise.

[0054] In this embodiment, the representation model of the sound pressure of the i-th microphone array element on the spherical array when the spherical microphone array receives L far-field harmonic plane waves in three-dimensional space in the frequency domain is expressed as:

[0055]

[0056] Where S l (k)(l=1,…,L) represents the amplitude of L harmonic plane waves, a i (r a ,Ψ l ) represents the steering vector from the lth sound source signal to the i-th microphone array element in the spherical microphone array, r a is the radius of the spherical microphone array. The steering vector contains the relative position relationship between the incident angle of the sound source and the microphone array element, and also contains the spatial characteristics of the microphone array. Considering the noise carried by the microphone array element itself, the representation model of formula (4) can be further expressed in a compact form as a frequency domain sound pressure signal model:

[0057] P(k)=A(r a ,Ψ)S(k)+V(k) (5)

[0058] Among them, A(r a ,Ψ) is represented as an I×L dimensional steering vector matrix, S(k) is an L×N S dimensional signal data matrix, V(k) is I×N S dimensional uncorrelated sensor noise data matrix. Assume that the noise component is spatial white noise with zero mean and Gaussian distribution and is uncorrelated with the signal S(k). The steering vector matrix is ​​expressed as:

[0059] A(r a ,Ψ)=[a(r a ,Ψ 1 ) a(r a ,Ψ 2 ) … a(r a ,Ψ L )] (6)

[0060] Among them, a(r a ,Ψ l )=[a 1 (r a ,Ψ 1 ) 2 (r a ,Ψ 2 )…a I (r a ,Ψ L )],a(r a ,Ψ 1 ) represents the sound pressure generated by the lth plane sound source at the i-th microphone, which can be expressed as:

[0061]

[0062] Among them, Φ i represents the angular position of the microphone element on the spherical array, b n (k,r a ) is the n-order spherical modal intensity, and its expression is expressed as:

[0063]

[0064] Among them, j n (kr) and j′ n (kr a ) are the first kind of spherical Bessel function and its derivative, h n (kr) and h′ n (kr a ) are the second kind of spherical Bessel function and its derivative. The radius of the spherical array is r a , generally r = r a .

[0065] Substituting (7) into a(r a ,Ψ l ), the steering matrix is ​​rewritten as:

[0066] A(r a ,Ψ)=Y(Φ)[B(kr a )y H (Ψ 1 )…B(kr a )y H (Ψ L )] (9)

[0067] Where Y(Φ) is (N+1) 2 dimensional matrix, the i-th row vector matrix is

[0068] Next, step S30 is executed: performing spherical Fourier transform on the frequency domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model.

[0069] In step S30, a spherical Fourier transform is performed on the frequency domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model, including:

[0070] Step S301: obtaining a deformation matrix of the steering vector matrix according to the steering vector matrix in the frequency domain sound pressure signal model, the corresponding spherical modal intensity, and the angular position of the microphone array on the spherical array;

[0071] Step S302: performing spherical Fourier transform on the received sound pressure of the multi-channel microphone array to obtain a spherical harmonic domain receiving matrix;

[0072] Step S303: obtaining a spherical harmonic domain sound pressure coefficient model for sound source localization according to the spherical harmonic domain reception matrix, the sampling weight condition corresponding to the spherical harmonic domain reception matrix, and the deformation matrix.

[0073] In this embodiment, according to the frequency domain sound pressure signal model P(k)=A(r a ,Ψ)S(k)+V(k), and obtain the deformation matrix A(r a ,Ψ)=Y(Φ)[B(kr a )y H (Ψ 1 )…B(kr a )y H (Ψ L )]; At (r, Φ) = (r, θ, φ), the microphone receives the sound pressure and performs a spherical Fourier transform (SFT) (or spherical harmonic decomposition), and the spherical harmonic domain receiving matrix is ​​obtained as follows:

[0074]

[0075] In practice, the received sound pressure is not continuous. It is spatially sampled at the microphone position. Therefore, the spherical Fourier transform (SFT) of the sound pressure is approximated as:

[0076]

[0077] where n∈[0,N],m∈[-n,n],a i The weights for each microphone sample are written in matrix form as P nm (k,r)=Y H (Φ)ΓP(k,r,Φ) (12)

[0078] Where P nm =[P 00 ,P 1(-1) ,P 10 ,P11 ,…,P NN ] T , and Γ=diag(a 1 ,a 2 ,…,a I ) is an I×I dimensional sampling weight matrix, and the sampling weights satisfy Y H (Φ)ΓY(Φ)=I, where I represents the identity matrix;

[0079] Substituting equation (8) into equation (5), and combining equation (11) and the sampling weight condition, we get the spherical harmonic domain sound pressure coefficient model for sound source localization:

[0080] P nm =[B(kr 1 )y H (Ψ 1 )…B(kr L )y H (Ψ L )]S(k)+V nm (k) (13)

[0081] P nm =A nm (r,Ψ)S(k)+V nm (k) (14)

[0082] Among them, V nm (k) = Y H (Φ)ΓV(k),A nm (r,Ψ) is the spherical harmonic domain steering vector matrix, that is

[0083] A nm (r,Ψ)=[B(kr 1 )y H (Ψ 1 )…B(kr L )y H (Ψ L )] (15)

[0084] Next, step S40 is executed to process the spherical harmonic domain sound pressure coefficient model to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal.

[0085] In step S40, the spherical harmonic domain sound pressure coefficient model is processed to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal, including:

[0086] Step S401: Eliminate the influencing factors in the spherical harmonic domain sound pressure coefficient model by plane wave decomposition to obtain a spherical harmonic domain data model; wherein the influencing factors include the spherical array modal intensity;

[0087] Step S402: performing steering vector matrix expansion on the spherical harmonic domain data model to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal.

[0088] In this embodiment, the high-order cumulant matrix may be a fourth-order cumulant matrix. From the above-mentioned equations (14) and (15), the spherical harmonic domain data model obtained by further eliminating the influence of the spherical array modal intensity on the spherical harmonic domain sound pressure coefficient through plane wave decomposition is as follows:

[0089]

[0090] in, B(kr) is (N+1) 2 ×(N+1) 2 dimensional modal strength matrix;

[0091] Generally, the fourth-order cumulant of the received signal satisfies the following formula:

[0092]

[0093] Where cum(·) represents the cumulative amount, and * is the complex conjugate. Substituting the spherical harmonic domain data model (16) after eliminating the influence of the spherical array modal intensity into the fourth-order cumulative amount matrix of the above formula (17), it can be further expressed as:

[0094]

[0095]

[0096] In step S402, the spherical harmonic domain data is expanded by a steering vector matrix to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal, including:

[0097] Step S4021: obtaining a high-order cumulant matrix of a spherical harmonic domain signal source and a high-order cumulant matrix of a spherical harmonic domain noise respectively according to the spherical harmonic domain data model;

[0098] Step S4022: Processing the spherical harmonic data model according to the high-order cumulant matrix of the spherical harmonic signal source and the high-order cumulant matrix of the spherical harmonic noise to obtain a high-order cumulant matrix corresponding to the spherical harmonic received signal.

[0099] Preferably, the high-order cumulant matrix is ​​a fourth-order cumulant matrix.

[0100] In equation (18), Y is the spherical harmonic domain steering vector matrix after eliminating the influence of the spherical array modal intensity, S is the incident signal matrix, V is the noise matrix, is the Kronecker product, H is the Hermitian transpose, and * is the complex conjugate. To simplify the expression of (17), the following formula can be obtained according to the properties of the Kronecker product:

[0101]

[0102] Among them, C S is the fourth-order cumulant matrix of the spherical harmonic domain signal source, C V is the fourth-order cumulant matrix of spherical harmonic noise.

[0103] make Then we can get the fourth-order cumulant matrix R corresponding to the received signal in the spherical harmonic domain. nm4 as follows:

[0104]

[0105] Among them, A cum4 is the expanded steering vector, whose dimension is (N+1) 4 ×(N+1) 4 , while the dimension of the steering vector of the conventional spherical harmonic domain algorithm based on the cross power spectrum matrix is ​​(N+1) 2 ×(N+1) 2 , which greatly improves the dimension of the steering vector matrix, so theoretically more target sound sources can be processed.

[0106] Next, step S50 is executed to perform singular value decomposition on the high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain a noise subspace.

[0107] In step S50, singular value decomposition is performed on the high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain a noise subspace, including:

[0108] Step S501: performing eigenvalue decomposition on the high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain eigenvalues ​​and corresponding eigenvectors;

[0109] Step S502: classify the eigenvalues ​​to obtain large eigenvalues ​​and small eigenvalues;

[0110] Step S503: using the space spanned by eigenvectors corresponding to small eigenvalues ​​as the noise subspace.

[0111] In this embodiment, the high-order cumulant matrix may be a fourth-order cumulant matrix. nm4 Perform eigenvalue decomposition and arrange its eigenvalues ​​from large to small as The corresponding eigenvectors are Also because Then R nm4 YesL2 The larger eigenvalues ​​and I 2 -L 2 Smaller eigenvalues, where I represents the number of microphone array elements and L represents the number of sound sources. The space spanned by the eigenvectors corresponding to small eigenvalues ​​is called the noise subspace Ω n ,Right now

[0112]

[0113] At the same time, define the signal subspace Ω s for:

[0114]

[0115] Since the signal subspace and the noise subspace are orthogonal to each other, the orthogonality condition can be obtained as:

[0116]

[0117] The noise matrix is ​​defined as:

[0118]

[0119] Next, step S60 is performed: constructing a MUSIC spatial spectrum according to the orthogonality property relationship between the corresponding expanded steering vectors and the noise subspace of the high-order cumulant matrix corresponding to the spherical harmonic domain received signal.

[0120] In this embodiment, the high-order cumulant matrix may be a fourth-order cumulant matrix. cum4 and the noise subspace U n The orthogonality property relationship of , the MUSIC spatial spectrum output by the spherical harmonic domain MUSIC algorithm based on the fourth-order cumulant is obtained as follows:

[0121]

[0122] Next, step S70 is executed to perform a grid scan search on the MUSIC spatial spectrum to obtain the peak position in the MUSIC spatial spectrum as the sound source position. By adding the high-order cumulative amount, the steering vector matrix has a higher effective rank, and thus more target sound sources can be localized; the accuracy of the sound source position estimation is effectively enhanced, the limitations of the traditional algorithm when the number of sound sources increases, and the overall effect of multi-sound source localization is improved.

[0123] In this embodiment, the definition of the effective rank of a matrix is ​​expressed as follows:

[0124] For matrix A, whose matrix dimension is M×N, the singular value decomposition is obtained to obtain A=UDV. Both matrices U and V are unitary matrices with matrix dimensions of M×M and N×N respectively. D is a diagonal matrix composed of eigenvalues ​​after decomposition, whose matrix dimension is M×N. The relationship between the eigenvalues ​​is σ 1 ≥σ 2 ≥…≥σ Q ≥0, Q=min(M,N). Define the eigenvalue vector σ=(σ 1 ,σ 2 ,..,σ Q ) T , the arrangement of eigenvalues ​​can be expressed as follows:

[0125] p k =σ k / ||σ|| 1 ,k=1,2,…,Q (28)

[0126] in‖·‖ 1 Represented as l 1 norm, using the parameter p k Find the effective rank of the matrix:

[0127] erank(A)=exp{H(p 1 ,p 2 ,…,p Q )}(29)

[0128] in It is expressed as information entropy.

[0129] Generally speaking, the rank of a matrix is ​​a set of integers and must be discrete. However, for the effective rank, since the effective rank is a parameter based on the consistency of the matrix eigenvalues, the effective rank of the matrix is ​​not necessarily an integer. It can be a continuous value, so it can be used to analyze systems with continuously changing parameters.

[0130] See also Figure 2 and Figure 3 , Figure 2 and Figure 3 In one embodiment, Figure 2 The figure shows the changes in the rank and effective rank of the steering vector matrix before and after adding the fourth-order cumulant at different highest modal orders in the spherical harmonic domain. As can be seen from the figure, with the increase of the modal order N in the spherical harmonic domain, the steering vector matrix A before and after the improvement cum4 The rank and effective rank of the matrix are increasing. When the highest modal order N = 3, the improved steering vector matrix A cum4The rank and effective rank of the guiding vector matrix A have reached full rank, while the maximum modal order required for the rank and effective rank of the guiding vector matrix A to reach full rank is N = 5. And under the same modal order, the effective rank of the improved wire vector matrix is ​​higher, so the improved algorithm can process more targets under the same modal order. Figure 3 The influence of the sound source positions at different angle intervals under different modal orders on the effective rank is analyzed. It can be seen from the figure that under the same modal order and the same sound source angle interval, the effective rank of the improved steering vector matrix is ​​significantly greater than the effective rank of the conventional SH-MUSIC algorithm steering vector matrix before the improvement. The steering vector matrix of the SH-MUSIC algorithm is the same as the steering vector matrix of the SH-MUSIC algorithm, so the effective value of the improved steering vector matrix is ​​also greater than the effective rank of the steering vector matrix of the SH-MUSIC algorithm. Therefore, the spherical harmonic domain MUSIC algorithm based on the fourth-order cumulant has a higher effective rank than the steering vector matrix of the traditional spherical harmonic domain sound source localization algorithm based on the cross-power spectrum due to the expansion of the steering vector matrix, thereby improving the accuracy of sound source localization and enabling the array to detect more target sound sources.

[0131] See also Figures 4 to 9 , Figures 4 to 9 In one embodiment, Figure 4 This is the sound source localization diagram of the SH-MUSIC algorithm with 3 sources; Figure 5 This is the sound source localization diagram of the improved CUM4-SH-MUSIC algorithm with 3 sources; Figure 6 This is the sound source localization diagram of the SH-MVDR algorithm with 4 sources; Figure 7 This is the sound source localization diagram with the improved CUM4-SH-MUSIC algorithm and 4 sources; Figure 8 This is the sound source localization diagram of the SH-MVDR algorithm with 6 sources; Fig. 9 This is the sound source localization diagram of the improved CUM4-SH-MUSIC algorithm and the number of sources is 6. The positioning comparison analysis using the SH-MUSIC algorithm and the SH-MVDR algorithm and the improved fourth-order cumulant SH-MUSIC algorithm under different sound source numbers shows that the simulation is divided into two cases, namely when the number of sound sources is less than the spherical harmonic coefficient and when the number of sound sources is greater than or equal to the spherical harmonic coefficient.

[0132] In the simulation comparison when the number of sound sources is less than the spherical harmonic coefficient, the SH-MUSIC algorithm in the traditional spherical harmonic domain algorithm and the fourth-order cumulant SH-MUSIC algorithm are selected for comparative analysis. The sound source is set to 3 sound sources, and the incident angles of the 3 sound sources are Ψ 1 =(80°,135°),Ψ 2 =(125°,65°) and Ψ 3=(185°, 130°), a 50-element rigid spherical microphone array with a radius of 9.75 cm was used in the experiment. Figure 4 and Figure 5 The normalized sound source localization simulation diagrams of the SH-MUSIC algorithm and the fourth-order cumulant SH-MUSIC algorithm are shown respectively when the number of sound sources is less than the spherical harmonic coefficients. The actual sound source position in the figure is represented by a black circle "o", and the estimated sound source position is represented by a black plus sign "+".

[0133] When the number of sound sources is greater than or equal to the spherical harmonic coefficient, the SH-MUSIC algorithm cannot obtain the noise subspace and fails. Therefore, when the number of sound sources is greater than or equal to the spherical harmonic coefficient, the SH-MVDR algorithm in the traditional spherical harmonic domain algorithm and the fourth-order cumulant SH-MUSIC algorithm are selected for comparative analysis. The sound sources are set to 4 sound sources and 6 sound sources, respectively, and the incident angles of the 4 sound sources are Ψ 1 =(80°,135°),Ψ 2 =(125°,65°),Ψ 3 =(185°,130°) and Ψ 4 =(260°,65°), 6 sound sources add two incident angles of Ψ 5 =(10°,60°),Ψ 6 =(95°,10°). Figure 6 and Figure 8 The simulation diagram of normalized sound source localization of the SH-MVDR algorithm is shown when the number of sound sources is greater than or equal to the spherical harmonic coefficients; Figure 7 and Fig. 9 The normalized sound source localization simulation diagram of the fourth-order cumulant SH-MUSIC algorithm is shown when the number of sound sources is greater than or equal to the spherical harmonic coefficients.

[0134] pass Figures 4 to 9 It can be seen that when the number of sound sources is less than the spherical harmonic coefficients, the traditional spherical harmonic domain algorithm can identify and locate all sound source positions, but the spatial resolution is significantly worse than that of the CUM4-SH-MUSIC algorithm; when the number of sound sources is greater than the spherical harmonic coefficients, the traditional spherical harmonic domain algorithm based on the cross-power spectrum matrix can no longer accurately locate the sound sources and cannot detect all sound source targets, while the CUM4-SH-MUSIC algorithm can accurately detect all sound source targets and locate the sound sources more accurately. The above results show that the CUM4-SH-MUSIC algorithm can accurately identify and locate sound source targets regardless of whether the number of sound sources is less than the spherical harmonic coefficients or there are excess spherical harmonic coefficients. The CUM4-SH-MUSIC algorithm is superior to the traditional spherical harmonic domain algorithm based on the cross-power spectrum matrix in terms of positioning resolution and target recognition number.

[0135] Please refer to 10. The present invention also provides a multi-sound source localization system 11, including: an acquisition unit 111, used to acquire time-domain sound pressure acquisition data of a multi-channel microphone array; a short-time Fourier transform unit 112, used to perform short-time Fourier transform on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model; a spherical Fourier transform unit 113, used to perform spherical Fourier transform on the frequency-domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model; a processing unit 114, used to process the spherical harmonic domain sound pressure coefficient model to obtain a spherical harmonic domain received signal a decomposition unit 115 for performing singular value decomposition on the high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain a noise subspace; a spatial spectrum construction unit 116 for constructing a MUSIC spatial spectrum according to the orthogonality property relationship between the corresponding expanded steering vector of the high-order cumulant matrix corresponding to the spherical harmonic domain received signal and the noise subspace; and a scanning unit 117 for performing a grid scanning search on the MUSIC spatial spectrum to obtain a peak position in the MUSIC spatial spectrum as the sound source position.

[0136] It should be noted that the multi-sound source localization system 11 provided in the above embodiment and the multi-sound source localization method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the multi-sound source localization system 11 provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0137] See also Fig.11 The electronic device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a multiple sound source localization program.

[0138] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 12 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 1, such as codes for multi-sound source localization, etc., but can also be used to temporarily store data that has been output or is to be output.

[0139] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 1, and uses various interfaces and lines to connect various components of the entire electronic device 1, and executes or executes programs or modules (such as multi-sound source localization programs, etc.) stored in the memory 12, and calls data stored in the memory 12 to execute various functions of the electronic device 1 and process data.

[0140] The processor 13 executes the operating system and various installed application programs of the electronic device 1. The processor 13 executes the application programs to implement the steps in the above-mentioned multiple sound source localization method.

[0141] Exemplarily, the computer program may be divided into one or more modules, and the one or more modules are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into an acquisition unit 111, a short-time Fourier transform unit 112, a spherical Fourier transform unit 113, a processing unit 114, a decomposition unit 115, a spatial spectrum construction unit 116, and a scanning unit 117.

[0142] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium, and the computer-readable storage medium can be non-volatile or volatile. The above-mentioned software function module is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to perform part of the functions of the multiple sound source localization method described in each embodiment of the present application.

[0143] In summary, the present invention discloses a method, system and electronic device for localizing multiple sound sources. The spherical harmonic domain signals are preprocessed by using high-order cumulants. The high-order cumulants can well suppress Gaussian measurement noise and improve the robustness of the algorithm to noise. At the same time, the spherical harmonic domain MUSIC algorithm of the high-order cumulants proposed by the present invention also increases the effective rank of the steering vector matrix by expanding the dimension of the steering vector matrix, thereby increasing the number of target sound sources that can be located. In complex environments, this method effectively enhances the accuracy of sound source position estimation, overcomes the limitations of traditional algorithms when the number of sound sources increases, and improves the overall effect of multiple sound source localization. Therefore, the present invention effectively overcomes various shortcomings in the prior art and has a high industrial utilization value.

[0144] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for localizing multiple sound sources, characterized in that: include: Obtain time-domain sound pressure acquisition data of a multi-channel microphone array; Performing short-time Fourier transform on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model; Performing spherical Fourier transform on the frequency domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model; The spherical harmonic domain sound pressure coefficient model is processed to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal; Performing singular value decomposition on a high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain a noise subspace; Constructing a MUSIC spatial spectrum according to the orthogonality property relationship between the corresponding expanded steering vector of the high-order cumulant matrix corresponding to the spherical harmonic domain received signal and the noise subspace; A grid scanning search is performed on the MUSIC spatial spectrum to obtain a peak position in the MUSIC spatial spectrum as a sound source position.

2. The method for localizing multiple sound sources according to claim 1, wherein: Acquire time-domain sound pressure data from a multi-channel microphone array, including: The time-domain sound pressure acquisition data of the plane wave on the spherical surface is obtained according to the spatial sampling distribution of the corresponding microphone array elements in the multi-channel microphone spherical array and the wave arrival direction of the harmonic plane wave.

3. The multi-sound source localization method according to claim 1, characterized in that: Performing a short-time Fourier transform on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model, including: According to the time-domain sound pressure acquisition data, a representation model of the time-domain sound pressure acquisition data in the frequency domain is obtained; The representation model is converted into the frequency domain sound pressure signal model according to the corresponding microphone array element in the multi-channel microphone spherical array that carries noise itself.

4. The method for localizing multiple sound sources according to claim 1, wherein: Performing a spherical Fourier transform on the frequency domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model, including: Obtaining a deformation matrix of the steering vector matrix according to the steering vector matrix in the frequency domain sound pressure signal model, the corresponding spherical modal intensity, and the angular position of the microphone array on the spherical array; Perform spherical Fourier transform on the received sound pressure of the multi-channel microphone array to obtain the spherical harmonic domain receiving matrix; The spherical harmonic domain sound pressure coefficient model for sound source localization is obtained according to the spherical harmonic domain reception matrix, the sampling weight condition corresponding to the spherical harmonic domain reception matrix and the deformation matrix.

5. The method for localizing multiple sound sources according to claim 1, wherein: The spherical harmonic domain sound pressure coefficient model is processed to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal, including: Eliminating the influencing factors in the spherical harmonic domain sound pressure coefficient model by plane wave decomposition to obtain a spherical harmonic domain data model; wherein the influencing factors include the spherical array modal intensity; The spherical harmonic domain data model is expanded by a steering vector matrix to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal.

6. The method for localizing multiple sound sources according to claim 5, characterized in that: The spherical harmonic domain data is expanded by a steering vector matrix to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal, including: According to the spherical harmonic domain data model, a high-order cumulant matrix of a spherical harmonic domain signal source and a high-order cumulant matrix of a spherical harmonic domain noise are obtained respectively; The spherical harmonic domain data model is processed according to the high-order cumulant matrix of the spherical harmonic domain signal source and the high-order cumulant matrix of the spherical harmonic domain noise to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal.

7. The method for localizing multiple sound sources according to claim 1, characterized in that: The high-order cumulant matrix is ​​a fourth-order cumulant matrix.

8. The method for localizing multiple sound sources according to claim 1, characterized in that: The high-order cumulant matrix corresponding to the spherical harmonic domain received signal is subjected to singular value decomposition to obtain a noise subspace, including: Performing eigenvalue decomposition on the high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain eigenvalues ​​and corresponding eigenvectors; Classifying the eigenvalues ​​to obtain large eigenvalues ​​and small eigenvalues; The space spanned by the eigenvectors corresponding to the small eigenvalues ​​is used as the noise subspace.

9. A multi-sound source localization system, characterized in that: include: An acquisition unit, used to acquire time-domain sound pressure acquisition data of a multi-channel microphone array; A short-time Fourier transform unit, used for performing a short-time Fourier transform on the time-domain sound pressure acquisition data to obtain a frequency-domain sound pressure signal model; A spherical Fourier transform unit, used for performing a spherical Fourier transform on the frequency domain sound pressure signal model to obtain a spherical harmonic domain sound pressure coefficient model; A processing unit, used for processing the spherical harmonic domain sound pressure coefficient model to obtain a high-order cumulant matrix corresponding to the spherical harmonic domain received signal; A decomposition unit, used for performing singular value decomposition on a high-order cumulant matrix corresponding to the spherical harmonic domain received signal to obtain a noise subspace; A spatial spectrum construction unit, configured to construct a MUSIC spatial spectrum according to an orthogonality property relationship between a corresponding expanded steering vector of a high-order cumulant matrix corresponding to the spherical harmonic domain received signal and the noise subspace; as well as The scanning unit is used to perform a grid scanning search on the MUSIC spatial spectrum to obtain a peak position in the MUSIC spatial spectrum as a sound source position.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the multi-sound source localization method as described in any one of claims 1 to 8.

Images