Sound source localization method based on distributed array

Abstract

The invention discloses a sound source localization method based on a distributed array, and relates to the field of sound source localization. A distributed array and a centralized array are combined, the horizontal position information of a sound source is determined through a distributed array algorithm, and the vertical position information of the sound source is determined through a centralized array algorithm. Compared with other sound source localization methods, the sound source localization method has the advantages that coarse localization is conducted firstly, then fine localization is conducted, the CLEAN-SC algorithm is used for optimizing the wave beams, the characteristics of low calculation complexity and high resolution of the CLEAN-SC algorithm are fully utilized, and the sound source localization method with more excellent performance is achieved.

Description

Technical Field

[0001] The present invention relates to the field of sound source localization, and in particular to a sound source localization method based on a distributed microphone array. Background Art

[0002] When a single microphone collects sound data, due to the spatial distribution characteristics of interference and noise, the movement of the sound source signal will cause the spatial position of the signal source to change, which in turn leads to a decrease in the quality of sound signal pickup. Compared with a single microphone, a microphone array obtains the spatial information of the sound source signal through the small time difference between the multi-channel data collected by multiple microphones. Although the current mainstream sound source localization technology, such as beamforming technology, can effectively target the main sound source, since beamforming technology relies on the pre-designed array distance to calculate the small time delay or frequency domain phase shift between multi-channel data, the spatial resolution of a single array is inevitably limited by physical size. In beamforming, in order to maintain excellent spatial resolution in a larger target area, the size of the array must be large enough, which inevitably leads to the problem of expensive and bulky arrays, greatly limiting the possibility of practical application. Therefore, in large environments, multiple microphone arrays, namely distributed microphone arrays, are required to cover the entire relevant space.

[0003] In a distributed microphone array, the position of the microphone is no longer limited to the traditional fixed geometry, providing a wide and flexible spatial coverage range. Multiple arrays will have overlapping areas when locating the sound source. In these areas, the results of integrating multiple arrays will obtain more accurate positioning results than a single array. When the sound source point is far away from the microphone array, the array's positioning effect on the sound source point will be reduced due to the attenuation of the sound wave. Therefore, when the geometry and background signal-to-noise ratio of the two arrays are the same, the closer array will estimate the position more accurately than the farther array, and the weight will account for a larger proportion. Compared with a single centralized array, a distributed array has advantages in acoustic coverage and has been widely used in fields such as volcanic activity monitoring and earthquake early warning. However, it also brings some challenges, mainly including variable array structure, different gains due to different distances from the sound source to the array, and asynchronous recording. Summary of the invention

[0004] The present invention proposes a method for three-dimensional spatial positioning of a sound source, which combines a distributed array with a centralized array, wherein the distributed array algorithm determines the horizontal position information of the sound source, and the centralized array algorithm determines the vertical position information of the sound source.

[0005] The technical solution of the present invention is a sound source localization method based on a distributed array: the method comprises:

[0006] Step 1: Arrange multiple microphone sensors in a row to form a single microphone array, and arrange the multiple microphone arrays in space;

[0007] Step 2: Use multiple microphone arrays to collect the total sound signal in the space, and pre-process the collected sound data signal, including: (1) sampling filtering to ensure the correct reconstruction of the sound signal; (2) audio validity detection, perform valid endpoint detection on the received data, remove invalid sound data, and ensure the real-time performance of the algorithm; (3) noise reduction, perform spectral subtraction noise reduction on the collected data; (4) windowing and framing, decompose the long audio signal into millisecond audio frames for smooth approximate processing;

[0008] Step 3: Perform wide-range sound source localization to estimate the horizontal position of the sound source;

[0009] Step 4: Determine the vertical position of the sound source; determine whether the frequency of the collected sound signal is lower than the set threshold, if so, proceed to step 5; if not, directly output the position;

[0010] Step 5: Use the CLEAN-SC algorithm to process and locate the sound source.

[0011] Furthermore, the specific method of step 3 is:

[0012] Step 3.1: The i-th sound source signal is used represents, t represents time, Indicates that at position l The position in space is represented by a three-dimensional vector, s i is a function of time and position, representing the acoustic signal of the i-th sound source signal at time t at position l; the signal received by the m-th microphone array is expressed as:

[0013]

[0014] in, Indicates that the sound signal comes from arrive The impulse response of the propagation path, Indicates the location of the mth microphone array. The location in space is represented by a three-dimensional vector. n m (t) represents all noise sources;

[0015] Step 3.2: For x m (t) is subjected to Fourier transform, and we get

[0016]

[0017] in, represents the Fourier transform of the ith sound source signal, Indicates from arrive The Fourier transform of the impulse response of the propagation path, represents the Fourier transform of the ambient noise signal at the mth microphone array, ω represents the frequency;

[0018] Step 3.3: Perform weighted processing to obtain weighted results

[0019]

[0020] in, It represents the definition, β is the parameter that controls the degree of spectrum whitening, and its value is [0,1], |.| β represents the β-th power of the 2-norm; weighted processing is to deconvolve the spectrum so that each frequency region contributes more evenly to the coherent sum of steering power, and the β parameter limits the enhancement of low-level spectrum regions dominated by noise;

[0021] Step 3.4: The weighted result of step 3.3 is expressed in the frequency domain as follows:

[0022]

[0023] Among them, S l represents the coherent sum estimated at the estimation point l, M represents the distributed array has a total of M microphone subarrays, It represents the weighted result of the Fourier transform of the signal received by the mth microphone divided by its own modulus to the power of beta. q represents the serial number of the microphone array, and m also represents the serial number of the microphone array. Assuming there are M microphone arrays in total, they are recorded as microphone arrays 1, 2...m...M. Here, the correlation between the signals received between any two microphone arrays is calculated.

[0024] Furthermore, the specific method of step 4 is:

[0025] Step 4.1: For a microphone array, there are Q microphone elements, each microphone is represented by a sub-index q, q∈{1,…,Q}, and each microphone outputs a discrete-time signal s q (n), at the spatial point x = [x, y, z] T The steering response power P0(x) on is expressed as:

[0026]

[0027] Where Z represents s q(n) The set of all points, indicates a macro equal to, meaning that P0(x) is equal to Similar to τ q (x) represents the time delay of the signal from the sound source at x to the qth microphone;

[0028] Step 4.2: After weighting P0(x), we get the weighted result P(x):

[0029]

[0030] in, is a frequency domain weighting function. For each microphone array, there are a total of Q microphones. Indicates that the q1th microphone receives the signal s q1 The Fourier transform of (n) is similarly S q2 (e jω ) represents the discrete signal s received by the q2th microphone q2 Fourier transform of (n), the superscript “*” indicates the conjugate transpose, is defined as:

[0031]

[0032] Among them, f s represents the sampling frequency of the system, v is the speed of sound, x q represents the position of the qth microphone, ‖·‖ represents the absolute value, and “” represents the rounding operator;

[0033] in,

[0034] Estimate P(x) on each candidate sound source grid point G. The grid point corresponding to the maximum value is the spatial position of the sound source. Right now:

[0035]

[0036] Furthermore, the specific method of step 5 is:

[0037] Step 5.1: Scan the source plane grid points get The sound source power at for:

[0038]

[0039] in, is the cross power spectrum matrix, Represents a vector, which is calculated by taking the unit vector of the steering vector. Assume that for the focus point The steering vector is The vector is The superscript "T" indicates transposition, from which the global maximum position is found Subtract globally The influence of the peak value is eliminated to obtain the sound power not affected by the peak value.

[0040]

[0041] in, Is The cross-spectral matrix caused by the sound source at With peak position The acoustic mutual power must be determined by Decide, that is, arbitrarily exist:

[0042]

[0043] in, yes The weight vector of It represents the guiding vector with the peak value as the sound source in the i-th iteration. represents the cross-spectral matrix of the microphone sound pressure signal obtained in the i-1th iteration. CLEAN-SC is a multi-iteration algorithm, and the superscript indicates the number of iterations. And assume It is composed of a single coherent sound source component The only solution is:

[0044]

[0045] in, represents the maximum sound intensity value of the scanned area in the i-1th iteration;

[0046]

[0047] (m,n)m and n are the indices of the microphones, and S represents all possible combinations of microphones. represents the sound pressure distribution calculated by the mth microphone, represents the sound pressure distribution calculated at the nth microphone; Contains diagonal elements Therefore, it is necessary to iterate at this time,

[0048]

[0049] It means that the matrix after multiplying the source component and its transpose takes the conjugate complex number;

[0050] Step 5.2: Update the sound source map obtained in step 4 by eliminating the peak position, and replace the coherent sound source with a clean beam:

[0051]

[0052] in, represents the maximum sound intensity value of the scanned area in the i-1th iteration, λ is a parameter that determines the bandwidth, and the safety factor The value range is 0 to 1, and the cross-spectral matrix becomes:

[0053]

[0054] The remaining acoustic map is:

[0055]

[0056] Continue iterating until:

[0057]

[0058] The final sound source intensity distribution diagram A j It is the superposition of the clean beam and the residual sound source intensity distribution map, that is:

[0059]

[0060] According to the sound source intensity distribution diagram A j Get the sound source location.

[0061] Compared with other sound source localization methods, the method of the present invention performs coarse positioning first and then fine positioning, and uses the CLEAN-SC algorithm to optimize the beam, making full use of its low computational complexity and high resolution to achieve a sound source localization method with better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 The figure is a flow chart of the sound source localization method.

[0063] Figure 2 Schematic diagram of distributed array sound source localization.

[0064] Figure 3 This is the sound field effect diagram obtained by the algorithm in step 3.

[0065] Figure 4 The beam patterns are obtained by processing the same signal with different algorithms.

[0066] Figure 5 This is the sound field effect diagram obtained by the algorithm in step 4.

[0067] Figure 6 This is the sound field effect diagram obtained by the step 4 algorithm under low-frequency signals.

[0068] Figure 7 This is the sound field effect diagram obtained by the CLEAN-SC algorithm under low-frequency signals. DETAILED DESCRIPTION

[0069] The specific implementation process of the sound source localization method is as follows: Figure 1 Step 1: Set up the collection device. In the audio collection stage, it is necessary to arrange the spatial distribution of the distributed microphone array, such as Figure 2 As shown in the figure; Step 2 collects audio and preprocesses the audio; the preprocessed sound signal is passed through the algorithm in Step 3 to obtain its approximate horizontal position; the algorithm in Step 4 is used to further locate the vertical position of the sound signal. If the frequency of the audio is low, the algorithm in Step 4 cannot obtain a better result, and the improved CLEAN-SC algorithm needs to be used for processing and positioning to finally obtain the accurate position of the sound source.

[0070] Step 3 algorithm is an algorithm for localizing a large range of sound sources using a distributed array. It requires multiple microphone arrays distributed at different locations in the space to collect the sound signals in the space. The i-th sound source signal is used Indicates that the signal received by the mth microphone array element is expressed as:

[0071]

[0072] in, Indicates that the sound signal comes from arrive Impulse response of the propagation path, n m (t) represents all noise sources, and the above equation is transformed by Fourier transform:

[0073]

[0074] right Perform PHAT-β weighting, defined as follows:

[0075]

[0076] Among them, β is a parameter that controls the degree of spectrum whitening, and its value is [0,1]. PHAT-β mainly deconvolves the spectrum so that each frequency region contributes more evenly to the coherent sum of steering power. The β parameter limits the enhancement of low-level spectrum regions dominated by noise. In this way, the weighted SRCP relative to position l can be expressed in the frequency domain as:

[0077]

[0078] Scan all grid points in the observation area and create an SRCP image for each grid point using the above formula. Figure 3 shown.

[0079] After obtaining the approximate horizontal position of the sound source signal using the algorithm in step 3, it can be positioned in a small range and accurately positioned using the algorithm in step 4 using a single microphone array.

[0080] Step 4 is a beamforming algorithm that searches for candidate positions or directions that maximize the directional delay and beamformer output. For a microphone array, there are Q microphones, each microphone is represented by a subindex q, q∈{1,…,Q}, and each microphone outputs a discrete-time signal s m (n), at the spatial point x = [x, y, z] T The SRP response on can be expressed as:

[0081]

[0082] Where Z represents s q (n) The set of all points, τ q (x) represents the time delay of the sound source signal at x to the qth microphone.

[0083] After weighting, we get:

[0084]

[0085] in, is a frequency-domain weighting function, is defined as:

[0086]

[0087] Among them, f s represents the sampling frequency of the system, v is the speed of sound, x q represents the position of the qth microphone, ‖·‖ represents the absolute value, and “” represents the rounding operator.

[0088] if is PHAT weighted, at this time

[0089]

[0090] PHAT weighting normalizes the amplitude and retains only the phase information. Step 4 The algorithm searches the sound source spatial grid and estimates P(x) on each candidate sound source grid point G. The grid point corresponding to the maximum SRP value is the spatial position x of the sound source.s ,Right now:

[0091]

[0092] Compared with the MVDR algorithm and the DSB algorithm, the step 4 algorithm has better directivity, is less affected by noise, and has higher credibility. The comparison results are as follows: Figure 4 The sound source effect diagram obtained by the algorithm in step 4 is as follows Figure 5 .

[0093] Since the sound frequency emitted by the sound source may be low-frequency, the low-frequency audio will cause the positioning result obtained by the algorithm in step 4 to have a low resolution. Therefore, the CLEAN-SC algorithm is used for low-frequency signals to obtain a higher-resolution positioning, such as Figure 6 Figure 7 shown.

[0094] The CLEAN-SC algorithm first scans the plane grid points of the sound source using traditional beamforming technology. get The sound source power at for:

[0095]

[0096] in, is the cross-power spectrum matrix, from which the global maximum position is found Subtract globally The influence of the peak value is eliminated to obtain the sound power not affected by the peak value.

[0097]

[0098] in, Is The cross-spectral matrix of the sound source at the location is obtained. The CLEAN-SC algorithm requires that any scanning point With peak position The acoustic mutual power must be determined by Decide, that is, arbitrarily exist:

[0099]

[0100] in, is with The relevant weight vector. When it satisfies And assume It is composed of a single coherent sound source component The only solution is:

[0101]

[0102] in,

[0103]

[0104] Contains diagonal elements Therefore, it is necessary to iterate at this time,

[0105]

[0106] The original sound source map is updated by removing the peak position, and the coherent sound source is replaced by a clean beam:

[0107]

[0108] Among them, λ is a parameter that determines the bandwidth, and the safety factor The value range is 0 to 1, and the cross-spectral matrix becomes:

[0109]

[0110] The remaining acoustic map is

[0111]

[0112] Continue iterating until:

[0113]

[0114] The final sound source intensity distribution map is the superposition of the clean beam and the residual sound source intensity distribution map, that is:

[0115]

[0116] The final sound source intensity distribution map has strong directivity and high accuracy.

Claims

1. A sound source localization method based on a distributed array, the method comprising: Step 1: Arrange multiple microphone sensors in a row to form a single microphone array, and arrange the multiple microphone arrays in space; Step 2: Use multiple microphone arrays to collect the total sound signal in the space, and pre-process the collected sound data signal, including: (1) sampling and filtering to ensure correct reconstruction of the sound signal; (2) Audio validity detection: perform valid endpoint detection on the received data and remove invalid sound data to ensure the real-time performance of the algorithm; (3) Noise reduction: perform spectral subtraction noise reduction on the collected data; (4) Windowing and framing: decompose the long audio signal into millisecond audio frames for smooth approximate processing; Step 3: Perform wide-range sound source localization to estimate the horizontal position of the sound source; Step 4: Determine the vertical position of the sound source; determine whether the frequency of the collected sound signal is lower than the set threshold, if so, proceed to step 5; if not, directly output the position; Step 5: Use the CLEAN-SC algorithm to process and locate the sound source.

2. A sound source localization method based on a distributed array as claimed in claim 1, characterized in that: The specific method of step 3 is: Step 3.1: The i-th sound source signal is used represents, t represents time, Indicates that at position l The position in space is represented by a three-dimensional vector, s i is a function of time and position, representing the acoustic signal of the i-th sound source signal at time t at position l; the signal received by the m-th microphone array is expressed as: in, Indicates that the sound signal comes from arrive The impulse response of the propagation path, Indicates the location of the mth microphone array. The location in space is represented by a three-dimensional vector. n m (t) represents all noise sources; Step 3.2: For x m (t) is subjected to Fourier transform, and we get in, represents the Fourier transform of the ith sound source signal, Indicates from arrive The Fourier transform of the impulse response of the propagation path, represents the Fourier transform of the ambient noise signal at the mth microphone array, ω represents the frequency; Step 3.3: Perform weighted processing to obtain weighted results in, It represents the definition, β is the parameter that controls the degree of spectrum whitening, and its value is [0,1], |.| β represents the β-th power of the 2-norm; weighted processing is to deconvolve the spectrum so that each frequency region contributes more evenly to the coherent sum of steering power, and the β parameter limits the enhancement of low-level spectrum regions dominated by noise; Step 3.4: The weighted result of step 3.3 is expressed in the frequency domain as follows: Among them, S l represents the coherent sum estimated at the estimation point l, M represents the distributed array has a total of M microphone subarrays, It represents the weighted result of the Fourier transform of the signal received by the mth microphone divided by its own modulus raised to the power of beta. q represents the serial number of the microphone array, and m also represents the serial number of the microphone array. Assuming there are M microphone arrays in total, they are recorded as microphone arrays 1, 2...m...M. Here, the correlation between the signals received between any two microphone arrays is calculated.

3. A sound source localization method based on a distributed array as claimed in claim 1, characterized in that: The specific method of step 4 is: Step 4.1: For a microphone array, there are Q microphone elements, each microphone is represented by a sub-index q, q∈{1,…,Q}, and each microphone outputs a discrete-time signal s q (n), at the spatial point x = [x, y, z] T The steering response power P0(x) on is expressed as: Where Z represents s q (n) the set of all points, indicates a macro equal to, meaning that P0(x) is equal to Similar to τ q (x) represents the time delay of the signal from the sound source at x to the qth microphone; Step 4.2: After weighting P0(x), we get the weighted result P(x): in, is a frequency domain weighting function. For each microphone array, there are a total of Q microphones. Indicates that the q1th microphone receives the signal s q1 The Fourier transform of (n) is similarly S q2 (e jω ) represents the discrete signal s received by the q2th microphone q2 Fourier transform of (n), the superscript "*" indicates the conjugate transpose, is defined as: Among them, f s represents the sampling frequency of the system, v is the speed of sound, x q represents the position of the qth microphone, ‖·‖ represents the absolute value, and “” represents the rounding operator; in, Estimate P(x) on each candidate sound source grid point G. The grid point corresponding to the maximum value is the spatial position of the sound source. Right now:

4. A sound source localization method based on a distributed array as claimed in claim 1, characterized in that: The specific method of step 5 is: Step 5.1: Scan the source plane grid points get The sound source power P j (0) for: in, is the cross power spectrum matrix, Represents a vector, which is calculated by taking the unit vector of the steering vector. Assume that for the focus point The steering vector is The vector is The superscript "T" indicates transposition, from which the global maximum position is found Subtract globally The influence of the peak value is eliminated, and the sound power P not affected by the peak value is obtained. j (i) : in, Is The cross-spectral matrix caused by the sound source at With peak position The acoustic mutual power must be determined by Decide, that is, arbitrarily exist: in, yes The weight vector of It represents the guiding vector with the peak value as the sound source in the i-th iteration. represents the cross-spectral matrix of the microphone sound pressure signal obtained in the i-1th iteration. CLEAN-SC is a multi-iteration algorithm, and the superscript indicates the number of iterations. And assume It is composed of a single coherent sound source component The only solution is: in, represents the maximum sound intensity value of the scanned area in the i-1th iteration; (m,n)m and n are the indices of the microphones, and S represents all possible combinations of microphones. represents the sound pressure distribution calculated by the mth microphone, represents the sound pressure distribution calculated at the nth microphone; Contains diagonal elements Therefore, it is necessary to iterate at this time, It means that the matrix after multiplying the source component and its transpose takes the conjugate complex number; Step 5.2: Update the sound source map obtained in step 4 by eliminating the peak position, and replace the coherent sound source with a clean beam: in, represents the maximum sound intensity value of the scanned area in the i-1th iteration, λ is a parameter that determines the bandwidth, and the safety factor The value range is 0 to 1, and the cross-spectral matrix becomes: The remaining acoustic map is: Continue iterating until: The final sound source intensity distribution diagram A j It is the superposition of the clean beam and the residual sound source intensity distribution map, that is: According to the sound source intensity distribution diagram A j Get the sound source location.

Images