A fast method for far-field multiple sound source localization based on dual-stage search MVDR

Abstract

The invention discloses a far-field multi-sound source rapid positioning method based on a dual-stage search MVDR, the method comprising: using a sound source identification and positioning system to obtain mono audio data, and performing low-pass filtering on the mono audio data; performing static MFCC feature extraction and zero-crossing feature extraction on the mono audio data, and fusing the two features; pre-training a classification matrix group based on a BP network, sending the fusion feature to the trained classification matrix group based on a BP network for identification, comparing the audio data FFT result with the preset sound source category corresponding frequency interval, and determining the frequency for MVDR calculation; using all microphones of a six-element uniform circular array for synchronous data acquisition, calculating the sound data covariance matrix, executing a dual-stage search MVDR rapid algorithm, and realizing pitch angle and azimuth angle estimation. The invention can perform rapid identification and positioning of multiple sound sources, and has the advantages of high accuracy, low calculation amount, stable operation, and high reliability.

Description

Technical Field

[0001] The invention relates to the technical fields of audio recognition and sound source localization, in particular to a far-field multiple sound source fast localization method based on a double-stage search MVDR. Background Art

[0002] With the improvement of hardware computing power and the advancement of algorithm accuracy, sound source localization technology has become increasingly handy in solving positioning problems in complex environments. It has penetrated into all levels from consumer applications, industrial applications to military applications, from aerial reconnaissance, ocean detection to industrial fault diagnosis, and then to smart home systems, smart wearable devices and remote video conferencing systems. However, since audio signals are broadband signals, traditional narrowband signal positioning methods are not applicable.

[0003] The conventional method to solve the problem of broadband incoherent audio signal positioning is to decompose the broadband sound source into multiple narrowband components in the frequency domain, directly perform narrowband processing on each subband, and obtain the spatial spectral density matrix of the subband corresponding to each frequency point by traversing a large number of azimuth and elevation angles. Other studies have used the generalized cross-correlation method, which ignores the limitations in the frequency domain and directly calculates the time domain signal, which has a certain robustness to environmental noise. However, when the sound source is located in a distant place outside the array range, the positioning accuracy of this method will be greatly reduced, and the performance of this method cannot be maintained in a reverberant environment. It can be seen that the existing processing methods for the positioning of broadband audio signals usually face the following problems: (1) The traditional far-field positioning method based on frequency domain decomposition has high computational complexity and a long single positioning time; (2) The method based on generalized cross-correlation is not suitable for the positioning of sound sources far away from the center of the array, and the actual sound source usually cannot be determined at a certain distance, so the effect is unstable. Summary of the invention

[0004] The object of the present invention is to provide a far-field multiple sound source rapid positioning method based on a double-stage search MVDR with high positioning accuracy, strong robustness, low computational complexity and low implementation cost.

[0005] The technical solution to achieve the purpose of the present invention is: a method for fast positioning of multiple sound sources in the far field based on a dual-stage search MVDR, comprising the following steps:

[0006] Step 1: Use a sound source recognition and positioning system to obtain mono audio data, and use short-time Fourier transform to perform low-pass filtering on the mono audio data;

[0007] Step 2: Perform static MFCC feature extraction, first-order difference MFCC feature extraction and zero-crossing feature extraction on the mono audio data, and fuse the two features;

[0008] Step 3: pre-train the classification matrix group based on BP network, send the fusion features into the trained classification matrix group based on BP network, output the sound source category, compare the FFT result of the audio data with the frequency interval corresponding to the preset sound source category, and determine the frequency used for MVDR calculation;

[0009] Step 4: Use all microphones of the six-element uniform circular array to perform synchronous data acquisition, calculate the sound data covariance matrix, and execute the two-stage search MVDR fast algorithm to achieve pitch angle and azimuth angle estimation.

[0010] Compared with the prior art, the present invention has the following significant advantages: (1) the output information of the sound source identification module is used to determine the frequency used for the positioning algorithm, thus avoiding multiple FFT calculations and having a certain degree of robustness against reverberation environment and out-of-range single-frequency noise interference; (2) the two-stage search MVDR method is used as the positioning algorithm, which reduces the probability of ghost images under multi-sound source positioning; (3) the calculation amount is low, it is easy to implement, the operation is stable, and the reliability is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 The present invention is a flowchart of a method for rapidly locating multiple sound sources in the far field based on a dual-stage search MVDR.

[0012] Figure 2 It is a confusion matrix diagram of the BP network test set when only static MFCC and first-order difference MFCC features are used as input in an embodiment of the present invention.

[0013] Figure 3 It is a confusion matrix diagram of the BP network test set when the fused static MFCC, the first-order difference MFCC features, and the zero-crossing point features are used as input in the embodiment of the present invention.

[0014] Figure 4 It is a trend diagram of the positioning RMSE values ​​of the traditional MVDR and the two-stage search MVDR algorithms as the SNR changes in the embodiments of the present invention. DETAILED DESCRIPTION

[0015] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0016] Combination Figure 1 The present invention provides a far-field multiple sound source rapid positioning method based on a dual-stage search MVDR (Minimum Variance Distortionless Response), comprising the following steps:

[0017] Step 1: Use a sound source recognition and positioning system to obtain mono audio data, and use short-time Fourier transform to perform low-pass filtering on the mono audio data;

[0018] Step 2: Perform static MFCC feature extraction, first-order difference MFCC feature extraction and zero-crossing feature extraction on the mono audio data, and fuse the two features;

[0019] Step 3: pre-train the classification matrix group based on BP network, send the fusion features into the trained classification matrix group based on BP network, output the sound source category, compare the FFT result of the audio data with the frequency interval corresponding to the preset sound source category, and determine the frequency used for MVDR calculation;

[0020] Step 4: Use all microphones of the six-element uniform circular array to perform synchronous data acquisition, calculate the sound data covariance matrix, and execute the two-stage search MVDR fast algorithm to achieve pitch angle and azimuth angle estimation.

[0021] As a specific example, the sound source identification and positioning system in step 1 includes a data processing module, a microphone array module and a multi-channel ADC module;

[0022] The data processing module is a microcomputer system integrating a processor core, a memory, an input / output interface and a functional module, performs data communication through an I2S bus and an external DAC module, stores matrix data through a built-in RAM, uses a logic calculation unit and a mini-DSP module to perform matrix calculations, and transmits sound source identification and positioning results to other devices through a built-in UART peripheral;

[0023] The microphone array module is an array composed of multiple capacitive microphones configured as a uniform circular array. When sound waves reach the microphones, the pressure of the sound waves causes the diaphragm to vibrate, thereby changing the distance between the diaphragm and the back plate, causing the capacitance value to change, and then causing the voltage signal to change. The voltage signal is amplified and processed by a dedicated audio operational amplifier circuit and output to the multi-channel ADC module;

[0024] The multi-channel ADC module uses a dedicated audio ADC chip to perform analog-to-digital conversion on the output signal of the analog microphone array to generate a data signal suitable for the I2S bus.

[0025] As a specific example, the microphone array module adopts a six-element uniform circular microphone array.

[0026] As a specific example, in step 1, a sound source identification and positioning system is used to obtain mono audio data, and a short-time Fourier transform is used to low-pass filter the mono audio data, as follows:

[0027] Step 1.1, using the sound source recognition and positioning system to obtain mono audio data;

[0028] Step 1.2: Perform low-pass filtering on the mono audio data using the short-time Fourier transform. The formula is as follows:

[0029]

[0030] where \(x(t)\) is the original signal, \(\omega\) is the sound source frequency, \(w(t - \tau)\) is the window function centered at \(\tau\). Since the range of non-zero values on the time axis is limited, the actual integration is performed within the non-zero region of the window function. Therefore, the window function is much smaller than the entire time axis.

[0031] Step 1.3: After STFT, obtain the sound source frequency range (\(f\) l , \(f\) h ), \(f\) l , \(f\) h are the minimum and maximum values of the sound source frequency, respectively.

[0032] As a specific example, in Step 2, perform static MFCC feature extraction, first-order difference MFCC feature extraction, and zero-crossing feature extraction on the mono audio data, and fuse the two types of features as follows:

[0033] Step 2.1: Perform MFCC feature extraction and first-order difference MFCC feature extraction on the filtered mono audio data as follows:

[0034] Step 2.1.1: Convert the linear frequency \(f\) to the Mel frequency \(f\) mel , and the formula is:

[0035]

[0036] Step 2.1.2: Convert the sound source frequency range (\(f\) l , \(f\) h ) to (\(f\) l-mel , \(f\) h-mel ) according to Equation (2). \(f\) l-mel , \(f\) h-mel represent the minimum and maximum values of the Mel frequency, respectively. Set the required MFCC dimension to \(\Upsilon\) and the number of Mel filters to \(M\), where \(\Upsilon < M\). Then the center frequency \(f\) mel \((m)\) of the \(m\)-th filter in the Mel scale is:

[0037]

[0038] Step 2.1.3: Convert the Mel center frequencies of each Mel filter back to the linear frequency. The formula is:

[0039]

[0040] where \(f\) is the actual frequency in Hz;

[0041] Step 2.1.4: Generate the Mel filter bank. The formula is:

[0042]

[0043] Among them, H m (k) is the Mel filter function, f(m), 1≤m≤M represents the center frequency of the mth Mel filter, each center frequency is equally spaced in the Mel domain and unequally spaced in the actual frequency domain;

[0044] Step 2.1.5: Take the logarithm of the output of the Mel filter bank. The formula is:

[0045]

[0046] Among them, X(k) is the linear spectrum, s(m) is the logarithmic spectrum, and K is the number of FFT points;

[0047] Step 2.1.6: Perform DCT transformation on the logarithmic output to obtain the static MFCC coefficients. The formula is:

[0048]

[0049] Among them, c γ is the γth discrete cosine transform coefficient, i.e., the static MFCC coefficient, where γ is the index of the DCT coefficient, ranging from 0 to Υ-1;

[0050] Step 2.1.7: Generate first-order difference MFCC coefficients using static MFCC coefficients. The formula is:

[0051] ΔMFCC d (t) = MFCC d (t)-MFCC d (t-1) (8)

[0052] Among them, ΔMFCC d (t) is the first-order difference of the d-th MFCC coefficient at time t, MFCC d (t) is the dth MFCC coefficient at the current moment, MFCC d (t-1) is the MFCC coefficient of the previous moment, and the differential coefficient of the first frame is the difference between the last MFCC factor and the first MFCC factor of each frame;

[0053] Step 2.2: Extract zero-crossing features from the filtered mono audio data, as follows:

[0054] Step 2.2.1, calculate the short-time zero crossing rate ZCR of the mono audio data, the formula is:

[0055]

[0056] In the current frame, the total number of collected signal points is N, and the signal is represented by s(n), n∈[1,N], where n is an integer and sgn(x) is a sign function;

[0057] Step 2.2.2, calculate the zero-crossing sequence z[n] of the mono audio data, the formula is:

[0058]

[0059] Step 2.2.3, calculate the zero-crossing interval length sequence ZCIS of the mono audio data, for each pair of consecutive zero-crossing points z[n i ] and z[n i+1 ], interval length I i is the number of sampling points between two zero crossings:

[0060] I i =n i+1 -n i (11)

[0061] where n i Indicates the number of signal sampling points corresponding to the i-th value of 1 in the zero-crossing sequence;

[0062] Step 2.2.4, perform zero-crossing feature analysis on various target signals to obtain ZCIS that is different from background noise;

[0063] Step 2.3: Fuse the static MFCC features and zero-crossing features of the mono audio data.

[0064] As a specific example, the pre-trained classification matrix group based on the BP network described in step 3 sends the fusion feature to the trained classification matrix group based on the BP network, outputs the sound source category, compares the FFT result of the audio data with the frequency interval corresponding to the preset sound source category, and determines the frequency used for MVDR calculation, as follows:

[0065] Step 3.1: Pre-train the classification matrix group based on BP network, as follows:

[0066] Step 3.1.1, configure BP network parameters:

[0067] The static MFCC and zero-crossing fusion features of mono audio data include 12th-order static MFCC features, 12th-order first-order difference MFCC features, 1st-order ZCR features, and 4th-order ZCIS features. Therefore, the number of input layer neurons p is configured to be 29, the number of output layer neurons q is configured according to the number of sound source categories, and the number of hidden layer neurons h is configured according to the number of input layer neurons p and the number of output layer neurons q. The formula is:

[0068]

[0069] Where a is a regulation function that is adjusted between 1 and 10;

[0070] Step 3.1.2, generate weight matrix and bias matrix:

[0071] Generate the weight matrix W from the input layer to the hidden layer (1) and the bias matrix b (1) , the dimensions are p×h and h×1 respectively;

[0072] Generate the weight matrix W from the hidden layer to the output layer (2) and the bias matrix b (2) , the dimensions are h×q and q×1 respectively;

[0073] Step 3.2: Send the fusion features to the trained BP network-based classification matrix group and output the sound source category, as follows:

[0074] Assuming the fusion feature of the input signal is matrix x and the recognition result is matrix Y, we have:

[0075]

[0076] The function f is the Softmax activation function, which is defined as follows:

[0077]

[0078] Step 3.3: Compare the FFT result of the audio data with the frequency interval corresponding to the preset sound source category to determine the frequency used for MVDR calculation, as follows:

[0079] A frequency interval is preset for each sound source category, and the maximum frequency component of the audio data FFT is compared with the frequency interval of the corresponding category. If the frequency falls within the interval, it is used for directional solution; if it does not fall within the interval, the second largest frequency component is compared until S frequencies fall within the interval, where S is the number of sound sources.

[0080] As a specific example, all microphones of the six-element uniform circular array described in step 4 are used to perform synchronous data acquisition, calculate the sound data covariance matrix, and execute the two-stage search MVDR fast algorithm to achieve pitch angle and azimuth orientation, as follows:

[0081] Step 4.1: Use all microphones of the six-element uniform circular array to collect data synchronously and calculate the covariance matrix as follows:

[0082]

[0083] The matrix R is the covariance matrix under a finite number of snapshots, with a dimension of 6×6, L is the number of snapshots, and the matrix X(l) is the sampling signal of each array element, with a dimension of 6×1;

[0084] Step 4.2: Perform a primary search:

[0085] Set the search step length θ of the primary search azimuth angle to I Set to 2.5°, the search step size of the pitch angle Set to 5°, at θ I ∈[0,π], Calculate the MVDR power within the range and find the peak point The formula is:

[0086]

[0087] in is the array manifold matrix of the six-element uniform circular array, assuming there are S sound sources, then The steering vector of the sth sound source is:

[0088]

[0089] Where λ is the wavelength of the sound source;

[0090] Step 4.3, perform a secondary search:

[0091] The azimuth search step and elevation search step of the secondary search are set to θ II and Set to 0.5°, in θ∈[θ C -θ I ,θ C +θ I ], Calculate the MVDR power within the range and find the peak point

[0092] As a specific example, the method is used to implement a sound source identification and positioning system suitable for a low-computing-power master control, and the method includes a sound source identification module and a sound source positioning module, wherein:

[0093] The sound source recognition module uses static MFCC features, first-order difference MFCC features, and zero-crossing features as fusion features to extract features from the received sound signal, inputs the features into a pre-trained BP classification matrix, outputs the sound source category, and selects each sound source frequency for the positioning module according to the preset frequency intervals of each category;

[0094] The sound source localization module obtains the covariance matrix of the received signal of the microphone array, runs the two-stage search MVDR algorithm based on the sound source frequencies output by the identification module, calculates the popular matrix under different pitch angles and azimuth angles, calculates the MVDR power, and finally obtains the accurate azimuth and pitch angle of the sound source.

[0095] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0096] Example 1

[0097] The present invention provides a far-field multiple sound source rapid positioning method based on a dual-stage search MVDR, comprising the following steps:

[0098] Step 1: Use a sound source recognition and positioning system to obtain mono audio data, and use short-time Fourier transform to perform low-pass filtering on the mono audio data;

[0099] Step 2: Perform static MFCC feature extraction, first-order difference MFCC feature extraction and zero-crossing feature extraction on the mono audio data, and fuse the two features;

[0100] Step 3: pre-train the classification matrix group based on BP network, send the fusion features into the trained classification matrix group based on BP network, output the sound source category, compare the FFT result of the audio data with the frequency interval corresponding to the preset sound source category, and determine the frequency used for MVDR calculation;

[0101] Step 4: Use all microphones of the six-element uniform circular array to perform synchronous data acquisition, calculate the sound data covariance matrix, and execute the two-stage search MVDR fast algorithm to achieve pitch angle and azimuth angle estimation.

[0102] As a specific example, the sound source identification and positioning system described in step 1 includes a data processing module, a microphone array module and a multi-channel ADC module;

[0103] The data processing module is a microcomputer system integrating a processor core, a memory, an input / output interface and a functional module, performs data communication through an I2S bus and an external DAC module, stores matrix data through a built-in RAM, uses a logic calculation unit and a mini-DSP module to perform matrix calculations, and transmits sound source identification and positioning results to other devices through a built-in UART peripheral;

[0104] The microphone array module is an array composed of multiple capacitive microphones configured as a uniform circular array. When sound waves reach the microphones, the pressure of the sound waves causes the diaphragm to vibrate, thereby changing the distance between the diaphragm and the back plate, causing the capacitance value to change, and then causing the voltage signal to change. The voltage signal is amplified and processed by a dedicated audio amplifier circuit and input into the multi-channel ADC module.

[0105] The multi-channel ADC module uses a dedicated audio ADC chip to perform analog-to-digital conversion on the output signal of the analog microphone array to generate a data signal suitable for the I2S bus.

[0106] As a specific example, the microphone array module adopts a six-element uniform circular microphone array.

[0107] As a specific example, in step 1, the mono audio data is obtained by using the sound source recognition and positioning system, and the mono audio data is low-pass filtered by using short-time Fourier transform, as follows:

[0108] Step 1.1, using the sound source recognition and positioning system to obtain mono audio data;

[0109] Step 1.2: Use short-time Fourier transform to low-pass filter the mono audio data. The formula is:

[0110]

[0111] Where x(t) is the original signal, ω is the sound source frequency, and w(t-τ) is the window function centered at τ. Since the range of non-zero values ​​on the time axis is limited, the actual integration is performed in the non-zero region of the window function, so the window function is much smaller than the entire time axis.

[0112] Step 1.3: After STFT, obtain the sound source frequency range (f l ,f h ).

[0113] As a specific example, step 2 performs static MFCC feature extraction, first-order difference MFCC feature extraction and zero-crossing feature extraction on the mono audio data, and fuses the two features, as follows:

[0114] Step 2.1: Perform MFCC feature extraction and first-order difference MFCC feature extraction on the filtered mono audio data, as follows:

[0115] Step 2.1.1, convert the linear frequency to Mel frequency, the formula is:

[0116]

[0117] Step 2.1.2: Set the sound source frequency range (fl , f h ) Convert to (f l-mel , f h-mel ), set the required number of MFCC dimensions to Υ and the number of Mel filters to M, where Υ < M. Then the center frequency of the m-th filter in the Mel scale is:

[0118]

[0119] Step 2.1.3. Convert the Mel center frequencies of each Mel filter back to linear frequencies. The formula is:

[0120]

[0121] where f is the actual frequency in Hz.

[0122] Step 2.1.4. Generate the Mel filter bank. The formula is:

[0123]

[0124] where H m (k) is the Mel filter function, and f(m), 1 ≤ m ≤ M represents the center frequency of the m-th Mel filter. Each center frequency is equally spaced in the Mel domain and unequally spaced in the actual frequency domain.

[0125] Step 2.1.5. Take the logarithm of the output of the Mel filter bank. The formula is:

[0126]

[0127] where X(k) is the linear spectrum, s(m) is the logarithmic spectrum, and K is the number of FFT points.

[0128] Step 2.1.6. Perform DCT transformation on the logarithmic output to obtain the static MFCC coefficients. The formula is:

[0129]

[0130] where c γ is the γ-th discrete cosine transform coefficient, i.e., the static MFCC coefficient, and γ is the index of the DCT coefficient, taking values from 0 to Υ - 1;

[0131] Step 2.1.7. Generate the first-order difference MFCC coefficients using the static MFCC coefficients. The formula is:

[0132] ΔMFCC d (t) = MFCC d (t) - MFCC d (t - 1) (8)

[0133] Where ΔMFCC d (t) is the first-order difference of the d-th MFCC coefficient at time t, MFCC d (t) is the dth MFCC coefficient at the current moment, MFCC d (t-1) is the MFCC coefficient of the previous moment, and the differential coefficient of the first frame is the difference between the last MFCC factor and the first MFCC factor of each frame;

[0134] Step 2.2: Extract zero-crossing features from the filtered mono audio data, as follows:

[0135] Step 2.2.1, calculate the short-time zero crossing rate ZCR of the mono audio data, the formula is:

[0136]

[0137] In the current frame, the total number of collected signal points is N, and the signal is represented by s(n), n∈[1,N], where n is an integer and sgn(x) is a sign function;

[0138] Step 2.2.2, calculate the zero-crossing point z[n] of the mono audio data, the formula is:

[0139]

[0140] Step 2.2.3, for each pair of consecutive zero-crossing points z[n i ] and z[n i+1 ], interval length I i is the number of sampling points between the two:

[0141] I i =n i+1 -n i (11)

[0142] where n i Indicates the number of signal sampling points corresponding to the i-th value of 1 in the zero-crossing sequence.

[0143] Step 2.2.4, perform zero-crossing feature analysis on various target signals to obtain ZCIS that is different from background noise;

[0144] Step 2.3: Fuse the static MFCC features and zero-crossing features of the mono audio data.

[0145] As a specific example, the pre-trained classification matrix group based on the BP network described in step 3 sends the fusion feature to the trained classification matrix group based on the BP network, outputs the sound source category, compares the FFT result of the audio data with the frequency interval corresponding to the preset sound source category, and determines the frequency used for MVDR calculation, as follows:

[0146] Step 3.1: Pre-train the classification matrix group based on BP network, as follows:

[0147] Step 3.1.1, configure BP network parameters:

[0148] The static MFCC and zero-crossing fusion features of mono audio data include 12th-order static MFCC features, 12th-order first-order difference MFCC features, 1st-order ZCR features, and 4th-order ZCIS features. Therefore, the number of input layer neurons p is configured to be 29, the number of output layer neurons q is configured according to the number of sound source categories, and the number of hidden layer neurons h is configured according to the number of input layer neurons p and the number of output layer neurons q. The formula is:

[0149]

[0150] Where a is a regulation function between 1 and 10. In this example, h is set to 10.

[0151] Step 3.1.2, generate weight matrix and bias matrix:

[0152] Generate the weight matrix W from the input layer to the hidden layer (1) and the bias matrix b (1) , the dimensions are p×h and h×1 respectively;

[0153] Generate the weight matrix W from the hidden layer to the output layer (2) and the bias matrix b (2) , the dimensions are h×q and q×1 respectively;

[0154] Step 3.2: Send the fusion features to the trained BP network-based classification matrix group and output the sound source category, as follows:

[0155] Assuming the fusion feature of the input signal is matrix x and the recognition result is matrix Y, we have:

[0156]

[0157] The function f is the Softmax activation function, which is defined as follows:

[0158]

[0159] Step 3.3: Compare the FFT result of the audio data with the frequency interval corresponding to the preset sound source category to determine the frequency used for MVDR calculation, as follows:

[0160] A frequency interval is preset for each sound source category, and the maximum frequency component of the audio data FFT is compared with the frequency interval of the corresponding category. If the frequency falls within the interval, it is used for directional solution; if it does not fall within the interval, the second largest frequency component is compared until S frequencies fall within the interval, where S is the number of sound sources.

[0161] In this example, the sound source categories are set to five categories, namely: tank, drone, tank + drone, male voice, and environmental noise. There are two preset sound sources, one is a tank and the other is a drone. After identification, the sound source frequency range is limited to the main sound frequency range of 50Hz to 350Hz for tanks and 500Hz to 2000Hz for drones. The maximum frequency component in each range is taken as the frequency for MVDR calculation.

[0162] As a specific example, all microphones of the six-element uniform circular array described in step 4 are used to perform synchronous data acquisition, calculate the sound data covariance matrix, and execute the two-stage search MVDR fast algorithm to achieve pitch angle and azimuth orientation, as follows:

[0163] Step 4.1: Use all microphones of the six-element uniform circular array to collect data synchronously and calculate the covariance matrix as follows:

[0164]

[0165] The matrix R is the covariance matrix under a finite number of snapshots, with a dimension of 6×6, K is the number of snapshots, and the matrix X(l) is the sampling signal of each array element, with a dimension of 6×1;

[0166] Step 4.2: Perform a primary search:

[0167] Set the search step length θ of the primary search azimuth angle to I Set to 2.5°, the search step size of the pitch angle Set to 5°, at θ I ∈[0,π], Calculate the MVDR power within the range and find the peak point The formula is:

[0168]

[0169] in is the array manifold matrix of the six-element uniform circular array, assuming there are S sound sources, then The steering vector of the sth sound source is:

[0170]

[0171] Among them, λ is the wavelength of the sound source;

[0172] Step 4.3, perform a secondary search:

[0173] The azimuth search step and elevation search step of the secondary search are set to θ II and Set to 0.5°, in θ∈[θ C -θ I ,θ C +θ I ], Calculate the MVDR power within the range and find the peak point

[0174] Example 2

[0175] This embodiment uses traditional MVDR and two-stage search MVDR algorithms to identify and locate tank sounds and drone sounds respectively. In this embodiment, a feature extraction method that fuses MFCC features and zero-crossing features is used to improve the recognition accuracy of tank sounds and drone sounds. MFCC features and improved fusion feature extraction methods are used to generate sound source feature data respectively, and the recognition accuracy of the two-stage search MVDR algorithm of the present invention is verified by comparing the confusion matrix of the BP network test set. In this simulation experiment, the feature data of the experimental group was generated using the fusion feature extraction method, and the feature data of the control group was generated using the MFCC feature extraction method. The confusion matrix of the control group's test set is shown as follows. Figure 2 As shown, the test set confusion matrix of the experimental group is as follows Figure 3 shown.

[0176] from Figure 2 , Figure 3 It can be seen that the experimental group using fusion features has a classification accuracy of 95.7%, while the control group has only 93.3%; for tanks, drones, and mixed sound sources of tanks and drones, the precision rates of the experimental group are 93.6%, 92.8%, and 95.8%, respectively, which are 2.4%, 5.1%, and 1.2% higher than those of the control group, respectively, and the indicators are better; in terms of recall rate, for tanks, drones, and mixed sound sources, the recall rates of the experimental group are 93.6%, 95.1%, and 97.9%, respectively, which are 5.6%, 0.8%, and 0.7% higher than those of the control group, respectively, and the indicators are better.

[0177] The above experiments show that, compared with using only MFCC features, the feature extraction strategy of fusing MFCC features and zero-crossing features can improve the recognition accuracy of the test set.

[0178] The simulation conditions for the sound source localization part of this embodiment are:

[0179] Assume that the frequencies of the two sound sources are 260 Hz and 1000 Hz respectively, the array sampling frequency is 8 kHz, the number of array elements is 6, the number of snapshots is 256, and the noise is Gaussian white noise.

[0180] To further verify the accuracy of the two-stage search MVDR algorithm, two experiments are conducted:

[0181] Experiment 1: The signal-to-noise ratio of the acoustic signal is fixed at 10 dB, the azimuth of the sound source is changed, and the azimuth and elevation angles of the sound source are estimated using the traditional MVDR method and the two-stage search MVDR method respectively, and the single positioning time of the two algorithms is timed using the tic and toc functions of MATLAB. The results are shown in Table 1. The average single positioning time of the traditional MVDR algorithm is 1.790 s, and the average time of the two-stage search MVDR algorithm is 0.339 s. It can be found that the operation time of the two-stage search MVDR algorithm is lower than that of the traditional MVDR algorithm, which is 81% shorter than that of the traditional MVDR algorithm on average.

[0182] Table 1

[0183]

[0184] Experiment 2: The azimuth angles of the two sound sources are fixed at 127.5° and 90°, and the elevation angles are fixed at 44° and 16°. The signal-to-noise ratio of the sound sources is changed. The azimuth angle and elevation angle estimation are performed using the traditional MVDR method and the two-stage search MVDR method. The results are shown in Table 2 and Table 3. The RMSE values ​​of the two algorithms change with the change of SNR. Figure 4 As shown in Table 2 and Table 3, Figure 4 It can be seen that there is no significant difference in positioning accuracy between the traditional MVDR algorithm and the two-stage search MVDR algorithm, and both are equally robust under low signal-to-noise ratio conditions.

[0185] Table 2

[0186]

[0187]

[0188] Table 3

[0189] SNR / dB First sound source estimation Second source estimate First sound source error Second sound source error RMSE / ° 5 123.5°,46.0° 96.5°,13.5° -4.0°,2.0° 6.5°,-2.5° 5.852 6 124.0°,46.0° 94.5°,17.5° -3.5°,2.0° 4.5°,1.5° 4.402 7 125.5°,45.5° 93.5°,16.5° -2.0°,1.5° 3.5°,0.5° 3.062 8 125.5°,45.5° 92.5°,16.5° -2.0°,1.5° 2.5°,0.5° 2.523 9 126.5°,44.5° 92.0°,17.5° -1.0°,0.5° 2.0°,1.5° 1.937 10 126.0°,45.0° 91.0°,16.5° -1.5°,1.0° 1.0°,0.5° 1.501 11 126.0°,45.0° 90.5°,15.5° -1.5°,1.0° 0.5°,-0.5° 1.369 12 127.0°,45.0° 91.0°,15.5° -0.5°,0.5° 1.0°,-0.5° 0.935 13 127.0°,44.0° 91.0°,15.5° -0.5°,0.0° 1.0°,-0.5° 0.866 14 127.5°,44.0° 90.5°,17.0° 0.0°,0.0° 0.5°,1.0° 0.791 15 127.0°,44.5° 90.5°,16.0° -0.5°,0.5° 0.5°,0.0° 0.612

[0190] The above experiments show that for incoherent sound sources, the two-stage search MVDR algorithm can significantly reduce the single positioning time while ensuring positioning accuracy.

[0191] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A far-field multiple sound source rapid localization method based on dual-stage search MVDR, characterized in that: The following steps are involved: Step 1: Use a sound source recognition and positioning system to obtain mono audio data, and use short-time Fourier transform to perform low-pass filtering on the mono audio data; Step 2: Perform static MFCC feature extraction, first-order difference MFCC feature extraction and zero-crossing feature extraction on the mono audio data, and fuse the two features; Step 3: pre-train the classification matrix group based on BP network, send the fusion features into the trained classification matrix group based on BP network, output the sound source category, compare the FFT result of the audio data with the frequency interval corresponding to the preset sound source category, and determine the frequency used for MVDR calculation; Step 4: Use all microphones of the six-element uniform circular array to perform synchronous data acquisition, calculate the sound data covariance matrix, and execute the two-stage search MVDR fast algorithm to achieve pitch angle and azimuth angle estimation; In step 2, static MFCC feature extraction, first-order difference MFCC feature extraction and zero-crossing feature extraction are performed on the mono audio data, and the two features are fused, as follows: Step 2.1: Perform MFCC feature extraction and first-order difference MFCC feature extraction on the filtered mono audio data, as follows: Step 2.1.

1. Convert the linear frequency f to Mel frequency f mel , the formula is: Step 2.1.2: Set the sound source frequency range (f l ,f h ) is converted into (f l-mel ,f h-mel ), f l-mel ,f h-mel Represent the minimum and maximum values ​​of the Mel frequency respectively, and set the required MFCC dimension to The number of Mel filters is M, Then the center frequency f of the mth filter in the Mel scale is mel (m) is: Step 2.1.3, convert the Mel center frequency of each Mel filter back to linear frequency, the formula is: Where f is the actual frequency in Hz; Step 2.1.4: Generate the Mel filter bank. The formula is: Among them, H m (k) is the Mel filter function, f(m), 1≤m≤M represents the center frequency of the mth Mel filter, each center frequency is equally spaced in the Mel domain and unequally spaced in the actual frequency domain; Step 2.1.5: Take the logarithm of the output of the Mel filter bank. The formula is: Among them, X(k) is the linear spectrum, s(m) is the logarithmic spectrum, and K is the number of FFT points; Step 2.1.6: Perform DCT transformation on the logarithmic output to obtain the static MFCC coefficients. The formula is: Among them, c γ is the γth discrete cosine transform coefficient, i.e., the static MFCC coefficient, γ is the index of the DCT coefficient, ranging from 0 to Step 2.1.7: Generate first-order difference MFCC coefficients using static MFCC coefficients. The formula is: ΔMFCC d (t)=MFCC d (t)-MFCC d (t-1) (8) Among them, ΔMFCC d (t) is the first-order difference of the d-th MFCC coefficient at time t, MFCC d (t) is the dth MFCC coefficient at the current moment, MFCC d (t-1) is the MFCC coefficient of the previous moment, and the differential coefficient of the first frame is the difference between the last MFCC factor and the first MFCC factor of each frame; Step 2.2: Extract zero-crossing features from the filtered mono audio data, as follows: Step 2.2.1, calculate the short-time zero crossing rate ZCR of the mono audio data, the formula is: In the current frame, the total number of collected signal points is N, and the signal is represented by s(n), n∈[1,N], where n is an integer and sgn(x) is a sign function; Step 2.2.2, calculate the zero-crossing sequence z[n] of the mono audio data, the formula is: Step 2.2.3: Calculate the zero-crossing interval length sequence ZCIS of the mono audio data. For each pair of consecutive zero-crossing points z[n i ] and z[n i+1 ], interval length I i is the number of sampling points between two zero crossings: I i =n i+1 -n i (11) Where n i Indicates the number of signal sampling points corresponding to the i-th value of 1 in the zero-crossing sequence; Step 2.2.4, perform zero-crossing feature analysis on various target signals to obtain ZCIS that is different from background noise; Step 2.3, fusing the static MFCC features and zero-crossing features of the mono audio data; The pre-trained classification matrix group based on the BP network described in step 3 sends the fusion features to the trained classification matrix group based on the BP network, outputs the sound source category, compares the FFT result of the audio data with the frequency interval corresponding to the preset sound source category, and determines the frequency used for MVDR calculation, as follows: Step 3.1: Pre-train the classification matrix group based on BP network, as follows: Step 3.1.1, configure BP network parameters: The static MFCC and zero-crossing fusion features of mono audio data include 12th-order static MFCC features, 12th-order first-order difference MFCC features, 1st-order ZCR features, and 4th-order ZCIS features. Therefore, the number of input layer neurons p is configured to be 29, the number of output layer neurons q is configured according to the number of sound source categories, and the number of hidden layer neurons h is configured according to the number of input layer neurons p and the number of output layer neurons q. The formula is: Where a is a regulation function that is adjusted between 1 and 10; Step 3.1.2, generate weight matrix and bias matrix: Generate the weight matrix W from the input layer to the hidden layer (1) and the bias matrix b (1) , the dimensions are p×h and h×1 respectively; Generate the weight matrix W from the hidden layer to the output layer (2) and the bias matrix b (2) , the dimensions are h×q and q×1 respectively; Step 3.2: Send the fusion features to the trained BP network-based classification matrix group and output the sound source category, as follows: Assuming the fusion feature of the input signal is matrix x and the recognition result is matrix Y, we have: The function f is the Softmax activation function, which is defined as follows: Step 3.3: Compare the FFT result of the audio data with the frequency interval corresponding to the preset sound source category to determine the frequency used for MVDR calculation, as follows: Preset a frequency interval for each sound source category, compare the maximum frequency component of the audio data FFT with the frequency interval of the corresponding category, and if the frequency falls within the interval, use it for directional solution; if not, compare the second largest frequency component until S frequencies fall within the interval, where S is the number of sound sources; In step 4, all microphones of the six-element uniform circular array are used to perform synchronous data acquisition, calculate the sound data covariance matrix, and execute the two-stage search MVDR fast algorithm to achieve pitch angle and azimuth angle orientation, as follows: Step 4.1: Use all microphones of the six-element uniform circular array to collect data synchronously and calculate the covariance matrix as follows: The matrix R is the covariance matrix under a finite number of snapshots, with a dimension of 6×6, L is the number of snapshots, and the matrix X(l) is the sampling signal of each array element, with a dimension of 6×1; Step 4.2: Perform a primary search: Set the search step length θ of the primary search azimuth angle to I Set to 2.5°, the search step size of the pitch angle Set to 5°, at θ I ∈[0,π], Calculate the MVDR power within the range and find the peak point The formula is: in is the array manifold matrix of the six-element uniform circular array, assuming there are S sound sources, then The steering vector of the sth sound source is: Where λ is the wavelength of the sound source; Step 4.3, perform a secondary search: The azimuth search step and elevation search step of the secondary search are set to θ II and Set to 0.5°, in θ∈[θ C -θ I ,θ C +θ I ], Calculate the MVDR power within the range and find the peak point 2. The far-field multiple sound source rapid positioning method based on dual-stage search MVDR according to claim 1 is characterized in that: The sound source identification and positioning system in step 1 includes a data processing module, a microphone array module and a multi-channel ADC module; The data processing module is a microcomputer system integrating a processor core, a memory, an input / output interface and a functional module, performs data communication through an I2S bus and an external DAC module, stores matrix data through a built-in RAM, uses a logic calculation unit and a mini-DSP module to perform matrix calculations, and transmits sound source identification and positioning results to other devices through a built-in UART peripheral; The microphone array module is an array composed of multiple capacitive microphones configured as a uniform circular array. When sound waves reach the microphones, the pressure of the sound waves causes the diaphragm to vibrate, thereby changing the distance between the diaphragm and the back plate, causing the capacitance value to change, and then causing the voltage signal to change. The voltage signal is amplified and processed by a dedicated audio operational amplifier circuit and output to the multi-channel ADC module; The multi-channel ADC module uses a dedicated audio ADC chip to perform analog-to-digital conversion on the output signal of the analog microphone array to generate a data signal suitable for the I2S bus.

3. The far-field multiple sound source rapid positioning method based on dual-stage search MVDR according to claim 2 is characterized in that: The microphone array module adopts a six-element uniform circular microphone array.

4. The far-field multiple sound source rapid positioning method based on dual-stage search MVDR according to claim 3 is characterized in that: In step 1, the sound source identification and positioning system is used to obtain mono audio data, and the short-time Fourier transform is used to low-pass filter the mono audio data, as follows: Step 1.1, using the sound source recognition and positioning system to obtain mono audio data; Step 1.2: Use short-time Fourier transform to low-pass filter the mono audio data. The formula is: Where x(t) is the original signal, ω is the sound source frequency, and w(t-τ) is the window function centered at τ. Since the range of non-zero values ​​on the time axis is limited, the actual integration is performed in the non-zero region of the window function, so the window function is much smaller than the entire time axis. Step 1.3: After STFT, obtain the sound source frequency range (f l ,f h ), f l ,f h are the minimum and maximum values ​​of the sound source frequency respectively.

5. The far-field multiple sound source rapid positioning method based on dual-stage search MVDR according to claim 1 is characterized in that: The method is used to implement a sound source identification and positioning system suitable for a low-computing power master control, and the method includes a sound source identification module and a sound source positioning module, wherein: The sound source recognition module uses static MFCC features, first-order difference MFCC features, and zero-crossing features as fusion features to extract features from the received sound signal, inputs the features into a pre-trained BP classification matrix, outputs the sound source category, and selects each sound source frequency for the positioning module according to the preset frequency intervals of each category; The sound source localization module obtains the covariance matrix of the received signal of the microphone array, runs the two-stage search MVDR algorithm based on the sound source frequencies output by the identification module, calculates the popular matrix under different pitch angles and azimuth angles, calculates the MVDR power, and finally obtains the accurate azimuth and pitch angle of the sound source.

Images