Mobile microphone array sound source localization method based on spectrum correction theory

Abstract

The invention discloses a mobile microphone array sound source positioning method based on a frequency spectrum correction theory. The method comprises the following steps: driving an N array element microphone array to move, presetting M sampling positions, monitoring the real-time position of the array, and establishing a sound field model; when the array moves to a preset sampling position and pauses to be stable, recording the sampling starting time, and performing synchronous sampling to obtain a sound source signal; sampling at all preset positions is completed, M * N sampling signals are obtained, discrete spectrum analysis and correction are carried out, and frequency domain data and correction frequency are obtained; taking the data of a certain sampling position as a reference, and performing phase compensation on the data of other positions by using the correction frequency to obtain a virtual microphone array; and extracting an azimuth angle and a pitch angle corresponding to a peak value by using a beam forming algorithm, namely, a sound source orientation. According to the method, the microphone array is driven to move to realize the dense microphone array, the sound source positioning cost is reduced, and the positioning accuracy is improved by introducing a frequency spectrum correction theory.

Description

Technical Field

[0001] The present invention relates to the technical field of sound source localization, and in particular to a method for sound source localization of a mobile microphone array based on the theory of spectrum correction. Background Art

[0002] The beamforming sound source localization method uses a microphone array to measure the sound pressure signal, and obtains a sound imaging diagram based on a specific beamforming method to visually indicate the sound source position, also known as an "acoustic camera". It has the advantages of high speed, being suitable for measuring medium and long-distance sound sources, and being easy to arrange, and is widely used in the field of noise source identification. The conventional beamforming algorithm collects the sound pressure signal through the microphone array and weights it to make the expected maximum output power of the signal point to the sound source position. Due to its high robustness, it is still widely used in current sound source localization. However, the resolution of the conventional beamforming algorithm depends on the array size and the number of microphones. In order to improve the resolution, it is necessary to increase the array aperture and the number of microphones, which increases the cost. Therefore, how to obtain an array with a large aperture and many microphones on the premise of only using a small number of microphones has become a research hotspot.

[0003] Currently, the method of asynchronous measurement is mainly used to solve the above problems. This method collects data by moving the microphone array, and then processes the data collected at different times to the same time, so as to use the beamforming technology to realize sound source localization. According to different data processing means, it can be divided into three methods. The first method is to sequentially move the microphone array in the measurement plane to scan and measure the sound source, and use the Bayesian probability method and the expectation maximization algorithm to reconstruct the sound field to obtain the sound source position, which has later developed into the problem of completing the cross-spectral matrix with missing data. However, this method has requirements for the position of the microphone array, and the calculation of the method for completing the cross-spectral matrix is too complex. The second method is to set a fixed reference microphone to synchronously sample with the microphone array with a changing position, and perform simple calculations on the data of the two to eliminate the phase difference between the data received by the microphone array at different times, and apply the beamforming algorithm to obtain the sound source position. However, this method requires setting a reference microphone and the measurement process takes a long time; the third method is to drive the microphone array to move uniformly, including two ways of translation and rotation, to form an array with a large aperture and many microphones. The whole process takes a short time and requires fewer microphones, so it has great research significance. This method is called the moving microphone array beamforming method.

[0004] The beamforming method for a moving microphone array is currently divided into two calculation methods: time-domain beamforming and frequency-domain beamforming. The time-domain beamforming method directly performs beamforming calculations in the time domain using the data collected when the microphone moves continuously, and obtains the sound source position. However, this method has a long calculation time and cannot distinguish sound sources of different frequencies. Therefore, the positioning performance is greatly affected by interfering sound sources of non-target frequencies, and the positioning accuracy is not high in actual working conditions. The frequency-domain beamforming method converts the time-domain data collected when the microphone moves continuously to the frequency domain through short-time spectrum analysis technology, and realizes the beamforming calculation in the frequency domain. The calculation is simple, time-consuming is short, and it can accurately distinguish sound sources of different frequencies. Therefore, it has more practical application significance. However, the establishment of this method is based on an assumption: within a short period of time involved in the time-domain data used for discrete spectrum analysis, the Doppler effect caused by the movement of the microphone is ignored. However, since the microphone is always moving and there is a relative velocity between it and the stationary sound source, there will be a Doppler frequency shift between the frequency of the signal received by the microphone and the true frequency of the sound source. In addition, there is also a certain error between the sound source frequency obtained by discrete spectrum analysis and the true frequency of the sound source. These two frequency errors will both cause an error in the compensation phase of the beamforming method, ultimately affecting the sound source positioning result. Summary of the Invention

[0005] In order to at least solve the problems existing in the prior art, the present invention provides a sound source positioning method for a moving microphone array based on the spectrum correction theory, which can avoid the frequency error caused by the Doppler effect, avoid the frequency error brought by the discrete spectrum analysis process, and finally obtain an accurate sound source frequency for phase compensation, improving the positioning accuracy of the original beamforming method for the moving microphone array, with simple calculation and low cost.

[0006] In order to achieve the object of the present invention, a sound source positioning method for a moving microphone array based on the spectrum correction theory provided by the present invention includes the following steps:

[0007] S1. Drive the microphone array with N elements to move, preset M sampling positions, monitor the real-time position of the microphone array, and establish a sound field model;

[0008] S2. When the microphone array moves to the preset sampling position, pause the movement and wait for stabilization, record the start sampling time, and perform synchronous sampling to obtain the sound source signal;

[0009] S3. Complete the sampling at all preset sampling positions, obtain M×N sampling signals, perform discrete spectrum analysis on the M×N sampling signals to obtain the frequency-domain data of the sampling signals, and perform discrete spectrum correction to obtain the corrected frequency;

[0010] S4. Using the frequency-domain data at a preset sampling position as a reference, perform phase compensation on the other frequency-domain data using the calibration frequency to obtain a virtual microphone array with M×N array elements;

[0011] S5. Extract the azimuth angle and elevation angle corresponding to the peak of the beamforming output to achieve the azimuth of the sound source.

[0012] The present invention avoids the influence of the Doppler effect and the frequency error brought by the spectrum analysis process, and improves the positioning accuracy of the mobile microphone array beamforming method.

[0013] Further, in S1, select the nth microphone on the microphone array with N array elements as the reference microphone, and use a position sensor to monitor the position of the reference microphone in real time. The positions of the other microphones can be calculated through their position relationships with the reference microphone to facilitate real-time monitoring of the position of the microphone array; on the movement path of the reference microphone, preset M sampling positions.

[0014] The established sound field model is a far-field signal model. Take a fixed point on the moving plane of the microphone array as the coordinate origin, and the sound source is incident on the microphone array in the form of a plane wave from the far field; and take a fixed point on the moving plane of the microphone array as the coordinate origin.

[0015] Further, in S2, when the microphone array with N array elements performs static synchronous sampling at the preset sampling positions, there is a time delay between the signals received by the array elements;

[0016] By recording the start sampling time, the time difference between the signals collected by the microphone array at different preset positions can be obtained.

[0017] Further, in S3, the microphone with N array elements samples at M preset sampling positions to obtain M×N sampling signals; perform discrete spectrum analysis on these signals to obtain their frequency-domain data; perform discrete spectrum calibration on these signals to obtain the calibration frequency, and use the average value of the calibration frequencies as the finally used calibration frequency;

[0018] Further, in S4, using the first preset sampling position as a reference, perform phase compensation on the signals sampled at the other positions, and process all the sampling signals to the same moment, so as to obtain a virtual microphone array.

[0019] Further, in S5, apply the beamforming algorithm to the obtained virtual microphone array to obtain the output of beamforming. By scanning the elevation angle and azimuth angle in space, the elevation angle and azimuth angle corresponding to the peak of the beam output are the azimuth of the sound source.

[0020] Compared with the prior art, the method for localizing a sound source using a moving microphone array based on the spectrum correction theory of the present invention has at least the following advantages:

[0021] (1) The present invention uses a method of asynchronous measurement by moving a microphone array to collect data, and finally obtains a microphone array with a larger aperture and denser microphones, which greatly reduces the cost of sound source localization while improving the performance of the beamforming algorithm; compared with other asynchronous measurement methods, the measurement process is easier and the calculation is simpler.

[0022] (2) The present invention uses the method of stationary and synchronous sampling of the microphone array at preset sampling positions to avoid the influence of the Doppler effect.

[0023] (3) The present invention applies the spectrum correction theory to obtain a more accurate sound source frequency, improves the phase compensation accuracy of the original moving microphone array beamforming method, and finally improves the localization accuracy. Description of the Drawings

[0024] Figure 1 is a flowchart of a method for localizing a sound source using a moving microphone array based on the spectrum correction theory provided by an embodiment of the present invention.

[0025] Figure 2 is a schematic diagram of the sound field model established in an embodiment of the present invention.

[0026] Figure 3 is a logarithmic contour map of the sound imaging space of the original method for localizing a sound source using a moving microphone array provided by an embodiment of the present invention.

[0027] Figure 4 is a logarithmic contour map of the sound imaging space of a method for localizing a sound source using a moving microphone array based on the spectrum correction theory provided by an embodiment of the present invention. Detailed Embodiments

[0028] The following will describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. The described preferred embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Without conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0029] A method for localizing a sound source using a moving microphone array beamforming based on the spectrum correction theory provided by the present invention, the flowchart of the method is as Figure 1 shown, and the specific steps are as follows:

[0030] Step S1: Drive a microphone array with N elements to move, preset M sampling positions, and use a position sensor to monitor the real-time position of the microphone array to establish a sound field model.

[0031] In some embodiments of the present invention, step S1 includes the following sub-steps:

[0032] Step 1.1: Select the nth microphone on the microphone array of N elements as the reference microphone, and use a position sensor to monitor the position of the reference microphone in real time. The positions of the remaining microphones can be calculated through their position relationships with the reference microphone, so as to facilitate monitoring the real-time position of the microphone array.

[0033] Step 1.2: Preset M sampling positions on the movement path of the reference microphone.

[0034] Step 1.3: Select the far-field signal model as the sound field model, and use a fixed point on the moving plane of the microphone array as the coordinate origin. The sound source is incident on the microphone array in the form of a plane wave with a direction of Ω s =(θ s , Φ s ), where Ω s represents the incident direction of the sound source, θ s is the elevation angle of the sound source, and Φ s is its azimuth angle.

[0035] In some embodiments of the present invention, select a moving mode of circular motion around any point in space at ω rad / s, and use the rotation center as the coordinate origin. The real-time position information of the reference microphone is described by the angle α n , α n is the angle experienced by the reference microphone starting from the preset first sampling position. Then the coordinate information of the reference microphone is (x n , y n )=(r n cosα n , r n sinα n ), r n is the distance from the nth microphone to the rotation center. Using the movement of the reference microphone to represent the movement of the microphone array of N elements, the obtained sound field model is as shown in Figure 2 . In the figure, the black solid circle is the current position of the reference microphone, and the hollow circle is the first preset sampling position.

[0036] Step S2: When the microphone array moves to the preset sampling position, pause the movement and wait until it stabilizes, record the start time of sampling, and perform synchronous sampling to obtain the sound source signal.

[0037] In this step, assume that when the microphone array performs synchronous sampling at the preset first sampling position, the start time of sampling is t _1 , and take this as the 0 moment. The sound source signal received by the rotation center is s(t);

[0038] When the microphone array performs synchronous sampling at the preset m-th sampling position, the starting sampling time is t _m , corresponding to the moment of t m , then t m can be expressed as:

[0039] t m = t _m - t _1

[0040] At this time, the sound source signal received by the rotation center is s(t + t m );

[0041] The time delay τ nm of the n-th microphone relative to the rotation center at the preset m-th sampling position is:

[0042] τ nm = (x nm sinθ s cosΦ s + y nm sinθ s sinφ s ) / c

[0043] Where: (x nm , y nm ) = (r n cosα nm , r n sinα nm ), α nm is the angle experienced by the reference microphone from the preset first sampling position to the preset m-th sampling position, c is the speed of sound, x nm , y nm are the abscissa and ordinate of the n-th microphone in the sound field model at the preset m-th sampling position.

[0044] Then the sound source signal x nm (t) received by the n-th microphone at the preset m-th sampling position can be expressed as:

[0045] x nm (t) = s(t + t m + τ nm )

[0046] Step S3, complete the sampling at all preset sampling positions, obtain M×N sampling signals, perform discrete spectrum analysis on the M×N sampling signals to obtain the frequency domain data of the sampling signals, and perform discrete spectrum correction to obtain the corrected frequency.

[0047] In this step, the microphone array with N elements samples at M preset sampling positions to obtain M×N sampling signals. Discrete spectrum analysis is performed on all M×N sampling signals, and the true sound source frequency f is selected. 0 The spectral line corresponding to the k-th one is obtained, and the frequency-domain data of all sampling signals is obtained. Among them, the frequency-domain data X nm (f 0 ) is expressed as:

[0048]

[0049] Among them, A knm is the amplitude of the frequency-domain data, is the phase of the frequency-domain data, represents the imaginary unit.

[0050] The ratio correction method is used to perform discrete spectrum correction on the M×N sampling signals to obtain M×N corrected frequencies. The corrected frequency obtained by performing discrete spectrum correction on the signal received by the n-th microphone at the m-th preset sampling position is f SCnm , then the finally obtained corrected frequency can be expressed as:

[0051]

[0052] Step S4: Using the frequency-domain data at a certain preset sampling position as a reference, phase compensation is performed on other frequency-domain data using the corrected frequencies to obtain a virtual microphone array with M×N elements.

[0053] In this step, there is a phase difference caused by the sampling time difference between the signals collected by the microphone array at different preset sampling positions. Using the data collected by the microphone array at the first preset sampling position as a reference, phase compensation is performed on the signals collected by the microphone array at other sampling positions to process all the data to the same moment, and a virtual microphone array with M×N elements is obtained.

[0054] The compensation phase for the signal received by the n-th microphone at the m-th preset sampling position is:

[0055]

[0056] The compensated signal is:

[0057]

[0058] Then X′ nm (f 0 ) is used as the signal received by the microphone at the m-th preset sampling position of the n-th microphone on the virtual microphone array.

[0059] Step S5: Apply the beamforming algorithm to extract the azimuth and elevation angles corresponding to the peak, and achieve the localization of the sound source.

[0060] In the beamforming algorithm, in order to align the signals received by all microphones on the virtual microphone array, it is necessary to perform phase compensation on the received signals. The phase compensation is based on the assumed sound source direction Ω = (θ, Φ). The compensated signal for the signal received by the nm-th microphone can be expressed as:

[0061]

[0062] where τ′ nm is calculated according to the assumed sound source direction Ω = (θ, Φ), and is the compensation time delay of the nm-th microphone on the virtual microphone array relative to the rotation center. θ is the elevation angle of the assumed sound source direction, and Φ is the azimuth angle of the assumed sound source direction;

[0063] The output of beamforming is obtained by summing all the compensated signals, and can be expressed as:

[0064]

[0065] where B(Ω s , Ω) is the output of beamforming.

[0066] By scanning the assumed sound source direction Ω = (θ, φ) in space, the direction corresponding to the peak of the beamforming output is the direction of the sound source. Among them, in the scanned sound source directions in space, the elevation angle θ ∈ [0°, 90°], and the azimuth angle φ ∈ [0°, 360°].

[0067] In some embodiments of the present invention, Figure 3 shown is the output of the original moving microphone array sound source localization method, Figure 4 shown is the output of the moving microphone array sound source localization method based on the spectrum correction theory of the present invention. Among them, the abscissa is the azimuth angle, the ordinate is the elevation angle, and the black circle represents the real sound source position. It can be seen from the figure that the energy output by the original moving microphone array sound source localization method is not concentrated near the real sound source and cannot provide the correct sound source direction information. While the energy output by the moving microphone array sound source localization method based on the spectrum correction theory of the present invention is concentrated near the real sound source, so it can provide the correct sound source direction information. The peak of the beam output of this method points to the sound source direction, achieving the accurate localization of the sound source.

[0068] The sound source localization method provided by the foregoing embodiments of the present invention uses an asynchronous measurement method to drive the movement of the microphone array, realizing a denser and larger microphone array, improving the resolution of the beamforming method while greatly reducing the cost of sound source localization; compared with other asynchronous measurement methods, the calculation is simple, the steps are simple, and the time consumption is short; the introduced spectrum correction theory greatly enhances the localization accuracy of the original moving microphone array beamforming method.

[0069] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A mobile microphone array sound source localization method based on spectrum correction theory, characterized in that: The following steps are involved: S1, drive the microphone array of N array elements to move, preset M sampling positions, monitor the real-time position of the microphone array, and establish a sound field model; S2, when the microphone array moves to the preset sampling position, it pauses to stabilize, records the time when sampling starts, and performs synchronous sampling to obtain the sound source signal; S3, completing sampling at all preset sampling positions to obtain M×N sampling signals, performing discrete spectrum analysis on the M×N sampling signals to obtain frequency domain data of the sampling signals, and performing discrete spectrum correction to obtain correction frequencies; S4, using the frequency domain data of a preset sampling position as a reference, using the correction frequency to perform phase compensation on other frequency domain data, and obtaining a virtual microphone array with M×N array elements; S5. Extract the azimuth and elevation angles corresponding to the peak values ​​of the beamforming output to achieve the orientation of the sound source.

2. The method for mobile microphone array sound source localization based on spectrum correction theory according to claim 1, characterized in that: The step S1 of driving the microphone array of N array elements to move, presetting M sampling positions, monitoring the real-time position of the microphone array, and establishing a sound field model specifically includes: In a microphone array of N elements, any one microphone is selected as a reference microphone, a position sensor is used to monitor the position of the reference microphone, and positions of other microphones are obtained based on the position of the reference microphone; On the motion path of the reference microphone, M sampling positions are pre-set; The far-field signal model is selected, and a fixed point on the moving plane of the microphone array is used as the coordinate origin. The sound source is Ω s =(θ s ,Φ s ) direction and the plane wave is incident on the microphone array, where Ω s represents the incident direction of the sound source, θ s is the pitch angle of the sound source, Φ s is its azimuth.

3. The method for sound source localization using a mobile microphone array based on spectrum correction theory according to claim 1, characterized in that: In step S2, when the microphone array of N elements is stationary and synchronously sampling at the preset sampling positions, there is a time delay between the signals received by the elements; through the recorded sampling start time, the time difference between the microphone array collecting signals at different preset sampling positions can be obtained.

4. The method for sound source localization using a mobile microphone array based on spectrum correction theory according to claim 1, characterized in that: In step S2, when the microphone array moves to the preset sampling position, the movement is paused to be stable, the time of starting sampling is recorded, and synchronous sampling is performed to obtain the sound source signal, which specifically includes: When the reference microphone moves to the preset sampling position, the entire microphone array stops moving. After the state is stable, the microphone array synchronously samples the sound source and records the time when sampling starts. When the microphone array samples synchronously at the preset first position, the sampling time is t _1 , and this time is taken as time 0, the sound source signal received by a fixed point on the moving plane of the microphone array is s(t), and the fixed point is taken as the coordinate origin; When the microphone array performs synchronous sampling at the preset mth sampling position, the sampling starts at t _m , corresponding to t m At time t m Expressed as: t m =t _m -t _1 At this time, the sound source signal received by the fixed point is s(t+t m ); The delay of the nth microphone relative to the fixed point is: t nm =(x nm sinth s cosΦ s +y nm sinth s sinΦ s ) / c Then the sound source signal received by the nth microphone at the preset mth sampling position is expressed as: x nm (t)=s(t+t m +τ nm ) Among them, (x n ,y n ) is the coordinate of the nth microphone in the microphone array on the moving plane, c is the speed of sound, θ s is the pitch angle of the sound source, Φ s is its azimuth.

5. The method for sound source localization using a mobile microphone array based on spectrum correction theory according to claim 1, characterized in that: The step S3 described in which sampling is completed at all preset sampling positions to obtain M×N sampling signals, discrete spectrum analysis is performed on the M×N sampling signals to obtain sampling signal frequency domain data, and discrete spectrum correction is performed to obtain correction frequency, specifically including: The microphone array of N array elements performs sampling at M preset sampling positions to obtain M×N sampling signals; Discrete spectrum analysis is performed on all M×N sampling signals, and the kth spectrum line corresponding to the real sound source frequency f0 is selected to obtain the frequency domain data of all sampling signals. The frequency domain data of the signal received by the nth microphone at the preset mth sampling position is expressed as: Among them, A knm is the amplitude of the frequency domain data, is the phase of the frequency domain data, j is the imaginary unit; Discrete spectrum correction is performed on M×N sampling signals to obtain M×N correction frequencies. The correction frequency obtained by performing discrete spectrum correction on the signal received by the nth microphone at the preset mth sampling position is f SCnm , then the final correction frequency used is Expressed as:

6. The method for mobile microphone array sound source localization based on spectrum correction theory according to claim 1, characterized in that: The method described in step S4 uses the frequency domain data of a preset sampling position as a reference, uses the correction frequency to perform phase compensation on other frequency domain data, and obtains a virtual microphone array with M×N array elements, specifically including: There is a phase difference caused by the sampling time difference between the signals collected by the microphone array at different preset sampling positions. The data collected by the microphone array at the preset first sampling position is used as a reference, and phase compensation is performed on the signals collected at other sampling positions. All data are processed at the same time to obtain a virtual microphone array with M×N array elements.

7. According to the method for sound source localization of a mobile microphone array based on spectrum correction theory in claim 1, the compensation phase of the signal received by the nth microphone at the preset mth position is for: The compensated signal X′ nm (f0) is: In the formula, is the correction frequency, t m is the starting sampling time when the microphone array performs synchronous sampling at the preset mth sampling position, A knm is the amplitude of the frequency domain data, is the phase of the frequency domain data, j represents the imaginary unit, is the compensation phase of the signal received by the nth microphone at the preset mth sampling position, and f0 is the real sound source frequency; X′ nm (f0) is the signal received by the nth microphone at the preset mth sampling position on the virtual microphone array.

8. A method for sound source localization using a mobile microphone array based on spectrum correction theory according to any one of claims 1 to 7, characterized in that: In step S5, a beamforming algorithm is used to extract the azimuth angle and the elevation angle.

9. The method for mobile microphone array sound source localization based on spectrum correction theory according to claim 8, characterized in that: In step S5, the step of obtaining the beamforming output includes: In the beamforming algorithm, in order to align the signals received by all microphones on the virtual microphone array, it is necessary to perform phase compensation on the received signals. The phase compensation is performed based on the assumed sound source direction Ω = (θ, Φ). The signal received by the nmth microphone is compensated. The compensated signal is expressed as: Among them, τ′ nm It is the compensation time delay of the nm-th microphone on the virtual microphone array relative to the fixed point, calculated based on the assumed sound source direction Ω = (θ, Φ); The beamformed output is obtained by adding all compensated signals.

10. The method for sound source localization using a mobile microphone array based on spectrum correction theory according to claim 9, characterized in that: The output of beamforming is expressed as: By scanning the assumed sound source direction Ω=(θ,Φ) in space, the direction corresponding to the peak of the beamforming output is the direction of the sound source.

Images