Sound source positioning method and system based on four microphones

Abstract

The invention provides a sound source positioning method and system based on four microphones, and belongs to the technical field of sound test.The method specifically comprises the steps that 1, the four microphones are placed in a three-dimensional right-angle mode, the microphone placed at the original point is marked as the microphone A, and the other microphones are marked as the microphones B, C and D; step 2, collecting sounds emitted by the four microphones; step 3, converting the data stored in the text document into an array in matlab, and then preprocessing the converted data; 4, extracting a first envelope sound signal of each channel based on the preprocessed data; step 5, taking the intercepted sound signal of the microphone A as a reference signal, and respectively calculating the time differences of sound signal receiving of the microphone A and the microphones B, C and D; and step 6, positioning the sound source based on the time difference calculated in the step 5. According to the invention, the sound source positioning precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of sound testing, and particularly relates to a sound source localization method and system based on four microphones. Background Art

[0002] With the development of technologies such as human-computer interaction and intelligent control, intelligent robots, intelligent driving, etc. have gradually entered people's lives. In these applications, accurate sound localization technology is particularly important. In bionic engineering and its applications, simulating the ability of the human two ears to distinguish tiny sounds and judge the orientation of objects, and developing a low-cost, high-speed, and high-precision sound localization system is of great significance for the research and application of bionic robots. Compared with the device for positioning based on optical signals, the sound localization device has its unique advantages. Since the speed of sound is much slower than the speed of light, the requirements for the timing system of the sound localization device can be greatly reduced. The positioning work can be completed without relying on external infrastructure, reducing the early-stage system investment. Moreover, sound localization has advantages that cannot be replaced by optical methods in the environment of light pollution. Sound localization has a wide range of applications and has application prospects in many fields such as determining the orientation of a speaker, measuring the positions of surrounding objects in the field of automotive autonomous driving, and determining the positions of vehicles with illegal horn honking in the urban noise monitoring system.

[0003] Sound source localization based on a microphone array is the mainstream method of current sound localization, which means using a certain number of microphones to collect sound signals, and then using relevant algorithms to analyze and process them to determine the position of the sound source. Using an array for sound source localization involves knowledge in multiple fields, including array signal processing, speech processing, compressive sensing, and artificial intelligence. Microphone array sound source localization mainly studies the azimuth and distance where the sound source is located, that is, azimuth estimation and distance estimation, so as to determine the unique position of the sound source in space.

[0004] Currently, the following three mainstream microphone array sound source localization methods have been developed. One is the controllable beamforming localization method, the second is the high-resolution spectrum estimation localization method, and the third is the localization method based on time delay estimation.

[0005] The controllable beamforming method needs to know in advance the statistical characteristics of the sound source, noise, and reverberation, and uses maximum likelihood estimation to judge the sound source information. Since these characteristics are often also obtained by estimation, it is easy to cause relatively large errors. At the same time, affected by the initial value, the objective function may have more than one maximum value, which will make the calculated result a local optimal solution and cannot meet the requirements of the global optimum. If a global optimal solution is to be obtained, the time complexity of the algorithm needs to be greatly increased. To sum up, this method needs to know a large amount of information about the sound source signal and it is difficult to obtain a result that is satisfactory in both accuracy and time complexity.

[0006] The high-resolution spectral estimation localization method records the correlation matrix of the same sound source signal by different microphones, and obtains the position of the sound signal by solving the matrix. The problem of this method is that it is mainly applicable to the case where the sound source signal is a stationary signal. However, most of the sound signals in life do not meet the conditions and are difficult to estimate.

[0007] The localization method based on time delay estimation has the advantages of low waveform requirements, small computational amount, low computational complexity, and high accuracy. Its basic principle is that due to the different positions of the microphones, the arrival time of the sound source signal will also be different. Using this time difference and the geometric relationship between the microphones, the position of the sound source can be determined. At present, the microphone array localization technology based on time delay is widely used, but this method has high requirements for the accuracy of time delay estimation. In environments such as noise and reverberation, this method has poor adaptability, resulting in large measurement errors. In addition, the computational amount for calculating the entire signal is large and the time consumption is long, making it difficult to achieve real-time measurement. Summary of the Invention

[0008] Object of the Invention: In order to solve the problems existing in the above-mentioned prior art, the present invention provides a sound source localization method and system based on four microphones.

[0009] Technical Solution: The present invention provides a sound source localization method based on four microphones, which specifically includes the following steps:

[0010] Step 1: Use four microphones and place the four microphones in a three-dimensional right angle to form a triangular pyramid shape. Specifically: select one microphone and place it at the origin of the three-dimensional coordinate system, denote this microphone as A, denote the microphone placed on the X-axis of the three-dimensional coordinate system as B, denote the microphone placed on the Z-axis of the three-dimensional coordinate system as D, and denote the microphone placed on the Y-axis of the three-dimensional coordinate system as C; the distances from microphones B, C, and D to microphone A are all d;

[0011] Step 2: Collect the sounds emitted by the four microphones and save them in the form of a text document;

[0012] Step 3: Convert the data saved in the text document into an array in Matlab, and then preprocess the converted data;

[0013] Step 4: Based on the preprocessed data, extract the first envelope sound signal of each channel;

[0014] Step 5: Take the sound signal of microphone A as the reference signal, and calculate the time differences between microphone A and microphones B, C, and D receiving the sound signals respectively;

[0015] Step 6: Locate the sound source based on the time differences calculated in Step 5.

[0016] Further, in step 2, the virtual oscilloscope 6000EU is used to simultaneously collect the sound signals of four microphones.

[0017] Further, the specific process of extracting the first envelope sound signal of each channel is as follows:

[0018] Step 4.1: For the sound signal of any one channel, use the findpeaks function to find all the maximum extreme points of the sound signal of this channel;

[0019] Step 4.2: Take the first maximum extreme point as the base point M, and judge whether M is greater than or equal to 2N0. If not, go to step 4.3; if so, intercept the sound signal in the time period of [M - N0, M + N0]. This signal is the extracted first envelope signal; N0 is a reference parameter, N0 = d×Fs / v, where Fs is the sampling frequency and v is the speed of sound;

[0020] Step 4.3: Zero the sound signal in the range of [M, 5N0], and then go to step 4.1 to find the base point M again.

[0021] Further, in step 5, the calculation method of the generalized cross-correlation function is adopted. Taking the sound signal of microphone A as the reference signal, the curves of the cross-correlation functions of microphones B, C, and D with microphone A are obtained, and the peaks on the curves are used as the time differences of the received sound signals between the corresponding microphones and microphone A.

[0022] Further, step 6 is specifically as follows:

[0023] Step 6.1: Establish the following equation:

[0024] ;

[0025] Among them, represents the time required for sound to travel from the sound source to microphone A, , and are all coefficients, , , The expressions of

[0026] ;

[0027] Among them, represents the time difference of the received sound signal between microphone B and microphone A, represents the time difference of the received sound signal between microphone C and microphone A, represents the time difference of the received sound signal between microphone D and microphone A; v represents the speed of sound;

[0028] Step 6.2: Set the following constraint conditions:

[0029] ;

[0030] Step 6.3: Calculate the two roots of the equation in Step 6.1, and select the root greater than zero and denote it as , based on , obtain the expression of the coordinate of the sound source as follows:

[0031] .

[0032] A system for a sound source localization method based on four microphones, comprising:

[0033] A sound collection module, configured to collect the sounds emitted by four set microphones;

[0034] A data conversion module, configured to convert the data into an array in Matlab;

[0035] A preprocessing module, configured to preprocess the data in the array in Matlab;

[0036] A first envelope sound signal extraction module, configured to extract the first envelope sound signal of each channel;

[0037] A time difference calculation module, configured to calculate the time differences between the sounds received by microphone A and microphones B, C, and D;

[0038] A sound source localization module, configured to calculate the sound source position according to the time differences.

[0039] Beneficial effects: The present invention uses four microphones placed in a triangular pyramid to collect sound signals. This microphone placement method can solve a unique value, eliminate the influence of false roots of the equation on time calculation. At the same time, compared with other microphone placement methods, this method has a small amount of calculation, and can effectively collect three-dimensional information of the sound, and can detect the spatial position of the sound source without dead angles at a large angle. At the same time, the present invention uses the method of finding the extreme value to intercept the sound signal in the interval to reduce the amount of calculation. The selection of N0 is restricted by the microphone coordinate d. The selection of this N0 value can ensure that the intercepted sound is not too short to capture the required sound, and at the same time is not too long to increase the amount of calculation, which is determined by the four microphones placed in a triangular pyramid in the present invention. The distances from the other three microphones to the central microphone are all d. To ensure that the same extreme value points of different sounds can be captured in the time difference measurement and there is a certain redundancy, considering the extreme situation, the intercepted time must be at least greater than twice of it. Therefore, the method of intercepting the sound in the present invention ensures the correctness and rapidity of the four-microphone algorithm for calculating the spatial position, and the two are matched and indispensable.

[0040] The present invention realizes real-time and automatic detection of the sound source position, achieving low-cost and high-precision measurement. Four microphones are placed in a three-dimensional right-angled manner to form a triangular pyramid. The four microphones receive sound signals and process the four collected sound signals. By finding the extreme value, the first extreme value point of the data is found and used as the base point. Using the method of intercepting with a fixed value range before and after, the first sound wave envelope information of the sound source signal is extracted, effectively removing the influence of reverberation. The four microphone signals extracted are more accurate, and the calculated time difference is more precise, so the sound source positioning is more accurate. The Labview program is used to display the three-dimensional image, and the sound source position can be visually seen from the image. Brief Description of the Drawings

[0041] Figure 1 It is a diagram of the microphone placement of the present invention.

[0042] Figure 2 It is a flowchart of the method of the present invention.

[0043] Figure 3 It is a generalized cross-correlation function diagram of the present invention.

[0044] Figure 4 It is a schematic diagram of the device of the present invention. Detailed Description of the Invention

[0045] The accompanying drawings forming a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0046] In an open and homogeneous medium space, sound propagates in a straight line at a fixed sound speed. Based on this principle, a classical time-delay estimation algorithm can be used to list the relationship between the distance from the sound source to the microphone and the propagation time.

[0047] As Figure 1 shown, in a three-dimensional XYZ coordinate system, microphone A is located at the origin of coordinates, and the three microphones B, C, and D are located at points B, C, and D respectively. The pentagram point O is an arbitrary point in space and serves as the sound source, and the coordinates of the sound source are (x, y, z). The distances from the three microphones B, C, and D to the central microphone A are all d, the sound speed is v, and the times when microphones A, B, C, and D collect sound signals are respectively 、 、 、 . Then the following equations can be listed:

[0048]

[0049] Actually, since it is very difficult to control the generation of the sound signal and the timing software to proceed simultaneously, directly obtaining 、 、 , is very difficult. Instead, the time difference between different microphones is more often calculated. Taking the microphone at the origin as a reference, the system of equations can then be changed to:

[0050]

[0051] represents the time required for sound to travel from the sound source to microphone A. Among these physical quantities, x, y, z, are unknowns, and d is a constant determined during the setup of the device. is the time difference between microphone B and microphone A. is the time difference between microphone C and microphone A. is the time difference between microphone D and microphone A. , , are all obtained through data analysis. From the above equations, we can get:

[0052]

[0053] Let be an unknown, and use to represent substituting x, y, z into the formula . Then the following equation can be obtained:

[0054]

[0055] By organizing the coefficients, the following quadratic equation is obtained:

[0056]

[0057] where:

[0058]

[0059] According to the analysis of geometric properties, from the relationship between the three sides of a triangle, we get:

[0060]

[0061] Therefore, , always holds, and it also satisfies when the sound source is far from the device (outside the device) . Therefore, let the two roots of the quadratic equation be , , satisfying:

[0062] , .

[0063] Since time is a positive number, we can conclude that it must be a positive root. After that, we can calculate x, y, and z:

[0064]

[0065] Based on the expression of x, y, z, , , Precise measurements are required.

[0066] In the actual environment, the walls, ceilings, and floors of the room can reflect sound waves. There are also tables, chairs, and other debris in the room, which makes reverberation very easy to occur. Reverberation corresponds to other routes, so different sound path differences will also be calculated. Since the line segment between two points is the shortest distance, the sound signal propagating along the straight line between the two points must reach the microphone first compared to the reverberation traveling along the broken line. Therefore, it is necessary to find the first wave packet on the waveform graph so that the sound signal can be found more accurately and the impact of reverberation can be reduced.

[0067] like Figure 2 As shown, the specific process is as follows:

[0068] Acquisition of sound signals: Using the virtual oscilloscope 6000EU, it is possible to simultaneously acquire sound signals from four microphones. The software that comes with the oscilloscope also supports converting the received analog signals into digital signals and saving them in the form of text documents.

[0069] Processing of sound signals and noise: The processing of sound signals includes reading sound files, normalization, low-pass filtering, extracting the first envelope of each channel of the sound signal, calculating time difference, calculating the sound source position, and visual display.

[0070] Data reading and normalization: First, translate the data saved in the text document into an array in MATLAB. Then, convert the data from the special format when reading it into an array format that can be calculated, and restore the original voice collected. Next, the data needs to be normalized, setting the maximum value to 1 and returning the sound signal less than 5% to zero.

[0071] Filtering (denoising): For sound information, first perform Fourier transform, and then use low-pass filtering to filter out background noise based on the characteristics of the sound signal.

[0072] First envelope sound signal extraction: Since the calculation amount of the generalized cross-correlation algorithm for the entire signal is large, in order to improve the operation rate and realize real-time measurement, it is necessary to select the first small segment of the effective sound signal from the lengthy original data for calculation. At the same time, since the sound signal that first reaches the microphone must be a signal that propagates in a straight line in space, intercepting the sound signal within a small range at the front of the effective signal is also helpful to remove the impact of echo and reverberation on the data.

[0073] Taking the origin microphone as the reference, use the findpeaks function to find the maximum extreme value of the normalized and filtered data set, take the first extreme value as the base point, and set the data point corresponding to the base point in the horizontal coordinate of the array in matlab to M. The horizontal coordinate M needs to meet the condition: M ≥ 2N 0， N0 is a reference parameter. The value of N0 is determined by the following method: within the measurement distance range of 0.1 meters to 10 meters, the sampling rate is set to Fs, N0 is d×Fs / v, Fs is the sampling frequency, and v is the speed of sound. If M<2 N0, the sound signal in the range of [M, 5N0] is reset to zero, and the extreme value of the zeroed sound signal is found again according to the above steps to determine the M value. After the M value is determined, the sound signal in the range of [M-N0, M+N0] is intercepted. This signal is the first envelope signal extracted. Using the same algorithm, data of corresponding lengths are intercepted for the four groups of sound data for subsequent calculations.

[0074] Calculate the time difference using the generalized cross-correlation function: Use the generalized cross-correlation function calculation method to calculate the cross-correlation function of microphones B, C, D and microphone A, taking the intercepted sound signal collected by microphone A as a reference. Figure 3 As shown, the cross-correlation functions of microphones B, C, D and microphone A are R BA , R CA , R DA There are three curves, each with a peak value, and the horizontal axis corresponding to the peak value is the time difference between the microphones. Figure 3 The three time differences are , , .

[0075] Calculate the sound source position: Using the sound source localization equation derived previously and its solution, we can get , , t2, x, y and z can be calculated, and the sound source position (x, y, z) can be obtained, which can be displayed graphically through the software.

[0076] Result visualization and integration: Through the Labview program, three-dimensional images of the microphone array and the sound source point can be drawn, as well as various required function images, such as waveform diagrams, generalized cross-correlation results, etc. Finally, all the above operations are written into a Labview program, and an operation interface for sound source localization can be obtained.

[0077] The device in this embodiment consists of a sound acquisition module, an amplification module, a power supply module, a virtual oscilloscope, a computer, and software, as Figure 4 shown. The sound acquisition module is composed of a pickup microphone head. The amplification module is implemented using an amplification circuit, and the power supply module is used to supply power to the amplification circuit. The signals are respectively connected to the CH1 to CH4 interfaces of the virtual oscilloscope, and the virtual oscilloscope is connected to the computer, and the device is then built.

[0078] To measure the accuracy of the sound localization of this system under different distance and direction conditions, three exploration experiments are set up, namely the determination of the sound position directly in front, the determination of the sound position at 45°, and the determination of the sound position at 30°. For the multiple measurement results, the average value is taken, and the angle θ between the coordinate vector of the average value and the coordinate vector of the true value is calculated. The θ angle is used as the measurement offset angle; and the distance d between the measured coordinate point and the true coordinate point at this position is calculated.

[0079] Experiment 1 is to measure the sound source at different distances directly in front of the device. The three position points are (0.00 cm, 100.00 cm, 100.00 cm), (0.00 cm, 80.00 cm, 80.00 cm), and (0.00 cm, 50.00 cm, 50.00 cm). The actual measurement results of this device at (0.00 cm, 100.00 cm, 100.00 cm) are shown in Table 1 below:

[0080] Table 1

[0081]

[0082] The actual measurement results of this device at (0.00 cm, 80.00 cm, 80.00 cm) are shown in Table 2 below:

[0083] Table 2

[0084]

[0085] The actual measurement results of this device at (0.00 cm, 50.00 cm, 50.00 cm) are shown in Table 3 below:

[0086] Table 3

[0087]

[0088] Experiment 2 was to measure the sound source position at different distances in the 45° direction of the device. The three position points were (50.00 cm, 50.00 cm, 50.00 cm), (120.00 cm, 120.00 cm, 80.00 cm), and (140.00 cm, 140.00 cm, 90.00 cm). The actual measurement results of this device at (50.00 cm, 50.00 cm, 50.00 cm) are shown in Table 4 below:

[0089] Table 4

[0090]

[0091] The actual measurement results of this device at (120.00 cm, 120.00 cm, 80.00 cm) are shown in Table 5 below:

[0092] Table 5

[0093]

[0094] The actual measurement results of this device at (140.00 cm, 140.00 cm, 90.00 cm) are shown in Table 6 below:

[0095] Table 6

[0096]

[0097] Experiment 3 was to measure the sound source position at different distances in the 30° direction of the device. The three position points were (86.00 cm, 50.00 cm, 50.00 cm), (172.00 cm, 120.00 cm, 90.00 cm), and (241.00 cm, 140.00 cm, 100.00 cm). The actual measurement results of this device at (86.00 cm, 50.00 cm, 50.00 cm) are shown in Table 7 below:

[0098] Table 7

[0099]

[0100] The actual measurement results of this device at (172.00 cm, 120.00 cm, 90.00 cm) are shown in Table 8 below:

[0101] Table 8

[0102]

[0103] The actual measurement results of this device at (241.00 cm, 140.00 cm, 100.00 cm) are shown in Table 9 below:

[0104] Table 9

[0105]

[0106] Summarize the relevant calculation results of the previous Experiments 1, 2, and 3 in Table 10 below, and calculate the offset angle, deviation distance, and percentage error of the deviation distance.

[0107] Table 10

[0108]

[0109] As can be seen from the above table, the offset angle measured by this device at different angles and different distances is controlled within 3°, and the percentage error of the deviation distance is controlled within 10%.

[0110] In addition, it should be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without conflict. To avoid unnecessary repetition, the present invention will not separately describe various possible combination methods.

Claims

1. A sound source localization method based on four microphones, characterized in that Specifically, it includes the following steps: Step 1: Use four microphones and place them in a three-dimensional right-angled manner to form a triangular pyramid shape. Specifically: Select one microphone and place it at the origin of the three-dimensional coordinate system, and denote this microphone as A. Denote the microphone placed on the X-axis of the three-dimensional coordinate system as B, the microphone placed on the Z-axis of the three-dimensional coordinate system as D, and the microphone placed on the Y-axis of the three-dimensional coordinate system as C; The distances from microphones B, C, and D to microphone A are all d; Step 2: Collect the sounds emitted by the four microphones and save them in the form of a text document; Step 3: Convert the data saved in the text document into an array in Matlab, and then preprocess the converted data; Step 4: Based on the preprocessed data, extract the first envelope sound signal of each channel; Step 5: Using the sound signal of microphone A as the reference signal, calculate the time differences between the sound signals received by microphone A and microphones B, C, and D respectively; Step 6: Locate the sound source based on the time differences calculated in Step 5.

2. The method for sound source localization based on four microphones according to claim 1, wherein In Step 2, a virtual oscilloscope 6000EU is used to collect the sound signals of the four microphones simultaneously.

3. A method for sound source localization based on four microphones according to claim 1, characterized in that, The preprocessing includes normalization and low-pass filtering.

4. A method for sound source localization based on four microphones according to claim 1, characterized in that, The specific extraction of the first envelope sound signal of each channel is as follows: Step 4.1: For the sound signal of any one channel, use the findpeaks function to find all the maximum extreme points of the sound signal of this channel; Step 4.2: Take the first maximum extreme point as the base point M, and judge whether M is greater than or equal to 2N0. If not, go to Step 4.3; If so, intercept the sound signal in the time period of [M - N0, M + N0], and this signal is used as the extracted first envelope signal; N0 is a reference parameter, N0 = d×Fs / v, Fs is the sampling frequency, and v is the speed of sound; Step 4.3: Zero the sound signal within the range of [M, 5N0], and then go to Step 4.1 to find the base point M again.

5. A method for sound source localization based on four microphones according to claim 1, characterized in that, In Step 5, the calculation method of the generalized cross-correlation function is used. Using the sound signal of microphone A as the reference signal, obtain the curves of the cross-correlation functions between microphones B, C, D and microphone A, and take the peaks on the curves as the time differences between the corresponding microphones and microphone A when receiving the sound signals.

6. A sound source localization method based on four microphones according to claim 1, characterized in that Step 6 is specifically as follows: Step 6.1: Establish the following equation: ； Among them, represents the time required for sound to travel from the sound source to microphone A, , and are all coefficients, , , The expressions of are as follows: ； Among them, represents the time difference of the received sound signal between microphone B and microphone A, represents the time difference of the received sound signal between microphone C and microphone A, represents the time difference of the received sound signal between microphone D and microphone A; v represents the speed of sound; Step 6.2: Set the following constraint conditions: ； Step 6.3: Calculate the two roots of the equation in Step 6.1, and select the root greater than zero and denote it as , based on , obtain the coordinate of the sound source The expression is as follows: 。 7. A system for implementing the method for sound source localization based on four microphones according to claim 1, characterized in that, Including: A sound acquisition module, used to collect the sounds emitted by the four set microphones; A data conversion module, used to convert the data into an array in Matlab; A preprocessing module, used to preprocess the data in the array in Matlab; A first envelope sound signal extraction module, used to extract the first envelope sound signal of each channel; A time difference calculation module, used to calculate the time differences between microphone A and microphones B, C, D when receiving the sound signals; A sound source location module, used to calculate the sound source position according to the time differences.

Images