Array posture angle compensation method for multi-microphone array sound source positioning

Abstract

This invention discloses an array attitude angle compensation method for sound source localization using a multi-microphone array. The method includes: first, estimating the azimuth and elevation angles of the sound source using a single-microphone array direction finding method based on time delay estimation, obtaining the sound source direction vectors of the sound source relative to each microphone array; then, measuring the array attitude angles using an attitude angle measurement device, and correcting the array attitude angles using a direction cosine matrix composed of the array attitude angles, thereby transforming the sound source direction vectors estimated by each microphone array to the same common horizontal coordinate system; finally, using the overall least squares algorithm to locate the sound source. This invention effectively solves the problem of increased sound source localization error caused by the inability to perfectly level the microphone arrays during multi-microphone array sound source localization, reduces the adverse effects of array attitude angles on sound source localization, and improves the accuracy of multi-microphone array sound source localization.

Description

Technical Field

[0001] This invention belongs to the field of sound source localization technology, specifically a method for array attitude angle compensation for sound source localization of multi-microphone arrays. Background Technology

[0002] Microphone array-based sound source localization technology has become a research hotspot both domestically and internationally in recent years. Its main principle is to collect sound source signals using a microphone array with a specific geometric topology, and then process and analyze the sound signals using array signal processing technology to determine the sound source location. Due to its advantages such as 360-degree detection capability, low cost, simple and convenient deployment, and immunity to terrain and electromagnetic interference, passive sound source localization technology is now widely used in various fields, including military and civilian applications, such as artillery reconnaissance, counter-sniper operations, audio / video conferencing systems, and intelligent robots.

[0003] Traditional sound source localization techniques mostly employ a network of single-microphone acoustic nodes or a single-microphone array. However, these methods are limited in detection range and accuracy due to factors such as microphone spacing and actual sound velocity. Sound source localization based on multi-microphone array nodes has become a research hotspot in recent years. It leverages the precise angle measurement capabilities of a single-microphone array node, treating it as an angle sensor, thus transforming the sound source localization problem into a localization problem based on multiple angles of arrival, thereby improving the detection range and localization accuracy of the acoustic detection system. References [Yang Yichun, Li Xiaodong, Tian Jing, Teng Pengxiao. Research on acoustic localization by multi-array data fusion [J]. Acoustic Technology, 2007.] fully utilize the advantages of the spatial distribution of multiple multi-microphone arrays and propose a multi-array joint processing algorithm based on data-level fusion, which improves the localization accuracy of indoor sound sources; References [Wang Wei. Research on acoustic localization system for aerial explosion points [D]. Nanjing University of Science and Technology, 2018.] construct an aerial explosion point localization system using three three-dimensional five-element arrays, which significantly improves the localization accuracy compared to a single node and can be used for orientation and distance determination of aerial explosion points; References [Xu Lei. Research on acoustic localization technology for explosion points by multi-array fusion [D]. Xi'an University of Technology, 2021.] propose and design a three-array fusion explosion point sound source test system, and verify through a system experimental prototype that the multi-array fusion method has high localization accuracy for explosion point sound localization.

[0004] However, existing research on sound source localization based on multiple microphones assumes that each microphone array is perfectly horizontal when performing sound source localization. In actual setup, it is difficult to achieve perfect horizontality, meaning that the pitch and roll angles of the array are non-zero. This will lead to a large error in the final sound source localization result. Therefore, it is necessary to correct the array attitude angle to reduce the adverse effects of array attitude on sound source localization. Summary of the Invention

[0005] The purpose of this invention is to propose a method for compensating the array attitude angle in sound source localization for multi-microphone array nodes, which solves the problem that the array nodes cannot be perfectly horizontal during the setup process, reduces the adverse effects of the array attitude angle on sound source localization, and improves the accuracy of sound source localization.

[0006] The technical solution to achieve the purpose of this invention is: a method for array attitude angle compensation in multi-microphone array sound source localization, comprising the following steps:

[0007] Step 1: Direction finding of sound source in single microphone array. The azimuth and elevation angles of the sound source under test are estimated using time delay estimation methods, thereby obtaining the sound source azimuth vector of the sound source under test relative to each microphone array.

[0008] Step 2: Array attitude angle correction. The array attitude angles, including yaw, pitch and roll angles, are measured using an array attitude angle measurement device. Then, the array attitude angles are corrected using a direction cosine matrix. The sound source azimuth vectors estimated by each microphone array in Step 1 are transformed into the same common horizontal coordinate system.

[0009] Step 3, Multi-microphone array sound source localization: First, measure the position coordinates of each microphone array in the global coordinate system. Combined with the azimuth vectors of each sound source in Step 2, use the overall least squares method to locate the sound source to be measured.

[0010] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described array attitude angle compensation method for multi-microphone sound source localization.

[0011] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described array attitude angle compensation method for multi-microphone sound source localization.

[0012] Compared with the prior art, the significant advantages of this invention are: (1) the array attitude angle is compensated and corrected during the sound source localization process of multi-microphone array, which solves the adverse effect of the array not being able to be set up horizontally on the sound source localization accuracy and further improves the sound source localization accuracy; (2) the method of this invention is convenient to implement, easy to implement, applicable to all microphone array structures, and has universality. Attached Figure Description

[0013] Figure 1 This is a flowchart of a method for compensating the array attitude angle in sound source localization for multi-microphone array nodes.

[0014] Figure 2 This is a schematic diagram of the global horizontal coordinate system (n-system) and the local carrier coordinate system (b-system) of a five-element stereo microphone array.

[0015] Figure 3 This is a schematic diagram of the coordinate system rotation process. Detailed Implementation

[0016] Combination Figure 1 The present invention provides a method for compensating the array attitude angle in sound source localization for multi-microphone array nodes, the specific steps of which are as follows:

[0017] Step 1: Estimate the azimuth vector of the sound source under test based on a single microphone array, specifically including the following steps:

[0018] Step 1-1: Detect the sound signals recorded by each channel of the single microphone array and extract the data portion containing the target sound signal;

[0019] Step 1-2: Calculate the arrival time difference between the target sound source and each array element channel using generalized cross-correlation. The specific steps are as follows:

[0020] Step 1-2-1: Taking the signals received by the two array elements as x1(t) and x2(t) respectively as an example, their corresponding cross-power spectral density functions are:

[0021]

[0022] The asterisk (*) indicates complex conjugation.

[0023] Step 1-2-2: Let H(f) be the weighting function used for generalized cross-correlation. Then, the cross-power spectral density function after weighting is:

[0024]

[0025] Steps 1-2-3: According to the Wiener-Khinchin theorem, the cross-correlation function of a signal and its cross-power spectral density are Fourier transform pairs. Therefore, the generalized cross-correlation function of x1(t) and x2(t) is...

[0026]

[0027] Steps 1-2-4: According to the principle of cross-correlation, The peak value corresponding to τ is the delay τ between signal times x1(t) and x2(t). 12 The estimated value;

[0028] Step 1-2-5: Repeat steps 1-2-1 to 1-2-4 to obtain the time delay estimate between each channel of the microphone array.

[0029] Step 1-2-6: Taking a 5-element microphone array as an example, based on the results of step 1-2-5, the TDOA vector τ of the sound source can be obtained as [τ]. 12 , τ 13 , τ 14 , τ 15 …τ 35， τ 45 ] T ;

[0030] Steps 1-3: Solve for the structure coefficient matrix of the microphone array, specifically including the following steps;

[0031] Step 1-3-1: Taking a 5-element microphone array as an example, the position coordinates of each element are represented by r. i (i = 1, 2, 3, 4, 5) represents;

[0032] Step 1-3-2: Obtain the structure coefficient matrix corresponding to the array.

[0033] Steps 1-4: Using the sound source direction finding method based on time delay estimation, the estimated azimuth and elevation angles of the sound source can be obtained. The specific solution steps are as follows;

[0034] Step 1-4-1: Let the unit vector of the target sound source S relative to the microphone array be represented as...

[0035]

[0036] Where θ is the azimuth angle of the sound source. The elevation angle of the sound source;

[0037] Step 1-4-2: Based on the direction-finding algorithm based on time delay estimation, the linear equation system Wk=τ can be obtained.

[0038]

[0039] Where W is obtained from steps 1-3, and τ is obtained from steps 1-2;

[0040] Step 1-4-3: Since Wk=τ is an overdetermined linear system of equations, there exists a unique linear least squares solution. Therefore, k can be obtained from the following formula, where Wk=τ ... + Let W be the Moore-Penrose inverse of the array coefficient matrix W;

[0041]

[0042] Step 1-4-4, let W + =[u1,u2,u3] T Where u1, u2, and u3 are all 10×1 column vectors, the estimated azimuth and elevation angles of the sound source can be obtained as follows:

[0043]

[0044]

[0045] Steps 1-5: Using the solution results from steps 1-4, the estimated value of the azimuth vector of the sound source S can be obtained.

[0046]

[0047] Step 2, Combining Figure 2 Taking a 5-element stereo microphone array as an example, the sound source orientation vectors relative to each microphone array obtained in step 1 are... The process of correcting the array attitude angle includes the following steps:

[0048] Step 2-1: Measure the array attitude angles using attitude angle sensors, denoted as yaw angle α, pitch angle β, and roll angle ψ, respectively, as defined below. Figure 2 As shown;

[0049] Step 2-2: Obtain the direction cosine matrix corresponding to the array attitude angle. The specific solution steps are as follows;

[0050] Step 2-2-1: According to Euler's rotation theorem, the process of transforming the array from the global horizontal coordinate system n to the local carrier coordinate system b can be achieved by sequentially rotating around z... n axis, y n axis, x n The rotation of the axis is achieved through three rotations, and the schematic diagram of the rotation process is shown below. Figure 3 As shown, the direction cosine matrices of the corresponding yaw angle α, pitch angle β, and roll angle ψ can be expressed as follows:

[0051]

[0052] Step 2-2-2: The direction matrix corresponding to the rotation from a completely horizontal array (corresponding to the global horizontal coordinate system n) to a non-horizontal array (corresponding to the local carrier coordinate system b). It can be represented as

[0053]

[0054] Step 2-2-3: Because the array remains in a Cartesian coordinate system throughout the rotation, all three direction cosine matrices are orthogonal identity matrices. Therefore, the direction matrices... It is also an orthogonal matrix, meaning it satisfies the relation

[0055]

[0056] Step 2-2-4: Direction cosine matrix from the carrier coordinate system b to the global horizontal coordinate system n. It can be represented as

[0057]

[0058] Steps 2-3: Use the obtained direction cosine matrix The sound source orientation vector relative to each microphone array obtained in step 1 Correcting the array attitude angles involves the following steps:

[0059] Step 2-3-1: Let the azimuth vector of the target sound source corresponding to the array in a completely horizontal state be...

[0060] Step 2-3-2, then Compared with the estimate obtained in step 1 Satisfying Relationships

[0061]

[0062] Using the above formula, the sound source azimuth vectors estimated by each node of the system can be transformed into a global common horizontal coordinate system.

[0063] Step 3: Combine the transformed target sound source azimuth vector from Step 2 with the coordinates of each array to locate the target sound source. The specific steps are as follows:

[0064] Step 3-1: Use measuring equipment to obtain the center coordinates of each array, and record them as (x... i ,y i ,z i (i = 1, 2, ..., n, where n is the number of microphone arrays used), the estimated azimuth and elevation angles of the sound source are denoted as θ. i and

[0065] Step 3-2: Let the coordinates of the target sound source S be (x, y, z), then the following relationship can be established.

[0066]

[0067] It can be abbreviated as Ms = q.

[0068] Step 3-3: Using the total least squares method, an estimate of s can be obtained.

[0069]

[0070] Where λ is the matrix C T The smallest eigenvalue of C, where C is the augmented matrix [M q];

[0071] Steps 3-4: Finally, obtain the estimated three-dimensional coordinates of the sound source S.

[0072]

[0073] In summary, this invention provides an attitude angle compensation method for sound source localization using a multi-microphone array. This invention effectively solves the practical problem that multi-microphone arrays cannot be perfectly horizontal when used for sound source localization, reducing the adverse effects of array attitude angles on sound source localization and improving the accuracy of sound source localization.

[0074] This invention is not limited to the contents involved in the claims and the above embodiments. Any invention created based on the concept of this invention should fall within the protection scope of this invention.

Claims

1. A method for array attitude angle compensation for multi-microphone sound source localization, characterized in that, Includes the following steps: Step 1: Direction finding of sound source in single microphone array. The azimuth and elevation angles of the sound source under test are estimated using time delay estimation methods, thereby obtaining the sound source azimuth vector of the sound source under test relative to each microphone array. The estimation of the azimuth vector of the sound source under test based on a single microphone array includes the following steps: Step 1-1: Detect the sound signals recorded by each channel of the single microphone array and extract the data portion containing the target sound signal; Step 1-2: Calculate the arrival time difference between the target sound source and each array element channel using generalized cross-correlation. The specific steps are as follows; Step 1-2-1: The signals received by the two array elements are x1(t) and x2(t), respectively, and their corresponding cross-power spectral density functions are: ； The asterisk (*) indicates complex conjugation. Step 1-2-2: Let the weighting function used for generalized cross-correlation be H(f), then the cross-power spectral density function after weighting is... ； Steps 1-2-3: According to the Wiener-Khinchin theorem, the cross-correlation function of a signal and its cross-power spectral density are Fourier transform pairs. Therefore, the generalized cross-correlation function of x1(t) and x2(t) is... ； Steps 1-2-4: According to the principle of cross-correlation, The peak value corresponding to τ is the delay τ between signal times x1(t) and x2(t). 12 The estimated value; Step 1-2-5: Repeat steps 1-2-1 to 1-2-4 to obtain the time delay estimates between each channel of the microphone array; Step 1-2-6: For a 5-element microphone array, based on the results of step 1-2-5, obtain the TDOA vector of the sound source. ; Steps 1-3: Solve for the structure coefficient matrix of the microphone array. The specific steps are as follows; Step 1-3-1: For a 5-element microphone array, the position coordinates of each element are represented by r. i This means that i = 1, 2, 3, 4, 5; Step 1-3-2: Obtain the structure coefficient matrix corresponding to the array. ; Steps 1-4: Using the sound source direction finding method based on time delay estimation, the estimated azimuth and elevation angles of the sound source are obtained; the specific steps are as follows; Step 1-4-1: Let the target sound source S be represented as a unit vector relative to the microphone array as... ； Where θ is the azimuth angle of the sound source and φ is the elevation angle of the sound source; Step 1-4-2: Based on the direction-finding algorithm based on time delay estimation, obtain the linear equation system. ; ； Where W is obtained from steps 1-3, and τ is obtained from steps 1-2; Step 1-4-3, due to If the system of equations is an overdetermined linear system, then there exists a unique linear least squares solution. It can be obtained from the following formula, where W + Let W be the Moore-Penrose inverse of the array coefficient matrix W; ； Step 1-4-4, let W + = [u1,u2,u3] T Where u1, u2, and u3 are all 10×1 column vectors, the estimated azimuth and elevation angles of the sound source are obtained as follows: ； ； Steps 1-5: Using the solution results from steps 1-4, obtain the estimated value of the azimuth vector of the sound source; Step 2: Array attitude angle correction. The array attitude angles, including yaw, pitch and roll angles, are measured using an array attitude angle measurement device. Then, the array attitude angles are corrected using a direction cosine matrix. The sound source azimuth vectors estimated by each microphone array in Step 1 are transformed into the same common horizontal coordinate system. The sound source orientation vector relative to each microphone array obtained in step 1 The process of correcting the array attitude angle includes the following steps: Step 2-1: Measure the array attitude angles using the attitude angle sensor, and record them as yaw angle α, pitch angle β and roll angle ψ, respectively; Step 2-2: Obtain the direction cosine matrix corresponding to the array attitude angle. The specific solution steps are as follows; Step 2-2-1: According to Euler's rotation theorem, the direction cosine matrices corresponding to the yaw angle α, pitch angle β, and roll angle ψ can be expressed as follows: ， ， ； Step 2-2-2: The direction matrix corresponding to the array rotating from a completely horizontal state to a non-horizontal state. It can be represented as ； Step 2-2-3: Because the array remains in a Cartesian coordinate system throughout the rotation, all three direction cosine matrices are orthogonal identity matrices. Therefore, the direction matrices... It is also an orthogonal matrix, meaning it satisfies the relation ； Step 2-2-4: Direction cosine matrix from the carrier coordinate system b to the global horizontal coordinate system n. It can be represented as ； Steps 2-3: Use the obtained direction cosine matrix The sound source orientation vector relative to each microphone array obtained in step 1 Correcting the array attitude angles involves the following steps: Step 2-3-1: Let the azimuth vector of the target sound source corresponding to the array in a completely horizontal state be... ; Step 2-3-2, then Compared with the estimate obtained in step 1 Satisfying Relationships ； Using the above formula, the sound source azimuth vectors estimated by each node of the system can be transformed into the global common horizontal coordinate system; Step 3, Multi-microphone array sound source localization: First, measure the position coordinates of each microphone array in the global coordinate system. Combined with the sound source orientation vectors from Step 2, use the overall least squares method to locate the sound source under test. The specific implementation steps are as follows: Step 3-1: Use measuring equipment to obtain the center coordinates of each array, and record them as (x... j ,y j ,z j ), j =1,2…n, where n is the number of microphone arrays used, and the estimated azimuth and elevation angles of the sound source are denoted as θ. j and φ j j = 1, 2…n; Step 3-2: Let the coordinates of the target sound source S be (x, y, z), then the following relationship can be established. ； This can be abbreviated as Ms = q; Step 3-3: Using the total least squares method, an estimate of s can be obtained. ； Where λ is the matrix C T The smallest eigenvalue of C, where C is the augmented matrix [M q]; Steps 3-4: Finally, obtain the estimated values ​​of the three-dimensional coordinates of the sound source S. 。 2. The array attitude angle compensation method for multi-microphone sound source localization as described in claim 1, characterized in that, Using the results from steps 1-4, the estimated value of the azimuth vector of the sound source is obtained: 。 3. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the array attitude angle compensation method for multi-microphone sound source localization as described in any one of claims 1-2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the array attitude angle compensation method for multi-microphone sound source localization as described in any of claims 1-2.

Images