Design and implementation method of two-layer information fusion model of distributed microphone array

Abstract

The application provides a design and implementation method of a two-layer information fusion model of a distributed microphone array, including: obtaining a harmonic fundamental frequency used for identifying a target feature of a sound source; performing first fusion on estimated results from different channels in a certain array of the distributed microphone array; and performing second fusion on estimated results from different arrays of the distributed microphone array by using reinforcement learning. The harmonic estimation method adopted by the application is suitable for a harmonic estimation problem under a long-distance, low-frequency band and low signal-to-noise ratio condition in far-field sound source detection, the computational complexity is reduced by decomposing and simplifying a target function, the real-time performance of harmonic estimation is improved, and the estimation accuracy, noise robustness and stability are ensured; the second-layer fusion based on reinforcement learning makes the final estimated result more consider the estimated value given by an array with high estimation accuracy, and has a positive effect on improving the array information fusion accuracy.

Description

Technical Field

[0001] This invention relates to sound source detection and reinforcement learning technologies, specifically to a design method for a two-layer information fusion model of a distributed microphone array based on reinforcement learning. Background Technology

[0002] In recent years, the detection of far-field sound sources has been increasingly widely used in both military and civilian fields, such as secondary sound source localization and security systems in the civilian sector, and drone reconnaissance and artillery positioning in the military sector. Compared with detection methods such as radar and imagery, sound-based sound source target detection methods have advantages such as low cost and high counter-surveillance difficulty. The acoustic signals used for sound source target detection are usually harmonic signals generated by the vibration of the target. These signals typically have low characteristic fundamental frequencies and complex and variable interference components, which makes the detection of sound source targets difficult.

[0003] Compared to single arrays, distributed microphone arrays can provide a better understanding of the observed object and obtain more accurate observation results. However, when using distributed microphone arrays for helicopter detection, different arrays will produce different estimates due to differences in location and environment. Under extreme conditions, the arrays may even be damaged and malfunction. How to accurately fuse the estimates from multiple arrays is a scientific problem that needs to be solved. Summary of the Invention

[0004] The purpose of this invention is to address the problem of obtaining accurate final fusion results from data from various arrays in complex and variable environments during far-field sound source detection. This invention proposes a two-layer information fusion model for distributed microphone arrays based on reinforcement learning, which can improve the accuracy of fusing data from distributed arrays. This represents an important attempt to apply reinforcement learning technology to array data fusion and sound source localization.

[0005] Technical solution

[0006] The implementation process of the method of this invention includes:

[0007] I. Obtaining the fundamental harmonic frequency used to identify the characteristics of the sound source target

[0008] The vector form of the harmonic signal model to be estimated is:

[0009] x = Z l (ω0)α l +e

[0010] Where x and e are the sampling sequences of x(n) and e(n) respectively, Z l α represents the sine and cosine components of the harmonic signal. l This represents a linear amplitude vector. The estimated value of the harmonic fundamental frequency can be derived as follows:

[0011]

[0012] Use J NLS (ω0,l) represents the objective function in the above equation, then J NLS (ω0,l)=x T Z l (ω0)α l Objective function J NLS Solving for (ω0,l) can be simplified to solving a system of equations consisting of the Toeplitz matrix and the Hankel matrix.

[0013]

[0014]

[0015] in a l and b l These are the magnitude vectors for cosine and sine classification, respectively, with α. l =[a l T -b l T ] T T l (ω0) is a Toeplitz matrix, whose adjacent elements along the main diagonal are identical, and whose elements are symmetric about the second diagonal; H l (ω0) is a Hankel matrix, where each element has the same value as its adjacent element along the secondary diagonal. By recursively solving for the amplitude vector and substituting it into the fundamental frequency estimation formula, the final fundamental frequency estimate can be calculated.

[0016] II. The estimation results from different channels within a distributed microphone array are then processed.

[0017] One fusion

[0018] Because the different channels within the same array are close together, operate in a relatively consistent noise environment, and experience essentially the same interference, data fusion can be directly performed within the same array by averaging. The calculation formula is as follows:

[0019]

[0020] Where F represents the fundamental frequency estimate obtained after the first fusion, per microphone array unit, f q This represents the fundamental frequency estimate obtained from the data of the q-th channel of a certain microphone array, where Q represents the total number of channels.

[0021] III. Using reinforcement learning to analyze the estimation results from different arrays of distributed microphone arrays.

[0022] Second Fusion

[0023] In the second-layer fusion, the environments in which each microphone array is located vary significantly, and the distances and orientations between the distributed arrays and the sound sources also differ considerably. Therefore, averaging the data is no longer feasible for fusion. Theoretically, arrays in environments with minimal interference and in good working order can produce more accurate estimates, while those in less favorable environments will be less accurate. Therefore, the second-layer fusion model employs a weighted summation of the estimates from the first-layer fusion. Each array has its corresponding weight value, denoted by F. fused This indicates the result after fusion.

[0024] F fused = w1F1 + w2F2 + ... + w n F n

[0025] Where w i Let F be the weight value of the i-th array. i The estimated fundamental frequency of the i-th array output. The weight sequence is dynamically adjusted using a model-free reinforcement learning algorithm based on the confidence level of the estimation results for each array.

[0026] In reinforcement learning algorithms, the state value is set as the fusion result s calculated using the current weight sequence and the estimation results of the first layer of each array. t = w1F1 + w2F2 + ... + w n F n The action is set to correct the weight values ​​of each array, divided into three types: increasing by a fixed percentage, decreasing by a fixed percentage, and keeping them unchanged. The reward value is calculated using Fisher information content, which has the ability to evaluate the amount of information contained in the observed data and can reflect the accuracy of the data. Therefore, this invention calculates Fisher information content at each time step and compares the information content at previous and subsequent time steps, with the final reward value set as follows:

[0027]

[0028] The reinforcement learning process, taking the Q-learning algorithm as an example, is as follows:

[0029] 1) Initialization: k = 0

[0030] Randomly initialize the array weights [w1, w2, ..., w n The constraints are w1 + w2 + ... + w n =1, and calculate the initial state value.

[0031] 2) Execute when k>=1

[0032] ① For the current state s t Based on the current Q-table, select action a using the ε-greedy strategy. t ;

[0033] ② Perform action a t Adjust the weight values ​​and observe the state s at the next time step. t+1 And calculate the reward value r t ;

[0034] ③ Update the Q value according to the following formula

[0035]

[0036] ④Let s t ←s t+1 ;

[0037] ⑤ When k = k + 1, proceed to step ②.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] (1) The harmonic estimation method adopted in this invention is applicable to the harmonic estimation problem under the conditions of long distance, low frequency band and low signal-to-noise ratio in far-field sound source detection. By decomposing and simplifying the objective function, the computational complexity is reduced and the real-time performance of harmonic estimation is improved. At the same time, the estimation accuracy, robustness to noise and stability are guaranteed.

[0040] (2) This invention fuses the distributed microphone array data twice according to the magnitude of the difference, and uses a reinforcement learning algorithm in the second fusion to improve the accuracy of the final fusion result. The reinforcement learning-based second-layer fusion makes the final estimation result take into account more the estimated values ​​given by the array with higher estimation accuracy, which has a positive effect on improving the accuracy of array information fusion. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of a two-layer information fusion model for a distributed microphone array based on reinforcement learning.

[0042] Figure 2 Flowchart of Q-learning-based information fusion algorithm Detailed Implementation

[0043] In sound source localization, microphones are efficient and low-cost signal acquisition devices. However, the signal quality of a single microphone deteriorates significantly under complex conditions such as noise, multipath reflections, and reverberation. Distributed microphone arrays, on the other hand, can perform joint filtering of aerodynamic noise signals in the time-frequency-spatial domain. Compared with single-microphone methods, distributed microphone arrays offer more processing degrees of freedom and superior performance. However, how to fuse data from different channels and arrays when using distributed microphone arrays is a scientific problem that needs to be solved. This invention introduces a reinforcement learning algorithm to improve the flexibility of information fusion and enhance the accuracy of the final fusion result.

[0044] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention will be presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to this embodiment. On the contrary, the purpose of describing the invention in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a deep understanding of the present invention, many specific details will be included in the following description. The present invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the present invention, some specific details will be omitted in the description. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0045] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0046] The design and implementation steps of a two-layer information fusion model for a distributed microphone array based on reinforcement learning are as follows:

[0047] First, it is necessary to obtain the fundamental harmonic frequency used to identify the characteristics of the sound source target.

[0048] Assuming there are 4 microphone arrays in a far-field sound source detection environment, each array having Q channels, the signal to be detected acquired by the q-th channel at time t is modeled as follows:

[0049]

[0050] Where ω0 represents the fundamental frequency of the harmonic signal, in rad / s, a i and b i This represents the amplitude value of the i-th harmonic component, where l = 4 and i is a natural number less than or equal to 4.

[0051] Use fq f represents the fundamental frequency estimate obtained from the data of the q-th channel of a certain microphone array, as described in the technical solution. q It can be estimated using the nonlinear least squares method, as follows.

[0052]

[0053]

[0054] Solving the objective function of the above nonlinear least squares method can be achieved by solving the following system of equations:

[0055]

[0056] Where a l and b l These are the magnitude vectors for cosine and sine classification, respectively, with α. l =[a l Tb l T ] T Find α l And substitute it into the fundamental frequency estimation f q The final fundamental frequency estimate can be obtained from the objective function (3). l (ω0) is a Toeplitz matrix, whose adjacent elements along the main diagonal are identical, and whose elements are symmetric about the second diagonal; H l (ω0) is a Hankel matrix, where each element has the same value as its adjacent element along the secondary diagonal. The two matrices are defined as follows:

[0057]

[0058]

[0059]

[0060]

[0061] II. First fusion of estimation results from different channels within a distributed microphone array.

[0062] Let F represent the fundamental frequency estimate obtained after the first fusion, on a unit of microphone array. F is calculated as follows:

[0063]

[0064] Third, reinforcement learning is used to perform a second fusion of the estimation results from different arrays of the distributed microphone array:

[0065] Each array has its corresponding weight value, denoted by F. fused The result after fusion is represented by the following formula:

[0066] F fused = w1F1 + w2F2 + ... + w n F n (10)

[0067] Where w i Let F be the weight value of the i-th array. i The estimated fundamental frequency of the i-th array output. The weight sequence is dynamically adjusted using a model-free reinforcement learning algorithm based on the confidence level of the estimation results for each array.

[0068] In reinforcement learning algorithms, the state value S t The state value S is set to be calculated using the current weight sequence and the estimation results of the first layer of each array, and is used to fuse the current fusion result. t The calculation formula is

[0069] s t = w1F1 + w2F2 + ... + w n F n (11)

[0070] Action a t The settings are configured to adjust the weight values ​​of each array, categorized into three types: increasing by a fixed percentage, decreasing by a fixed percentage, and maintaining the same value. In this case, it is assumed there are 4 arrays. Due to the characteristic that w1+w2+w3+w4=1, the action sequence for 3 of the arrays is a. t =[a 1t ,a 2t ,a 3t The weight value of the fourth array is...

[0071] Reward value r t The Fisher information content is used for calculation, which has the ability to assess the amount of information contained in observed data and can reflect the accuracy of the data. Therefore, this invention calculates the Fisher information content at each time step and compares the information content at different time steps. Finally, r t Reward value set to

[0072]

[0073] Where I represents the Fisher information content, calculated using the formula:

[0074]

[0075] In engineering applications, the true frequency of the target object is unknown, while the estimation result from the previous moment has a certain degree of accuracy. Therefore, the estimation result from the previous moment is used to replace the true value. The final formula for calculating Fisher information is:

[0076]

[0077] After completing the above reinforcement learning problem modeling, according to Figure 2 The reinforcement learning process based on the Q-learning algorithm adjusts the weights of each array and gives the final fusion result value.

[0078] Specifically, the reinforcement learning method process, taking the Q-learning algorithm as an example, is as follows:

[0079] 1) Initialization: k = 0

[0080] Randomly initialize the array weights [w1, w2, ..., w n The constraints are w1 + w2 + ... + w n =1, and calculate the initial state value.

[0081] 2) Execute when k>=1

[0082] 2.1) For the current state s t Based on the current Q-table, select action a using the ε-greedy strategy. t ;

[0083] 2.2) Execute action a t Adjust the weight values ​​and observe the state s at the next time step. t+1 And calculate the reward value r t ;

[0084] 2.3) Update the Q value according to the following formula

[0085]

[0086] 2.4) Let s t ←s t+1 ;

[0087] 2.5) When k = k + 1, go to step 2.2).

[0088] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1. A design and implementation method for a two-layer information fusion model of a distributed microphone array, characterized in that, include: Step 1: Obtain the fundamental harmonic frequency used to identify the characteristics of the sound source target; Step 2: Perform the first fusion of the estimation results from different channels within a distributed microphone array; Step 3: Use reinforcement learning to perform a second fusion of the estimation results from different arrays of the distributed microphone array; Step Two: Data fusion is performed directly by averaging within the same array, and the calculation formula is as follows: in This represents the fundamental frequency estimate obtained after the first fusion, on a unit basis using the microphone array. Indicates the number of microphones in a certain microphone array The fundamental frequency estimate obtained from the data of each channel. Indicates the total number of channels; Step 3: In the second-layer fusion model, the estimates from the first-layer fusion are weighted and summed; each array has its corresponding weight value, used as... This indicates the result after fusion. in For the first The weight values ​​of each array, No. The estimated fundamental frequency of each array output; the weight sequence is dynamically adjusted using a model-free reinforcement learning algorithm based on the confidence level of the estimation results for each array; In reinforcement learning algorithms, the state value is set as the fusion result calculated using the current weight sequence and the estimation results of the first layer of each array. The action is set to adjust the weight values ​​of each array, divided into three types: increasing by a fixed percentage, decreasing by a fixed percentage, and keeping them unchanged; the reward value is calculated using Fisher information; the Fisher information is calculated at each time step, and the information values ​​of previous and subsequent time steps are compared, with the final reward value set as follows: in This indicates the Fisher information content.

2. The method as described in claim 1, characterized in that, Step 1: The vector form of the harmonic signal model to be estimated is: in and They are and The sampling sequence, Represents the sine and cosine components of a harmonic signal. Represents a linear amplitude vector; the estimated value of the harmonic fundamental frequency is derived as follows: use Let the objective function in the above equation be denoted as , then we have ; objective function The solution is simplified to solving a system of equations consisting of Toeplitz and Hankel matrices. in , and These are the magnitude vectors for cosine and sine classification, respectively. , It is a Toeplitz matrix, in which adjacent elements along the main diagonal are identical, and each element is symmetric about the second diagonal. The fundamental frequency is a Hankel matrix, where each element has the same value as its adjacent element along the secondary diagonal. The magnitude vector is solved recursively and substituted into the fundamental frequency estimation formula to finally calculate the estimated fundamental frequency. .

3. The method as described in claim 1, characterized in that, The reinforcement learning process of the Q-learning algorithm is as follows: 1) Initialization: k =0 Randomly initialize the array weights The constraints are And calculate the initial state value. ; 2) When k> =1 Execute ① Regarding the current state Based on the current Q table, use -greedy strategy selects action ; ② Perform the action Adjust the weight values ​​and observe the state at the next time step. And calculate the reward value ; ③ Update the Q value according to the following formula ④ Order ; ⑤ When Then proceed to step ②.

Images