Transient sound real-time detection and positioning method and system based on microphone array

Abstract

The invention discloses a transient sound real-time detection and positioning method and system based on a microphone array. The method comprises the following steps: 1) real-time detection: processing multi-channel audio data by combining real-time updated noise statistics and a multi-state finite state machine so as to robustly detect a transient sound event and preliminarily define the boundary of the transient sound event; and 2) real-time positioning: further utilizing an energy centroid method to dynamically refine an analysis window for the preliminarily defined event segment, and efficiently solving a sound source direction in combination with a microphone pair optimization subset and a constant matrix which are pre-generated in an offline optimization stage. The system is based on an embedded real-time operating system, and reliable real-time cooperation of detection and positioning tasks is achieved through multi-task division and inter-task message queues. According to the invention, the challenge of poor detection robustness under the conditions of low signal-to-noise ratio and non-stationary noise is effectively dealt with, the problem that the computing resources of an embedded platform are limited is overcome, and a transient sound monitoring scheme with low cost, high real-time performance and high accuracy is provided.

Description

Technical Field

[0001] The present invention relates to a method and system for real-time detection and positioning of transient sounds based on a microphone array, and belongs to the technical field of acoustic signal processing. Background Art

[0002] Transient sound is a non-stationary signal characterized by short duration, broad spectral coverage, and concentrated energy. It is typically generated by physical events such as ruptures, impacts, or explosions. Rapid detection and accurate location of transient sound events are crucial in a variety of applications, including public safety monitoring, industrial equipment fault diagnosis, wildlife behavior monitoring, and human-computer interaction.

[0003] In terms of transient sound detection technology, traditional energy detection methods have been widely studied due to their simple implementation and intuitive physical meaning. However, their performance deteriorates sharply in low signal-to-noise ratio (SNR) environments, making them prone to false alarms or missed detections. To improve the robustness of detection in strong noise backgrounds, researchers have proposed a variety of improved algorithms. Wang Yan et al. proposed a dual noise reduction method of wavelet detection and local instantaneous energy density level. By using the empirical mode decomposition adaptive noise reduction method and calculating the Hilbert spectrum for local integration, they can effectively filter out high-frequency and low-frequency noise. Based on the time domain distribution characteristics of the cepstrum signal, Xu Jingfeng et al. proposed the use of cepstrum-wavelet for transient pulse detection. Simulations and experiments have shown that it has good detection effects even under extremely low signal-to-noise ratios. Yang Desen et al. combined transient signal detection with neural networks and used the learning and prediction capabilities of radial basis function neural networks to detect weak underwater acoustic transient signals. Although the above algorithms improve detection accuracy under specific conditions, they usually rely on complex signal transformations or iterative operations. Their high computational complexity poses a severe challenge to directly deploying the algorithms on embedded microcontrollers with limited memory and computing power.

[0004] For sound source direction estimation, direction of arrival (DOA) estimation algorithms based on microphone arrays are one of the main technical approaches. Among these, localization techniques based on time difference of arrival (TDOA) accurately estimate the time delay between sound waves reaching different array elements and thereby determine the sound source's direction. Among the numerous time delay estimation algorithms, the generalized cross-correlation-phase transform (GCC-PHAT) algorithm proposed by Knapp et al. has become one of the most widely used benchmark algorithms due to its superior performance and robustness in reverberant environments. Wang et al. proposed an improved GCC-PHAT-SM method by introducing a speech mask to suppress noise interference, significantly enhancing the robustness of robot speech signal localization in high-noise and reverberant environments. Although the algorithm itself and its improved versions offer superior performance, its core computational complexity is proportional to the square of the number of microphone channels. When the array size increases, the computational overhead in fast Fourier transform and cross-power spectrum calculation will rise sharply, which makes it still difficult to implement multi-channel real-time processing on resource-constrained embedded platforms.

[0005] Considering that the transient sound detection algorithm in the prior art has poor robustness in a non-stationary noise environment, and the sound source localization algorithm has high computational complexity and difficulty in ensuring real-time performance on an embedded platform, the present invention is proposed. Summary of the Invention

[0006] The present invention provides a method and system for real-time detection of transient sounds based on a microphone array, which are used to realize real-time detection of transient sounds; further provides a method and system for real-time positioning of transient sounds based on a microphone array, which are used to locate the sound source of the detected transient sounds.

[0007] The technical solution of the present invention is:

[0008] According to a first aspect of the present invention, a method for real-time detection of transient sounds based on a microphone array is provided, comprising:

[0009] Audio acquisition step: multi-channel audio data is acquired in a time division multiplexing manner to obtain one-dimensional raw audio data in TDM format;

[0010] The transient sound detection step includes: deinterleaving the one-dimensional original audio data, recombining the multi-channel interleaved original data into independent channel data streams, synchronously framing the obtained channel data streams by using an overlapping sliding window strategy, inputting each generated data frame into a "transient sound detection algorithm" for transient sound detection, and performing the following key operations if the rough boundary of a transient sound event is determined by the comprehensive judgment of the "transient sound detection algorithm": dynamically allocating a linear data buffer, copying the complete segment of the transient sound event in the main ring buffer into the linear data buffer, and constructing a message body containing a pointer to the linear data buffer and the data length thereof; otherwise, the transient sound detection step is continuously performed on the next data frame.

[0011] Further, the transient sound detection algorithm specifically includes:

[0012] updating the noise statistics of each audio channel data frame in real time by using an exponential weighted moving average model;

[0013] calculating an adaptive detection threshold based on the real-time updated noise statistics;

[0014] constructing a finite state machine, performing single-channel state recognition based on the finite state machine according to the noise statistics and the adaptive detection threshold;

[0015] performing comprehensive judgment based on the multi-channel state to preliminarily define the boundary of the transient sound event.

[0016] Further, the single-channel state recognition based on the finite state machine according to the noise statistics and the adaptive detection threshold specifically includes:

[0017] a four-state finite state machine including "silence", "energy rise", "energy sustain" and "energy drop" is designed for each channel;

[0018] the state definition and transition conditions are as follows:

[0019] (1) silence state: in this state, the noise statistics are completely updated and when and , the state is transferred to the "energy rise" state; otherwise, no transition; wherein, and respectively represent the energy mean and standard deviation of the i-th channel in the t-th frame; represents the energy of the i-th channel in the t-th frame; represents the energy gradient of the i-th channel in the t-th frame; η1 represents a gradient threshold factor of energy rise;

[0020] (2) Energy rising state: in this state, the update of the noise statistics is frozen; if and x≥x r , transition to the "energy sustaining" state; if transition to the "silence" state; otherwise, no transition; x denotes the state sustaining frame number, x r denotes the minimum frame number of the rising state sustaining;

[0021] (3) Energy sustaining state: in this state, the update of the noise statistics is frozen; if and x≥x f , transition to the "energy falling" state; otherwise, no transition; wherein, η2 denotes the gradient threshold factor of the energy falling, x f denotes the minimum frame number of the falling state sustaining;

[0022] (4) Energy falling: in this state, the noise statistics is partially updated; if transition to the "energy rising" state; if transition to the "silence" state; otherwise, no transition.

[0023] Further, the comprehensive decision is performed based on the multi-channel states to preliminarily define the boundary of the transient acoustic event, specifically:

[0024] The initial state of all channels is set to the "silence state", and the noise statistics thereof is assigned with a preset initial value;

[0025] For each data frame, after the current state of each channel is determined by the finite state machine, the number of channels currently in the non-"silence state" is counted in real time, denoted as N active ;

[0026] The data frame that causes the value of N active to change from less than the preset channel number threshold N th to greater than or equal to N th is defined as the "start frame" of an effective transient acoustic event;

[0027] After the "start frame" is detected, the data frame at which the state of all channels returns to the "silence state" for the first time is defined as the "end frame" of the effective transient acoustic event.

[0028] According to a second aspect of the present application, a microphone array-based transient acoustic real-time detection system is provided, comprising:

[0029] An audio acquisition task module, configured to acquire multi-channel audio data in a time division multiplexing manner to obtain one-dimensional original audio data in a TDM format;

[0030] The transient sound detection task module is used for deinterleaving the one-dimensional original audio data, recombining the original data interleaved by multiple channels into independent channel data streams, using an overlapping sliding window strategy to synchronously frame the obtained channel data streams, inputting each generated data frame into a transient sound detection algorithm for transient sound detection, and if the rough boundary of a transient sound event is determined through the comprehensive judgment of the transient sound detection algorithm, the following key operations are performed: dynamically allocating a linear data buffer, copying the complete segment of the transient sound event in the main ring buffer into the linear data buffer, and constructing a message body containing a pointer to the linear data buffer and the data length thereof; otherwise, the transient sound detection task module is continuously executed on the next data frame.

[0031] According to a third aspect of the application, a microphone array-based transient sound real-time positioning method is provided, comprising: a sound source positioning step: receiving a message body containing a pointer to a linear data buffer and the data length thereof generated by the method according to the first aspect of the application; obtaining a signal segment in the linear data buffer according to the pointer; and executing a sound source positioning algorithm on the signal segment to calculate the direction of arrival of the sound source.

[0032] Further, the sound source positioning algorithm specifically comprises: processing the transient sound event segment in the linear data buffer based on an energy centroid method, and further dynamically refining an analysis window; reading an R-microphone pair optimal subset constructed by a first offline optimization method, using the data signal segment in the analysis window after accurate segmentation to calculate a time difference of arrival (TDOA); and solving the direction of arrival (DOA) according to the calculated R TDOA values and a constant matrix calculated in advance by a second offline optimization method.

[0033] Further, the first offline optimization method is a microphone pair optimization strategy based on a genetic algorithm, specifically comprising: encoding all possible microphone pair combinations in the microphone array into binary chromosomes; constructing a fitness function that comprehensively considers positioning accuracy and computational complexity to evaluate the advantages and disadvantages of the microphone pair combination represented by each chromosome; iteratively evolving a chromosome population by repeatedly performing selection, crossover and mutation operations to generate offspring with higher fitness; and when the iteration meets a preset termination condition, selecting the chromosome with the highest fitness in the entire evolution process as the optimal solution and decoding it into the microphone pair optimal subset.

[0034] Further, the second offline optimization method specifically comprises:

[0035] Establishing a physical model: for an array composed of M microphones, the TDOA τ1 klfor:

[0036]

[0037] Where c is the speed of sound, P k and P l are the position vectors of the kth and lth microphones respectively, and u is the unit vector in the direction of the sound source;

[0038] Constructing a linear equation system: Based on the R pairs of microphones selected by the first offline optimization method, R equations of the above form can be established. These equations are written in matrix form to obtain an overdetermined linear equation system:

[0039] τ1=Gp

[0040] where the time delay vector τ1 is composed of the TDOA measurements of R microphone pairs, and p = [cosφ sinφ] T is the sound source position vector to be solved, φ is the direction of arrival; G is an M×2 geometric matrix; p is solved by the least squares method:

[0041] p=(G T G) -1 G T τ1

[0042] Determine the constant matrix: In p=(G T G) -1 G T In τ1, since the geometric matrix G is a fixed constant, the matrix A=(G T G) -1 G T is a constant matrix that can be precomputed.

[0043] According to a fourth aspect of the present invention, a real-time positioning system for transient sound based on a microphone array is provided, comprising: a sound source positioning task module for receiving a message body including a pointer to a linear data buffer and its data length obtained by the method described in the first aspect; obtaining a signal segment in the linear data buffer according to the pointer; and executing a "sound source positioning algorithm" on the signal segment to calculate the direction of arrival of the sound source.

[0044] The beneficial effects of the present invention are:

[0045] 1) The detection algorithm of the present invention combines a dynamic threshold with a finite state machine, which can effectively adapt to the dynamic changes of background noise and maintain a high detection rate (over 91%) and a low false alarm rate (below 5%) under low signal-to-noise ratio and non-stationary noise conditions;

[0046] 2) In the localization method of the present invention, through the acceleration strategy of microphone pair optimization and constant matrix pre-calculation, the average time required for single sound source localization of an 8-channel microphone array is reduced from over 55ms to approximately 26ms, improving computational efficiency by approximately 55%, fully meeting the real-time processing requirements of embedded systems;

[0047] 3) The overall solution of the present invention is based on a low-cost micro-electromechanical system (MEMS) microphone and a general-purpose microcontroller, without relying on expensive PCs or dedicated DSP chips. It provides a low-cost, high-efficiency, and easy-to-deploy transient acoustic event monitoring technology solution for application scenarios such as security monitoring and industrial inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 Schematic diagram of a finite state machine state transition and noise parameter update strategy for transient sound detection in an embodiment of the present invention.

[0050] Figure 2 4 is a flowchart of microphone pair optimization based on genetic algorithm in an embodiment of the present invention.

[0051] Figure 3 Schematic diagram of the hardware connection between the microphone array, WiFi module and microcontroller in an embodiment of the present invention.

[0052] Figure 4 It is a task framework diagram of the system software in an embodiment of the present invention.

[0053] Figure 5 3 is a diagram showing the recognition effect of the transient sound detection algorithm in the embodiment of the present invention under a non-stationary noise background.

[0054] Figure 6 3 is a comparison diagram of the positioning errors of the transient acoustic positioning algorithm at different angles before and after optimization in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other in any way.

[0056] Example: Figures 1-6 As shown, this embodiment provides a method and system for real-time detection and localization of transient sounds based on a microphone array. The system is built on an embedded platform, and its hardware platform and software framework are described below.

[0057] Step 1: Construction of system hardware platform

[0058] Reference Figure 3 , construct the hardware system of the present invention, which is mainly composed of a data acquisition module, a data processing module and a data communication module.

[0059] Step 1.1: Data Acquisition Module

[0060] A circular microphone array consisting of eight TDK InvenSense ICS-52000 digital MEMS microphones is used. The microphones are connected in a daisy-chain fashion, sharing the serial clock (SCK) and frame synchronization (WS) signals from the microcontroller to ensure synchronous acquisition of multi-channel data. The serial data (SD) output pins of the eight microphones are connected to the data input pins of the same serial audio interface (SAI) of the microcontroller, utilizing time-division multiplexing (TDM) technology to transmit eight channels of audio signals on a single data line.

[0061] Step 1.2: Data processing module

[0062] An STMicroelectronics STM32H743VIT microcontroller was selected as the core processor. Based on the ARM Cortex-M7 core, this microcontroller boasts a 480MHz clock speed, a built-in double-precision floating-point unit, and a digital signal processing extension instruction set, providing the necessary computing power for subsequent signal processing algorithms. The microcontroller's SAI1 peripheral is configured to receive the TDM data stream from the microphone array.

[0063] Step 1.3: Data Communication Module

[0064] The ESP8266 Wi-Fi module is selected and connected to the microcontroller through a universal asynchronous receiver / transmitter (UART). This module is responsible for wirelessly transmitting information such as the positioning results of the data processing module to the host computer, or receiving configuration instructions from the host computer.

[0065] Step 2: Design and implementation of the system software framework

[0066] Reference Figure 4 On the hardware platform built in step 1, software development is carried out based on the FreeRTOS real-time operating system, ensuring the real-time performance and multi-task parallel processing of the system in a task-based manner. The software execution process is as follows:

[0067] Step 2.1: Perform audio acquisition tasks

[0068] This task is given the highest priority. It utilizes the microcontroller's Direct Memory Access (DMA) to continuously acquire 8 channels of TDM audio data from the SAI interface in ping-pong buffering mode. When a buffer is full, an interrupt service routine is triggered. Within the interrupt, the original 24-bit integer is converted to a floating-point number and the processed data is placed in a ring buffer, which acts as a producer for subsequent tasks to consume.

[0069] Step 2.2: Perform transient sound detection task

[0070] This task has a high priority. As a consumer of the ring buffer, it is responsible for continuously reading audio data from the ring buffer and performing the following processing steps in sequence:

[0071] Step 2.2.1: Data deinterleaving

[0072] The read one-dimensional audio data in TDM format is deinterleaved, and the original multi-channel interleaved data is reorganized into 8 independent channel data.

[0073] Step 2.2.2: Sliding window framing

[0074] To ensure complete capture of transient acoustic events, an overlapping sliding window strategy is used to synchronously frame the eight-channel data streams obtained in step 2.2.1. Specifically, the sliding window length (i.e., the data frame length) is set to 1024 samples, and the window shift step (i.e., the frame shift) is set to 512 samples. This method continuously generates a series of partially overlapping data frames. Each data frame is a two-dimensional data matrix with dimensions of 1024 × 8, which serves as a single input to the subsequent transient acoustic detection algorithm.

[0075] Step 2.2.3: Perform single-channel state identification

[0076] For each channel in the data frame from step 2.2.2, perform the following operations independently:

[0077] Step 2.2.3.1: Real-time update of noise statistics

[0078] The noise statistics of each audio channel data frame are updated in real time using an exponentially weighted moving average model:

[0079]

[0080] in, is the energy of the t-th frame of channel i, α∈(0,1) is the forgetting factor, which determines the degree of influence of historical data on the current noise estimate; and Respectively represent the energy mean and standard deviation of the t-th frame of channel i. The noise statistics include the energy mean and standard deviation; the data frame starts counting from 1, and and Assign a value of 0.001,

[0081] Step 2.2.3.2: Adaptive detection threshold calculation

[0082] Based on the real-time updated energy mean and standard deviation, the adaptive detection threshold T t i Defined as:

[0083]

[0084] in, is the sensitivity coefficient, which is used to control the offset of the detection threshold relative to the noise level. Its value needs to be calibrated according to the actual application scenario and the requirements for detection sensitivity. t i represents the adaptive detection threshold of the t-th frame of channel i.

[0085] Step 2.2.3.3: State identification based on finite state machine

[0086] Reference Figure 1 In order to more accurately capture the dynamic characteristics of transient sound, a four-state finite state machine containing "silence", "energy rise", "energy persistence" and "energy decline" is designed for each channel, where η1 and η2 represent the gradient threshold factors of energy rise and decline, respectively, x represents the number of frames of state persistence, and x r and x f Represents the minimum number of frames for the rising and falling states to continue. With the adaptive detection threshold T t i Comparison results of energy gradient The size and duration of the state are determined by the state transition between the four states. The detailed state definitions and transition conditions are as follows:

[0087] (1) Silent state: This is the initial state of each channel, indicating that there is no transient sound event. In this state, the noise statistics are completely updated according to S1. and when and , it transfers to the "energy rising" state; otherwise, it does not transfer; Represents the energy gradient of the tth frame of channel i, that is, the difference between the energy of the tth frame and the energy of the t-1 frame. For example, for the state of the first frame of data frame, when the virtual frame is in silence, the noise statistics are completely updated according to step 2.2.3.1 and Based on this, for each channel of the first frame, when and , the state of channel i in the first frame is transferred to the “energy rising” state; otherwise, it does not transfer.

[0088] (2) Energy Rising State: Indicates that the signal energy exceeds the threshold and is on an upward trend. In this state, the update of the noise statistics is frozen. Based on the energy gradient and the number of continuous frames, it is transferred to the "Energy Continuous" state or the "Silent" state. Specifically:

[0089] if And x ≥ x r , transfer to the "energy persistence" state;

[0090] if Move to "silent" state;

[0091] Otherwise, no transfer;

[0092] In the above, the update of the frozen noise statistics means that in the energy rising state, for “ and ” respectively with “ and "same.

[0093] (3) Energy persistence state: corresponds to the peak or plateau period of transient sound energy. In this state, the update of noise statistics is frozen. According to the energy gradient and the number of continuous frames, it is transferred to the "energy decrease" state. Specifically:

[0094] if And x ≥ x f , indicating that the energy begins to decrease steadily and shifts to the “energy decrease” state;

[0095] Otherwise, no transfer.

[0096] (4) Energy Decline State: This indicates that the transient sound is coming to an end. In this state, the noise statistics are partially updated. Depending on the energy and energy gradient, the state transitions to the Energy Rise or Silence state.

[0097] if Shift to an "energy up" state;

[0098] if Move to "silent" state;

[0099] Otherwise, no transfer.

[0100] The partial updating of noise statistics specifically includes: updating the energy mean and freezing the energy standard deviation.

[0101] Step 2.2.4: Perform multi-channel comprehensive judgment and data packaging

[0102] Step 2.2.4.1: Multi-channel comprehensive decision

[0103] The state recognition results obtained from step 2.2.3 for all channels are integrated to reliably detect transient acoustic events and preliminarily define their start and end boundaries. The specific process is as follows:

[0104] Step 2.2.4.1.1: When the system starts, set the initial state of all channels to "silent state" and assign preset initial values ​​to their noise statistics.

[0105] Step 2.2.4.1.2: For each data frame, after the current state of each channel is determined by the finite state machine in step 2.2.3, the system counts the number of channels that are not in the "silent state" (i.e., in one of the "energy rising", "energy continuing" or "energy falling" states) in real time, and records it as N. active .

[0106] Step 2.2.4.1.3: Will result in N active The value is less than the preset channel number threshold N th The state changes to be greater than or equal to N th The data frame of the state is defined as the "start frame" of a valid transient sound event. This "start frame" marks the beginning of a new transient sound event. In this embodiment, the preset channel number threshold N th The value of is 4.

[0107] Step 2.2.4.1.4: After detecting the "start frame," the system continues processing subsequent data frames and monitoring the status of each channel. The first data frame after the "start frame" in which all channels return to a "silent state" is defined as the "end frame" of the valid transient acoustic event.

[0108] Step 2.2.4.2: Event data packaging and sending

[0109] Step 2.2.4.2.1: Dynamically allocate a linear data buffer large enough to accommodate all multi-channel audio data from the start frame to the end frame.

[0110] Step 2.2.4.2.2: Copy the complete signal segment corresponding to the transient acoustic event in the main ring buffer to the newly allocated linear data buffer.

[0111] Step 2.2.4.2.3: Construct a message body containing a pointer to the newly allocated linear data buffer and its data length. Then, send this message body to the "Sound Source Localization Task" through the FreeRTOS task message queue.

[0112] Otherwise, continue to perform this detection task on the next data frame.

[0113] Step 2.3: Perform the sound source localization task

[0114] The priority of this task is set to medium. After the task is created, it enters a blocked state to wait for the message of the transient sound detection task. When there is a message in the task message queue, the task is awakened and obtains the message body from the queue to obtain the pointer to the linear data buffer containing a complete transient sound event data and its data length. The "sound source localization algorithm" is executed on the received linear data buffer to calculate the direction of arrival DOA of the sound source. After completing all calculations, the task is responsible for releasing the memory of the linear data buffer allocated for this event to prevent memory leaks. The sound source localization algorithm is specifically as follows:

[0115] Step 2.3.1: Refinement of transient acoustic event segments based on energy centroid

[0116] In order to further extract the core part with the most concentrated energy from the rough event fragment obtained in step 2.2, this step is performed directly on the linear data buffer received by the sound source localization task:

[0117] Step 2.3.1.1: Calculate the total energy of each of the eight channels in the linear data buffer and determine the channel with the largest total energy value as the "reference channel."

[0118] Step 2.3.1.2: Apply the energy centroid formula to calculate the data of the reference channel and determine the time point t when the energy is most concentrated. c Energy center of mass t c Defined as:

[0119]

[0120] Where s(n) is the amplitude of the data segment, n is the sampling point index in the data segment, and N1 and N2 are the start and end indexes of the data segment, respectively.

[0121] Step 2.3.1.3: Take the calculated energy center of mass t c As the center, intercept a fixed length analysis window as the input of the subsequent TDOA estimation. Let the starting index of the linear data buffer be N start , the end index is N end , the preset length of the analysis window is L win The final starting index W of the analysis window start and end index W end Determined by:

[0122]

[0123] Step 2.3.2: Estimation of Time Difference of Arrival (TDOA)

[0124] Step 2.3.2.1: Read the preferred subset of microphone pairs generated by the first offline optimization method from the Flash memory of the microcontroller.

[0125] Step 2.3.2.2: For the R pairs of microphones in the subset, use the signal segments accurately segmented in step 2.3.1 to execute the GCC-PHAT algorithm to calculate the arrival time difference TDOA. For the microphone pair (i, j), its arrival time difference τ ij The calculation formula is:

[0126]

[0127] in, represents the inverse fast Fourier transform, X i (f) and X j (f) are the fast Fourier transforms of the two microphone signals, (·) * represents the complex conjugate.

[0128] Step 2.3.3: Direction of Arrival (DOA)

[0129] Step 2.3.3.1: Combine the R TDOA values ​​calculated in step 2.3.2 into a time difference of arrival vector τ.

[0130] Step 2.3.3.2: Obtain the constant matrix A pre-calculated by the second offline optimization method from the Flash memory and perform a multiplication operation between the constant matrix A and the real-time TDOA vector τ: You can quickly get the position vector

[0131] Step 2.3.3.3: Using the inverse tangent function From the position vector Calculate the final sound source azimuth in,

[0132] The offline optimization method includes two steps that are completed before the system is running, and the results are stored in the microcontroller's Flash to accelerate real-time positioning calculations. Specifically:

[0133] The first offline optimization method: Genetic algorithm-based optimal subset generation of microphone pairs

[0134] Reference Figure 2 This step is achieved by running a genetic algorithm process, aiming to select an optimal subset from all possible microphone pairs, where i is a loop counter variable, N p Represents the population size. The specific implementation is as follows:

[0135] (1) Chromosome encoding: For an 8-channel microphone array, there are a total of C(8,2) = 28 possible microphone pairs. A binary string of length 28 is used as the chromosome, where each bit corresponds to a unique microphone pair. If the bit is "1", it indicates that the corresponding microphone pair is selected for calculation; if it is "0", it is not selected.

[0136] (2) Fitness function design: In order to optimize both positioning accuracy and computational complexity, the following comprehensive fitness function is designed to evaluate the quality of each chromosome:

[0137]

[0138] Among them, E avg It is the average sound source azimuth angle error obtained by simulation calculation under multiple preset sound source azimuth angles, N pairs is the number of microphone pairs represented by the chromosome, a measure of computational effort. w1 and w2 are weight coefficients used to balance the importance of accuracy and computational effort. A higher fitness function value indicates a better combination.

[0139] (3) Genetic algorithm iteration: First, an initial population containing multiple chromosomes is randomly generated. Then, by repeatedly performing selection, crossover, and mutation standard genetic algorithm operations, new offspring populations are constantly generated, and in each generation, chromosomes with higher fitness are retained and propagated.

[0140] (4) Optimal solution decoding: When the algorithm reaches the preset maximum number of iterations or the average fitness of the population converges, the chromosome with the highest fitness throughout the evolutionary process is selected as the optimal solution. Decoding this binary string yields the optimal microphone pair combination, i.e., the microphone pair preferred subset. In this embodiment, 12 microphone pairs are ultimately selected for subsequent real-time computation.

[0141] Second offline optimization method: Constant matrix pre-computation

[0142] This method aims to pre-compute a constant matrix for solving the sound source position based on the fixed geometry of the microphone array, thereby simplifying the complex computation in the real-time positioning stage to a single matrix multiplication. The derivation process is as follows:

[0143] (1) Establish a physical model: In the far-field condition where the distance between the sound source and the microphone array is much greater than the array aperture, the sound wave can be approximated as a plane wave. For an array composed of M microphones, the TDOA τ1 kl between microphone k and microphone l is:

[0144]

[0145] where c is the speed of sound, P k and P l are the position vectors of the kth and lth microphones, respectively, and u is the unit vector of the sound source direction.

[0146] (2) Construct a linear equation system: For DOA estimation in a two-dimensional plane, the R microphone pairs selected in the first offline optimization method can establish R equations of the above form. Writing these equations in matrix form yields an overdetermined linear equation system:

[0147] τ1 = Gp

[0148] where the delay vector τ1 is composed of the TDOA measurements of the R microphone pairs, p = [cos φ sin φ] T is the sound source position vector to be solved, and φ is the direction of arrival; G is a M x 2 geometric matrix whose specific form is determined by the array geometry and the selection of microphone pairs. p can be solved by the least squares method:

[0149] p = (G T G) -1 G T τ1

[0150] (3) Determining constant matrix: in the above formula, since the geometric matrix G is a constant, the matrix A = (G T G) -1 G T is a constant matrix that can be calculated in advance. The matrix is calculated in the offline stage and stored in the form of an array in the Flash memory of the microcontroller.

[0151] Step 2.4: Perform human-computer interaction task

[0152] This task has a low priority. Its function is to send the azimuth angle result calculated by the sound source positioning task to the host computer through the ESP8266 module, and process the configuration commands from the host computer.

[0153] In order to verify the effectiveness of the method and system described in the present application, simulation and physical experiments are carried out.

[0154] (1) Transient sound detection performance verification

[0155] In order to evaluate the performance of the transient sound detection algorithm based on dynamic threshold and finite state machine proposed in the present application, a simulation experiment is designed. First, record a clean audio containing complete transient events (handclap sound) as a reference signal. Then, splice the signal repeatedly and add Gaussian white noise with different signal-to-noise ratios, and dynamically change the noise level of some segments to simulate non-stationary noise environment. At the same time, randomly amplify or attenuate the amplitude of some transient signals to simulate transient sound events with different strengths.

[0156] Detection rate (DR) and false alarm rate (FAR) are used as evaluation indicators. The detection rate is defined as the ratio of the number of correctly detected transient events to the total number of transient events. The false alarm rate is defined as the ratio of the number of non-transient periods incorrectly identified as transient events to the total number of non-transient periods.

[0157] Table 1 Comparison of transient sound detection algorithm performance under different simulation conditions

[0158] Simulation conditions DR (%) FAR (%) Average SNR = 5dB, transient sound amplitude fluctuates randomly 88.76 7.47 Average SNR = 0dB, transient sound amplitude fluctuates randomly 86.33 8.52 Average SNR = -5dB, some transient sounds are attenuated 85.23 6.20 SNR dynamically changes from -5dB to 10dB 91.12 4.25

[0159] From Table 1, it can be seen that the algorithm of the present application performs well under various challenging conditions. Especially in non-stationary noise with dynamic SNR, the detection rate reaches more than 91%, and the false alarm rate is less than 5%, which proves the strong adaptability of the algorithm to environmental noise changes.

[0160] Figure 5The recognition effect of the algorithm under simulation condition 4 (SNR dynamically changes between -5dB and 10dB) is shown. The blue solid line in the figure is the noise signal, and the yellow dashed line marks the transient sound interval recognized by the algorithm. It can be seen that even in the case of fluctuating background noise and varying transient sound energy, the algorithm can accurately capture most transient events.

[0161] (2) Sound source positioning performance and real-time verification

[0162] To verify the effect of the acceleration strategy of the sound source positioning algorithm and the overall positioning ability of the system, simulation and physical experiments are performed.

[0163] Simulation verification: The RIR-Generator software is used to simulate a semi-anechoic chamber environment, the sound source is 3 meters away from the array center, and the input signal-to-noise ratio is 10dB. The sound source azimuth angle is tested from 0° to 330° (interval 30°) for a total of 12 angles. Figure 6 The average absolute error (Mean Absolute Error, MAE) of the positioning of the two schemes using all 28 microphones and the 12 microphones selected by the application is compared.

[0164] As shown in Figure 6 , the positioning error of the scheme using the optimized 12 microphones (red curve, MAE = 5.28°) is very close to the scheme using all 28 microphones (blue curve, MAE = 5.21°) at each angle, and the performance is almost not lost. This proves that the microphone selection strategy of the application can effectively maintain the positioning accuracy while greatly reducing the computational load.

[0165] Physical experiment and real-time analysis: In the anechoic chamber, the sound source is placed at a distance of 1.2 meters, 2.4 meters and 3 meters from the array center for testing. The system positioning performance is shown in Table 2.

[0166] Table 2 Sound source positioning performance at different distances

[0167] Distance (m) MAE(°) 1.2 6.15 2.4 9.32 3.0 15.70

[0168] The results show that the system maintains high positioning accuracy within a 3-meter effective working distance.

[0169] In terms of real-time performance, a key requirement of the application is that the data processing speed must be faster than the data generation speed to avoid data loss. The system sampling rate is 48kHz, with 1024 points per frame, and a double-frame ring buffer is used. It takes 42.66ms to fill the double-frame buffer, so the processing time of each frame of data must be less than 42.66ms. Through the task runtime statistics function of FreeRTOS, it is measured that:

[0170] Table 3 Average execution time of core tasks

[0171] Task Name Average execution time (ms) Audio acquisition and preprocessing 0.81 Transient sound detection 1.28 Sound source localization 24.25 total 26.34

[0172] The average processing time from detection to positioning output is approximately 26ms, far less than the real-time requirement of 42.66ms. In comparison, running an unoptimized positioning algorithm on the same hardware platform (using all 28 microphone pairs) results in an average single positioning time of approximately 55.62ms, which falls short of real-time processing requirements. The acceleration strategy presented in this paper improves computational efficiency by approximately 55%, successfully addressing the real-time bottleneck of embedded platforms.

[0173] By applying the above technical solution, it can be seen that the method of the present invention includes: 1) real-time detection: by combining real-time updated noise statistics with a multi-state finite state machine, multi-channel audio data is processed to robustly detect transient sound events and preliminarily define their boundaries; 2) real-time positioning: for the preliminarily defined event fragments, the energy centroid method is further used to dynamically refine the analysis window, and the direction of the sound source is efficiently solved by combining the preferred subset of microphone pairs and the constant matrix pre-generated in the offline optimization stage. The system of the present invention is based on an embedded real-time operating system, and through multi-task division and inter-task message queues, reliable real-time collaboration of detection and positioning tasks is achieved. The present invention effectively addresses the challenge of poor detection robustness under low signal-to-noise ratio and non-stationary noise, and overcomes the problem of limited computing resources of embedded platforms, providing a low-cost, high real-time, and high-accuracy transient sound monitoring solution.

[0174] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A method for real-time detection of transient sound based on a microphone array, characterized in that: include: Audio acquisition step: Collect multi-channel audio data in a time division multiplexing manner to obtain one-dimensional raw audio data in TDM format; Transient sound detection steps: Deinterleaving the one-dimensional original audio data to reorganize the multi-channel interleaved original data into independent channel data streams; The overlapping sliding window strategy is used to synchronously frame the acquired channel data stream; Each generated data frame is input into the "Transient Sound Detection Algorithm" for transient sound detection. If the "Transient Sound Detection Algorithm" comprehensively determines the rough boundary of a transient sound event, the following key operations are performed: a linear data buffer is dynamically allocated; the complete fragment of the transient sound event in the main ring buffer is copied to the linear data buffer; a message body is constructed, which contains a pointer to the linear data buffer and its data length; otherwise, the transient sound detection step continues for the next data frame.

2. The method for real-time detection of transient sound based on a microphone array according to claim 1, wherein: The transient sound detection algorithm is specifically: The noise statistics of each audio channel data frame are updated in real time using an exponentially weighted moving average model; Calculate the adaptive detection threshold based on the real-time updated noise statistics; Construct a finite state machine and perform single-channel state recognition based on the noise statistics and adaptive detection threshold; A comprehensive decision is performed based on the multi-channel status to preliminarily define the boundaries of transient acoustic events.

3. The method for real-time detection of transient sound based on microphone array according to claim 2, characterized in that: The single-channel state recognition is performed based on the finite state machine according to the noise statistics and the adaptive detection threshold, specifically: A four-state finite state machine including "silent", "energy up", "energy sustained" and "energy down" is designed for each channel; The status definitions and transition conditions are as follows: (1) Silent state: In this state, the noise statistics are completely updated and when and When , it transfers to the "energy rising" state; otherwise, it does not transfer; and They represent the mean and standard deviation of the energy of the t-th frame of channel i respectively; represents the energy of the t-th frame of channel i; represents the energy gradient of channel i in the tth frame; η1 represents the gradient threshold factor of energy rise; (2) Energy rising state: In this state, the update of noise statistics is frozen; if And x ≥ x r , transfer to the "energy continuation" state; if Transfer to "silent" state; otherwise, do not transfer; x represents the number of frames the state lasts, x r Indicates the minimum number of frames that the rising state lasts; (3) Energy persistence state: In this state, the update of noise statistics is frozen; if And x ≥ x f , transfer to the "energy down" state; otherwise, do not transfer; where η2 represents the gradient threshold factor of energy down, x f Indicates the minimum number of frames that the falling state lasts; (4) Energy reduction: In this state, the noise statistics are partially updated; if Transfer to "Energy Up" state; if Transfer to "quiet" state; otherwise, do not transfer.

4. The method for real-time detection of transient sound based on a microphone array according to claim 2, wherein: The comprehensive judgment is performed based on the multi-channel status to preliminarily define the boundary of the transient sound event, specifically: Set the initial state of all channels to "silent state" and assign preset initial values ​​to their noise statistics; For each data frame, after the current state of each channel is determined by the finite state machine, the number of channels that are not in the "silent state" is counted in real time, recorded as N active ; will result in N active The value is less than the preset channel number threshold N th The state changes to be greater than or equal to N th The data frame of the state is defined as the "start frame" of a valid transient sound event; After the "start frame" is detected, the first data frame in which all channels return to the "silent state" is defined as the "end frame" of the valid transient sound event.

5. A real-time transient sound detection system based on a microphone array, characterized in that: include: An audio collection task module, configured to execute the audio collection step in claim 1; The transient sound detection task module is used to perform the transient sound detection step in claim 1.

6. A method for real-time transient sound positioning based on a microphone array, characterized in that: include: Sound source localization steps: Receive a message body including a pointer to a linear data buffer and its data length obtained according to the method of claim 1; According to the pointer, obtaining the signal segment in the linear data buffer; A "sound source localization algorithm" is executed on the signal segment to calculate the direction of arrival of the sound source.

7. The method for real-time transient sound positioning based on a microphone array according to claim 6, wherein: The sound source localization algorithm is specifically: Based on the energy centroid method, the transient acoustic event segments in the linear data buffer are processed to further dynamically refine an analysis window; Reading a preferred subset of microphone pairs constructed by R pairs of microphones generated by the first offline optimization method, and calculating a time difference of arrival (TDOA) using data within the analysis window; The direction of arrival (DOA) is solved based on the calculated R TDOA values ​​and the constant matrix pre-calculated by the second offline optimization method.

8. The method for real-time transient sound positioning based on a microphone array according to claim 7, wherein: The first offline optimization method is a microphone pair optimization strategy based on a genetic algorithm, specifically: Encoding all possible microphone pair combinations in the microphone array into binary chromosomes; Constructing a fitness function that combines positioning accuracy and computational effort to evaluate the quality of the microphone pair combination represented by each chromosome; By repeatedly performing selection, crossover, and mutation operations, a chromosome population is iteratively evolved to produce offspring with higher fitness; When the iteration satisfies a preset termination condition, the chromosome with the highest fitness in the entire evolution process is selected as the optimal solution and decoded as the preferred subset of microphone pairs.

9. The method for real-time transient sound positioning based on a microphone array according to claim 7, wherein: The second offline optimization method is specifically: Establish a physical model: For an array consisting of M microphones, the TDOAτ1 between microphone k and microphone l kl for: Where c is the speed of sound, P k and P l are the position vectors of the kth and lth microphones respectively, and u is the unit vector in the direction of the sound source; Constructing a linear equation system: Based on the R pairs of microphones selected by the first offline optimization method, R equations of the above form can be established. These equations are written in matrix form to obtain an overdetermined linear equation system: τ1=Gp where the time delay vector τ1 is composed of the TDOA measurements of R microphone pairs, and p = [cosφsinφ] T is the sound source position vector to be solved, φ is the direction of arrival; G is an M×2 geometric matrix; p is solved by the least squares method: p=(G T G) -1 G T τ1 Determine the constant matrix: In p=(G T G) -1 G T In τ1, since the geometric matrix G is a fixed constant, the matrix A=(G T G) -1 G T is a constant matrix that can be precomputed.

10. A transient sound real-time positioning system based on a microphone array, characterized in that: include: A sound source localization module, configured to execute the sound source localization step described in claim 6.

Images