Direction of arrival estimation method, system and equipment based on microphone array and medium

Abstract

The invention provides a direction-of-arrival estimation method, system and device based on a microphone array and a medium, and belongs to the field of signal processing, and the method comprises the steps: obtaining a target sound source signal; converting the target sound source signal into a GCC-PHAT feature image and a TFD feature image; carrying out weighted fusion on the two feature images, inputting the obtained fused feature image into an improved MLLA network, and carrying out convolutional embedding and serialization on the fused feature image by using a convolutional Stem; the sequence is sent to multiple stages of Basic Layers for feature extraction, and down-sampling and channel expansion are completed after each Basic Layer layer is finished; the tail end of the network applies normalization to an output sequence, and the output sequence is converged into global representation through self-adaptive one-dimensional average pooling; and processing the global representation through a classification head, and outputting a DOA estimation result. According to the method, the robustness of DOA estimation to noise interference and the overall angle estimation precision can be greatly improved.

Description

Technical Field

[0001] This invention belongs to the field of signal processing technology, specifically relating to a method, system, device, and medium for direction-of-arrival estimation based on a microphone array. Background Technology

[0002] Direction of Arrival (DOA) estimation is a crucial research area in array signal processing. Microphone arrays combined with DOA estimation can achieve sound source localization, finding wide applications in radar, sonar, communications, and speech processing. Its essential goal is to determine the azimuth angle of a sound source signal arriving at each element of the array. Accurately estimating the channel's angle of arrival from a noisy background environment is one of the main tasks in DOA estimation algorithms. In real-world environments, noise types are diverse, and signal-to-noise ratios (SNRs) range are wide and difficult to define: there exist both quiet environments with high SNR and complex environments with low SNR, significantly increasing the difficulty of DOA estimation. Existing DOA estimation algorithms exhibit relatively ideal performance in relatively quiet environments (such as inside a train car, a quiet bedroom, or an open field). However, they struggle to meet the requirements for estimation accuracy and robustness in complex environments with low signal-to-noise ratios (SNR), such as workshops, restaurants, and karaoke bars. This is because when the SNR is too low, noise masks the signal's time delay characteristics, leading to significant deviations in time delay estimation. Furthermore, it reduces the clarity of the peaks representing the sound source direction in the spatial spectrum; these peaks can even be masked by spurious noise peaks. In subspace algorithms, it also disrupts the orthogonality between the signal and noise subspaces, further affecting the accuracy of spatial features and resulting in misjudgments and low accuracy in DOA estimation. Therefore, traditional DOA estimation algorithms perform poorly when directly applied to complex indoor environments.

[0003] Traditional DOA estimation methods primarily achieve DOA estimation by establishing a mathematical model of the signal arrival array. Representative algorithms include Capon's minimum variance beamformer, TDOA-based localization methods, and subspace algorithms such as MUSIC and ESPRIT, among other improved MUSIC algorithms. These traditional algorithms can achieve high-precision estimation under ideal conditions, approaching the theoretical performance limit. However, traditional algorithms are highly sensitive to signal-to-noise ratio (SNR), performing excellently at high SNRs. Under unfavorable conditions such as low SNR or model mismatch, estimation performance significantly degrades, and computational complexity increases significantly. Furthermore, subspace algorithms involve complex matrix decomposition, resulting in high computational complexity, and accuracy is further reduced when non-ideal factors such as array element coupling, amplitude and phase errors, or array element position errors exist. While many improved algorithms have enhanced DOA estimation accuracy, they still cannot solve the multi-source localization problem in low SNR environments.

[0004] In recent years, deep learning technology has been widely applied to DOA estimation, employing a combination of data-driven and model-driven approaches. Common architectures include multilayer perceptrons, convolutional neural networks, and combinations of traditional algorithms with deep learning. These methods have demonstrated good performance in scenarios with low signal-to-noise ratios (SNR), a limited number of sound sources, and the processing of coherent sources. However, excessively low SNR can lead to noise power far exceeding signal power, masking signal characteristics and making it difficult to learn key signal features. Furthermore, low SNR can result in overfitting in high-dimensional state spaces, limiting the estimation stability when applied to DOA estimation. Summary of the Invention

[0005] To address the issues of low accuracy, robustness, and stability in existing direction-of-arrival (DOA) estimation methods, this invention provides a microphone array-based DOA estimation method.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A direction-of-arrival estimation method based on a microphone array includes: The target sound source signal in its environment is acquired by a microphone array arranged in a ring. The target sound source signal is converted into a cross-correlation-phase transform (GCC-PHAT) matrix and a time-frequency domain (TFD) matrix. The GCC-PHAT matrix and the TFD matrix are then converted into GCC-PHAT feature images and TFD feature images, respectively. The GCC-PHAT feature images and the TFD feature images are then weighted and fused to obtain a fused feature image. The fused feature image is input into an improved MLLA network, which is based on the original MLLA network. The improved MLLA network introduces a dynamic forget gate mechanism and a position bias design into the linear attention mechanism module of each MLABlock of the original MLLA network. The fused feature image is convolutionally embedded and serialized through the convolutional Stem module of the improved MLLA network to generate a one-dimensional embedding sequence. The one-dimensional embedding sequence is fed into a multi-level BasicLayer module. The multi-level BasicLayer extracts high-dimensional semantic features related to time delay, frequency domain energy, and spatial location from the fused feature image through stacked MLABlocks. The high-dimensional semantic feature sequence output by each level of BasicLayer is downsampled and channel-expanded in the spatial dimension. The downsampled and channel-expanded features are converged into a global representation vector through adaptive one-dimensional average pooling. The global representation is processed by a classification head to output the direction of arrival (DOA) estimation result.

[0007] Preferably, obtaining high-dimensional semantic features through a multi-level BasicLayer module includes the following steps: A one-dimensional embedded sequence is input into the first BasicLayer and normalized by its normalization layer. The normalized sequence is then fed into an MLLABlock stacked from the first BasicLayer. The MLLABlock first normalizes the input and performs a linear transformation to generate Q, K, and V. Kernel functions are applied to Q and K, and positional encoding is applied to introduce positional information. A linear transformation is performed on V to generate a preliminary representation with a forgetting gate. The forgetting coefficient is obtained by combining the positional bias with Sigmoid activation, and the V vector is updated by gating. A learnable positional bias is introduced when calculating the linear attention output, and LePE is superimposed to reinforce local features. The feature extraction of a single MLLABlock is completed through residual connections and MLP nonlinear transformation. After all MLABlocks in the first layer have been processed, the first layer output sequence is generated by downsampling and expanding the channels through PatchMerging. The output sequence is used as the input of the next layer, BasicLayer, and features are extracted layer by layer. After the last BasicLayer completes feature extraction, the final high-dimensional semantic feature sequence is output.

[0008] Preferably, converting the GCC-PHAT matrix into a GCC-PHAT feature image specifically includes the following steps: Normalize the signals in the GCC-PHAT matrix; The time delay is calculated using the GCC-PHAT algorithm on the normalized signal matrix to obtain the time delay feature matrix, which is then converted into a GCC-PHAT feature image.

[0009] Preferably, converting the TFD matrix into a TFD feature image specifically includes the following steps: Perform a short-time Fourier transform on the TFD matrix to obtain the complex time-frequency representations of each element in the time and frequency dimensions; The covariance matrix of the array output vector is calculated based on each frequency point and time frame, and accumulated and averaged over all frequency and time dimensions to obtain the overall time-frequency domain covariance feature matrix. The real, imaginary, and phase information of the time-frequency domain covariance feature matrix are mapped to the three RGB channels respectively. The phase part is normalized and mapped to the [0,1] interval. The entire three-channel matrix is ​​then globally normalized and finally adjusted to a uniform size using bilinear interpolation to obtain the TFD feature image.

[0010] Preferably, the fused feature image includes temporal delay characteristics and spatial statistical features, and the fusion process is represented as follows: ; In the formula, Represents the fused feature image. Represents the GCC-PHAT feature image. Represents the TFD feature image. and These represent the weights of the GCC-PHAT feature image and the TFD feature image, respectively.

[0011] Preferably, the step of performing convolutional embedding and serialization of the fused feature image by the convolutional Stem module specifically involves: The fused feature image is subjected to three convolutional segments to extract low-level features and then spatial downsampling to obtain the feature map; Flattening and transposing the feature map yields a length of... The sequence representation.

[0012] Preferably, the step of feeding the one-dimensional sequence into a multi-level BasicLayer for feature extraction specifically involves: The present invention also provides a direction-of-arrival estimation system based on a microphone array, comprising: The data acquisition module is used to acquire target sound source signals in its environment through a circular array of microphones. The data conversion module is used to convert the target sound source signal into a cross-correlation-phase transform (GCC-PHAT) matrix and a time-frequency domain (TFD) matrix, and to convert the GCC-PHAT matrix and TFD matrix into GCC-PHAT feature images and TFD feature images, respectively; and to perform weighted fusion of the GCC-PHAT feature images and TFD feature images to obtain a fused feature image. The DOA estimation module is used to input the fused feature image into an improved MLLA network. This improved MLLA network is based on the original MLLA network, introducing a dynamic forget gate mechanism and position bias design into the linear attention mechanism module of each MLABlock in the original MLLA network. The fused feature image is convolutionally embedded and serialized through the convolutional Stem module of the improved MLLA network to generate a one-dimensional embedding sequence. This one-dimensional embedding sequence is then fed into a multi-level BasicLayer module. The multi-level BasicLayer extracts high-dimensional semantic features related to time delay, frequency domain energy, and spatial location from the fused feature image through stacked MLABlocks. The high-dimensional semantic feature sequence output by each BasicLayer is downsampled and channel-expanded in the spatial dimension. The downsampled and channel-expanded features are then converged into a global representation vector through adaptive one-dimensional average pooling. The global representation is processed by a classification head to output the direction-of-arrival (DOA) estimation result.

[0013] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in the microphone array-based direction-of-arrival estimation method.

[0014] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, is capable of executing any of the steps in the microphone array-based direction-of-arrival estimation method.

[0015] The microphone array-based direction-of-arrival estimation method provided by this invention has the following advantages: This invention first utilizes a ring microphone array to acquire target sound source signals, converting them into GCC-PHAT and TFD matrices, and further transforming them into feature images. Weighted fusion is then used to obtain a fused feature image that considers time-delay correlation, time-frequency energy, and spatial information, enhancing effective signal features under noise masking and reducing the loss of key information. Subsequently, the fused feature image is input into an improved MLLA network. This network introduces a dynamic forget gate and positional bias design in the MLLABlock linear attention module. The dynamic forget gate adaptively filters features and suppresses noise, while the positional bias design improves spatial feature accuracy. Combined with convolutional Stem module serialization processing, multi-level BasicLayer module for deep extraction of high-dimensional semantic features, and downsampling, channel expansion, and adaptive average pooling to converge global representations, the feature discrimination in low signal-to-noise ratio environments is effectively improved, mitigating overfitting in high-dimensional spaces. Finally, a classification head outputs accurate and stable DOA estimation results, thus maintaining stable DOA estimation performance even in low signal-to-noise ratio scenarios. This invention effectively solves the problems of signal features being masked and overfitting in deep learning DOA estimation at low signal-to-noise ratios by combining feature enhancement and network optimization, thereby improving the estimation robustness and accuracy. Attached Figure Description

[0016] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the microphone array-based direction-of-arrival estimation method according to Embodiment 1 of the present invention; Figure 2 For the experimental environment; Figure 3 For TFD feature images; Figure 4GCC-PHAT feature image; Figure 5 To fuse feature images; Figure 6 This is a flowchart illustrating the overall process framework of the method of the present invention; Figure 7 A flowchart illustrating data processing in the Linear Attention module of the improved MLLA network of this invention; Figure 8 This is a comparison chart of the estimation error GOSPA index of different algorithms under different signal-to-noise ratios (SNR). Detailed Implementation

[0018] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0019] Example 1 To address the problems of traditional DOA estimation algorithms being susceptible to interference under strong noise conditions, significantly reduced positioning accuracy, and insufficient generalization ability of existing deep learning methods due to reliance on single features, this invention provides a microphone array-based DOA estimation method, specifically a fused feature image-based improved (FFI) and improved MLLA network-based DOA estimation method (FFI-IMLLA-DOA). First, this invention extracts generalized cross-correlation phase transform (GCC-PHAT) feature images and time-frequency domain (TFD) matrix images from the array-received signals, respectively, and obtains a fused feature image through weighted fusion, comprehensively reflecting the time, frequency, and spatial characteristics of the signal. Next, the fused feature image is input into the improved MLLA network, utilizing an improved structure that introduces an enhanced dynamic forget gate and a position bias mechanism to improve the model's feature extraction accuracy and noise suppression capability under low signal-to-noise ratio (SNR). Finally, through training, a predicted angle classification is obtained, achieving DOA estimation for multiple sound sources.

[0020] The following is in conjunction with the appendix Figure 6 The method for estimating the direction of arrival of a sound source signal provided by this invention will be further explained.

[0021] This invention first establishes an experimental framework and trains the improved MLLA network using simulation data.

[0022] The entire experimental framework is mainly divided into two modules: data preprocessing and improved network training. First, the data preprocessing module performs the following specific processing steps: Data preprocessing is performed at the upper layer of the framework. First, the signal received by the microphone array is obtained through simulation. Then, the signal is transformed into a cross-correlation-phase transform (GCC-PHAT) matrix and a time-frequency domain (TFD) matrix. The two matrices are then converted into cross-correlation-phase transform (GCC-PHAT) feature images and time-frequency domain (TFD) feature images, respectively. Finally, the GCC-PHAT feature images and TFD feature images are fused to obtain the network input.

[0023] The specific implementation process of the data preprocessing module includes the following steps: The first step is to obtain the simulation signal.

[0024] Reference Figure 2 The provided experimental environment simulation diagram shows that the simulation scenario of this invention is conducted in a sealed room with dimensions of 6 meters x 6 meters x 4 meters (length x width x height). A circular microphone array is positioned at the very center of the room, at a height of 2 meters. This array is omnidirectional, meaning it has the same sensitivity to sound sources from all directions, and the reverberation time RT is set to 0.4 seconds. This invention does not involve elevation angles; therefore, the height of the sound sources is the same as the microphone array. Target sound sources will randomly appear at positions ranging from 0 degrees to 359 degrees, that is, at any position on a circle with a radius of 2 meters from the microphone array. There are two target sound sources. After determining the simulation environment, the true incident angle of each target sound source is recorded as the ground truth for subsequent labeling and evaluation.

[0025] according to Figure 2 The simulation environment can obtain the signal received by the i-th microphone. It can be expressed as the following formula: ; In the formula, This describes the room's impulse response, including characteristics such as reverberation and attenuation. Represents speech signals. The arrival delay of this array element, To observe noise signals, "" indicates convolution.

[0026] The second step involves converting the obtained target sound source signal into a cross-correlation-phase transform (GCC-PHAT) matrix and a time-frequency domain (TFD) matrix. Then, the GCC-PHAT and TFD matrices are converted into GCC-PHAT feature images and TFD feature images, respectively. Figure 3 and Figure 4 As shown.

[0027] (1) Convert the GCC-PHAT matrix into a GCC-PHAT feature image, such as Figure 4 As shown. Generalized cross-correlation has become an important algorithm in time delay estimation due to its high computational efficiency, simple model structure, good portability, ease of transfer learning, and ease of use with other signal processing tools. The microphone array consists of 8 elements. The signals received by the 8 elements are obtained by convolving the room pulses at the corresponding angles with the clean speech signal and adding noise. The signals are first normalized to prevent signal pruning distortion and ensure the accuracy of subsequent feature extraction. The time delay is calculated using the GCC-PHAT algorithm on the normalized signal matrix. Because the number of calculations between microphone pairs is large, this invention is simplified. Assume that the signals of a pair of microphone elements are represented as follows: and First, perform a Fourier transform on the two signals to obtain... and Then calculate the cross power spectrum. To obtain phase information related to time delay, the phase information is obtained by normalizing and removing the amplitude information of the cross-power spectrum. Finally, an inverse Fourier transform is performed on the phase-transformed spectrum to obtain the microphone's generalized cross-correlation function in the time domain:

[0028] ; In the formula, This indicates that microphone 1 and microphone 2 are receiving signals. y 1 and y The generalized cross-correlation function of 2 (GCC-PHAT sequence) is a function of time delay variables. The function is f; f is the frequency.

[0029] To convert the signal into an image, for an array of eight microphones, this invention selects the signal between the first microphone and each of the other microphones (i.e., microphones 2 to 8, a total of seven pairs), and uses the GCC-PHAT algorithm to calculate the generalized cross-correlation function of each microphone pair. (i=2,3,...,8), thus extracting the time delay features associated with the first microphone. Each GCC-PHAT function is a time-domain vector of length N (the number of time delay sampling points), representing the microphone pair at different time delays. The correlations are then analyzed. The seven GCC-PHAT functions are stacked into a 7×N matrix, where each row corresponds to the time delay features of a microphone pair. To feed these time delay features into a convolutional neural network (CNN), this matrix is ​​transformed into a 7×N image. Each row of the image represents the time delay range of a microphone pair, and each column represents the correlation of all microphone pairs at a specific time delay. Images generated in this way are suitable for input into a CNN to learn sound source directions.

[0030] (2) Convert the TFD matrix into a TFD feature image, such as Figure 3 As shown. The received signal of the microphone array in the time-frequency domain is the array's output vector. , The covariance matrix can be expressed as:

[0031] ; in, M 1 indicates the size is For frequency points, The number of time frames considered. h This represents the frequency enumeration index, which is the h-th frequency point. k h This represents the STFT frequency index, indicating the number of frequencies in the selected set. The location of each actual discrete frequency point X H Representing vectors The conjugate transpose of . In practical applications, the covariance matrix can also be calculated in this way: calculated at the lower edge time index at each frequency point. The covariance matrix is ​​calculated, and finally the average of the matrices for all frequency points is taken.

[0032] Specifically, a short-time Fourier transform is first performed on the microphone received signal to obtain the complex-valued time-frequency representation of each array element in the time and frequency dimensions. Then, the covariance matrix of the array output vector is calculated based on each frequency point and time frame, and accumulated and averaged across all frequency and time dimensions to obtain the overall time-frequency domain covariance features. Inspired by RGB three-channel input data in image processing, the real, imaginary, and phase information of the obtained matrix are mapped to the RGB three channels respectively. The phase part is normalized and mapped to the [0,1] interval, and the entire three-channel matrix is ​​globally normalized. Finally, bilinear interpolation is used to adjust it to a uniform size. The resulting time-frequency domain feature image integrates the signal's energy distribution and spatial correlation information, providing structured input features for the subsequent DOA depth estimation model.

[0033] (3) Weighted fusion of the GCC-PHAT feature image and the TFD feature image is performed to generate a fused feature image, such as... Figure 5 As shown. The fused feature image contains the temporal delay characteristics and spatial statistical features required for DOA estimation, containing rich information. This method is no longer limited to calculating only one type of feature information, but can more comprehensively capture information related to DOA estimation, which is more advantageous for DOA estimation. The fusion process can be represented as:

[0034] ; In the formula, This represents the fused feature image input to the network. Represents the GCC-PHAT feature image. Represents the TFD feature image. and Both represent the weights of the fusion; the former represents the weight of the GCC-PHAT feature image, and the latter represents the weight of the time-frequency domain feature image.

[0035] Finally, the obtained fused feature image samples are divided into training, validation, and test sets in an 8:1:1 ratio and fed into the improved MLLA network for training.

[0036] Second, the data processing process of the improved MLLA network training module is as follows: First, a dynamic forgetting gate mechanism and a positional bias design are introduced into the linear attention mechanism module of each MLLABlock in the original MLLA network to obtain an improved MLLA network, enabling the model to adaptively adjust feature weights according to signal characteristics. The main process of this invention is as follows: Figure 6As shown: First, the Stem performs convolutional embedding and serialization on the fused feature image; then, the sequence is fed into a multi-level BasicLayer for feature extraction, with several MLLABlocks stacked within each layer, and PatchMerging is performed after each BasicLayer layer to complete downsampling and channel expansion; at the end of the network, normalization (Norm) is applied to the output sequence, which is then converged into a global representation through adaptive one-dimensional average pooling, and finally input into the ClassificationHead to obtain 360-dimensional predictions. The Sigmoid function is used to obtain the independent existence probability vector of each DOA angle label, and then the vector is binarized according to a threshold to obtain the multi-label prediction of the DOA. Because the number of training samples is small, overfitting may occur during network training. Therefore, the epochs are set to 100 during the training phase, and an early stopping mechanism is introduced. Training stops when the F1 score does not improve significantly within 20 epochs, reducing unnecessary resource consumption. Choosing an appropriate batch size affects the training speed and resource consumption of the model; too small a batch size will hinder convergence, while too large a batch size will consume a lot of memory resources. Here, the batch size is set to 64. Because of the small sample dataset, all training does not use transfer learning and starts from scratch. The optimizer chosen is the AdamW adaptive moment estimation optimizer with decoupled weight decay to accelerate convergence.

[0037] like Figure 1 As shown, the method for direction-of-arrival estimation based on the improved MLLA network includes the following steps: The target sound source signal in its environment is acquired by a microphone array arranged in a ring.

[0038] The target sound source signal is converted into a cross-correlation-phase transform (GCC-PHAT) matrix and a time-frequency domain (TFD) matrix. The GCC-PHAT matrix and the TFD matrix are then converted into GCC-PHAT feature images and TFD feature images, respectively. The GCC-PHAT feature images and the TFD feature images are then weighted and fused to obtain a fused feature image.

[0039] The fused feature image is input into an improved MLLA network, which is based on the original MLLA network. The improved MLLA network introduces a dynamic forget gate mechanism and a position bias design into the linear attention mechanism module of each MLABlock of the original MLLA network. The fused feature image is convolutionally embedded and serialized through the convolutional Stem module of the improved MLLA network to generate a one-dimensional embedding sequence. The one-dimensional embedding sequence is fed into a multi-level BasicLayer module. The multi-level BasicLayer extracts high-dimensional semantic features related to time delay, frequency domain energy, and spatial location from the fused feature image through stacked MLABlocks. The high-dimensional semantic feature sequence output by each level of BasicLayer is downsampled and channel-expanded in the spatial dimension. The downsampled and channel-expanded features are converged into a global representation vector through adaptive one-dimensional average pooling. The global representation is processed by a classification head to output the direction of arrival (DOA) estimation result.

[0040] Example 2 After the improved MLLA network is trained, this invention applies the trained improved MLLA network to a microphone array-based direction-of-arrival estimation method in a real-world scenario, such as... Figure 6 and Figure 7 As shown, the implementation of this method includes the following steps: Step 1: Acquire the target sound source signal of the environment through a circular microphone array. The number of sound sources cannot exceed the number of microphone array elements (8).

[0041] Step 2: First, convert the target sound source signal into a GCC-PHAT matrix and a TFD matrix. Then, convert the GCC-PHAT matrix and the TFD matrix into GCC-PHAT feature images and TFD feature images, respectively. Finally, perform weighted fusion of the GCC-PHAT feature images and the TFD feature images to obtain a fused feature image.

[0042] Step 3: Input the fused feature image into the trained, improved MLLA network, and output the DOA estimation result. Details are as follows:

[0043] Step 31: The convolutional STEM embeds and serializes the fused feature image, with the fused feature image as the network input. (in For high, For width, (Number of channels). First, it undergoes three convolutions (including two). Convolutional layers (including a downsampling convolution with a stride of 2) extract low-level features and perform spatial downsampling; subsequently, the feature map is flattened and transposed to obtain a length of... Sequence representation ,in:

[0044] ; The overall downsampling factor for Stem (determined by the convolution stride). If Absolute Position Embedding (APE) is enabled, then... The learnable position vectors are additively superimposed and then fed into the encoder.

[0045] Step 32: Input the sequence into the MLLAEncoder, extract features layer by layer, and downsample between layers. The MLLAEncoder consists of multiple cascaded BasicLayers. Each BasicLayer contains several stacked MLABlocks for feature extraction. At each layer, PatchMerging is used to reduce the spatial resolution and increase the number of channels. Let the... Layer input is The output is Then we have:

[0046] ; The specific calculation process for each BasicLayer is as follows: Figure 6 The lower part indicates that within each BasicLayer, the input sequence first undergoes a normalization (norm) operation, then enters MLLABlock to complete attention and feature fusion, and subsequently adds the residuals with the input sequence. The specific process of the core module MLLABlock is as follows: (1) The obtained serialized vector is passed through a normalization (Norm) layer.

[0047] (2) First, perform a linear transformation on the normalized serialized vector to obtain and to Apply kernel function mapping: ; Obtain non-negative Nonnegative kernel mapping is beneficial for subsequent decomposable aggregation and numerical stability.

[0048] (3) In order to introduce relative / absolute position information, for Applying RotaryPositionEncoding (RoPE), we obtain ,in: ; (4) Simultaneously apply the dynamic forgetting gate and position bias to Before attention calculation, an input-dependent, location-aware forget gate is introduced into the Value branch. First, a preliminary representation of the forget gate is generated using the Linear Attention module: ; Then, a normalized two-dimensional coordinate grid is constructed. And the position bias is generated through a lightweight MLP: ; Finally, the linear transformation result is added to the scaled position bias, and then passed through the Sigmoid activation function to obtain the final forget gate: ; in, This represents the Sigmoid function. Finally, the forget gate is applied to the features. :

[0049] ; here This represents element-wise multiplication (Hadamard product).

[0050] (1) To enhance local details and edge structures, LePE is superimposed after the attention output: a depthwise separable convolution (DWConv) is applied to the output, along with a learnable weight matrix. Performing linear fusion, thereby capturing both global dependencies and local details simultaneously, helps improve the ability to model edge contours and spatial structures. ; (2) After linear projection, the residual is added to the input of that branch before entering the Block.

[0051] (3) Next, the result after residual is normalized again using Norm, and then fed into the feedforward network MLP for nonlinear transformation and fusion in the channel dimension. Then, the output of the MLP is expected to be connected to the corresponding branch input again using residual connection to further stabilize the training and enhance the representation ability.

[0052] (4) After two residual connections are performed, the end of MLLABlock is reached, and PatchMerging is executed. The spatial dimension is downsampled (size is halved) and the number of channels is increased to provide a more compact and information-dense representation for the higher-level semantic modeling of the next layer; then the result is sent as the output of this layer to the next layer.

[0053] Step 33: After all levels of the BasicLayer cascade are completed, the final sequence representation is obtained. .

[0054] First, the channels are normalized to obtain more stable output values: ; To suppress internal covariate bias and to unify the scale for global convergence; subsequently Adaptive one-dimensional average pooling is performed along the sequence dimension, first reorganizing the tensor into... ,through Aggregate into And flatten it to obtain the image-level global representation vector. The image-level global representation vector is a high-dimensional semantic feature in the fused feature image that is related to time delay, frequency domain energy, and spatial location.

[0055] The image-level global representation vector is input into the MLP of the classification head for mapping to obtain the final DOA evaluation result.

[0056] Specifically, Input linear classification head mapping to Dimensional Angular Space: ; Finally, the sigmoid activation function is used to... Convert to probability, prediction angle A correct prediction is represented by 1, and an incorrect prediction by 0. Therefore .

[0057] Experimental verification: To verify the FFI-IMLLA-DOA estimation method proposed in this invention, an ablation experiment was conducted to compare the impact of signal-to-noise ratio on the accuracy of direction-of-arrival estimation. The experimental results are as follows: Figure 8 As shown.

[0058] The comparison algorithms include traditional GCC-PHAT and MUSIC, as well as deep learning-based algorithms such as GCC-PHAT-MLLA, TFD-MLLA, FFI-ResNet50, FFI-MLLA, CM-CNN, and the FFI-IMLLA-DOA proposed in this invention. Graphics processing units (GPUs) are used to accelerate computation, which can significantly improve the training efficiency of the models. During the simulation signal generation, the sound propagation speed of the target sound source is set to 343 m / s, and the carrier signal frequency is set to 1000 Hz.

[0059] This experiment investigates the impact of signal-to-noise ratio (SNR) on the accuracy of Data of Ability (DOA) estimation. The SNR ranged from -20dB to 5dB with a step size of 5dB. Ten fixed angle combinations were randomly generated for each sound source, used across all SNR values. Each angle combination was randomly generated, but with a 20° interval between angles to prevent duplicate angles from interfering with the experimental results. Each angle combination generated 1000 samples for each SNR, resulting in a total of 60,000 speech samples. These samples were then divided into a training set of 48,000 mixed samples and a validation and test set of 6,000 mixed samples in an 8:1:1 ratio. DOA estimation was performed with all other parameters identical except for the SNR. By mixing data from different SNR values ​​(including low SNR) to train the model, the robustness of the model under various noise conditions can be better observed.

[0060] like Figure 8 Line graphs show the GOSPA (GoDelta Optical Target Sparing Assignment) index of various algorithms under different signal-to-noise ratios (SNR). The GOSPA value comprehensively measures the localization error (deviation between the estimated direction and the true direction), missed detection error (failure to detect the true signal source), and false detection error (incorrect detection of a non-existent signal source) of DOA estimation. The trend of the line graphs for different algorithms shows that the GOSPA value of all algorithms exhibits a significant decreasing trend with increasing SNR, because when noise interference is strong, the sound source localization becomes more difficult and the accuracy is lower. However, the performance differences between algorithms are significant, especially in the low SNR region (-20dB to -10dB). The GCC-PHAT curve reaches over 25° at -20dB and only drops to around 17° at -5dB, showing its limitations in strong noise environments. MUSIC is slightly better than GCC-PHAT at low SNR, but the curve still maintains a high GOSPA value, and the traditional algorithm's GOSPA value decreases more gradually, indicating that the traditional algorithm is sensitive to noise, lacks robustness, and is difficult to cope with real-world complex scenarios.

[0061] In contrast, ablation experiments clearly show that deep learning methods exhibit a steeper descent trend, especially after applying the strategy of fusing feature images. The curves of GCC-PHAT-MLLA and TFD-MLLA are significantly lower than those of traditional methods, demonstrating the advantage of neural networks in capturing nonlinear features. Further observation reveals that the curves of FFI-ResNet50, FFI-MLLA, and FFI-IMLLA-DOA based on fused feature images have lower initial GOSPA values, faster convergence speeds, and maintain the lowest potential across the entire SNR range. The ablation experiment results of deep learning demonstrate that feature fusion images effectively improve the model's noise suppression capability by integrating multimodal information from the time, frequency, and spatial domains. Among them, the curve of the proposed FFI-IMLLA-DOA consistently remains at the bottom, showing a significant performance advantage, especially in the low SNR range. This reflects that the method of introducing a dynamic forget gate mechanism and incorporating positional bias further enhances the accuracy and generalization ability of feature extraction in the MLLA network. This improvement combines linear transformation with biases generated by positional MLP, applies a sigmoid gate to the value vector, dynamically forgetting noise interference or irrelevant spatial information, while enhancing position sensitivity, making the network more adaptable to handling complex array signals. Overall, the line graph trend verifies the significant advantage of the feature fusion strategy in improving the robustness of DOA estimation, while highlighting the optimization effect of the improved MLLA network in low SNR scenarios, significantly improving the model's stability in noise-dominated environments.

[0062] Under extremely low SNR conditions of -20dB, the GOSPA of traditional GCC-PHAT and MUSIC are 25.01° and 23.68°, respectively, with large errors that are difficult to meet the needs of practical applications. GCC-PHAT-MLLA, which incorporates an MLLA network, significantly reduces its GOSPA to 10.61°, a reduction of 57.6%. TFD-MLLA achieves a GOSPA of 18.42°, outperforming the traditional algorithm but 7.8° higher than GCC-PHAT-MLLA, indicating that GCC-PHAT features have better compatibility within the MLLA framework. The advantages of the feature fusion method are significant: the GOSPA of FFI-ResNet50 is reduced to 8.87°, the GOSPA of FFI-MLLA is further reduced to 7.32°, while the GOSPA of the proposed FFI-IMLLA-DOA reaches 6.08°, which is 16.9% lower than FFI-MLLA, 31.5% lower than FFI-ResNet50, 75.7% lower than GCC-PHAT, and 67% lower than TFD-MLLA. The difference in GOSPA values ​​indicates that the fused feature images not only enrich the dimensions of the input features, but also improve the noise robustness of the model through multi-scale information integration. As the SNR increases to -10dB, the difference in performance metrics widens rapidly. The GOSPA of traditional methods remains between 17° and 20°, while the GOSPA of the feature image fusion method drops to 0.3°-0.7°. The 0.3° value of FFI-IMLLA-DOA is 94.4% lower than MUSIC's 17.84° and 98% lower than GCC-PHAT's 20.13°, demonstrating that deep fusion performs better than traditional algorithms under moderate noise conditions. In the high SNR region (such as 0dB and 5dB), all algorithms tend to converge. The feature image fusion method, especially the improved MLLA network, maintains a small error of 0.01°-0.03°, far lower than the 9°-13° of traditional methods, proving its higher accuracy ceiling in low-noise environments. In particular, the improvement of FFI-IMLLA-DOA compared to FFI-MLLA is more significant at low SNR (e.g., a reduction of 1.24° at -20dB and 0.36° at -15dB), while it tends to be consistent at high SNR. This indicates that the network improvement mainly targets noise-sensitive points and enhances the attention allocation to weak signals. Spatial attention allocation to weak signals is enhanced through positional bias, while the dynamic forgetting mechanism ensures effective screening in the high-dimensional feature space, avoiding overfitting or noise amplification. These quantitative results confirm the effectiveness of the proposed improvement, which not only theoretically extends the robustness of the linear attention mechanism but also provides measurable performance improvements in experiments.

[0063] In summary, the experimental results fully demonstrate the superiority of feature-fused images in DOA estimation. It significantly reduces the GOSPA index through multimodal feature collaboration, especially in low SNR environments. Furthermore, the improved MLLA network further amplifies this advantage by introducing an enhanced dynamic forget gate and positional bias, providing a more refined feature processing mechanism and improving the model's adaptability to noise and spatial variability, showcasing the significant advantages of deep learning in low SNR conditions.

[0064] Compared with existing technologies, the direction-of-arrival (DOA) estimation method based on fused feature images and an improved MLLA network provided by this invention has significant advantages. This invention obtains a fused feature image that simultaneously reflects information in the time, frequency, and spatial domains of the signal by weighted fusion of the generalized cross-correlation-phase transform (GCC-PHAT) feature image of the array received signal and the time-frequency domain (TFD) feature image. This significantly enhances the completeness and discriminative power of the input features, and improves the model's noise robustness and feature representation ability under complex noise conditions. Furthermore, the improved Mamba-Like LinearAttention (MLLA) network introduces an enhanced dynamic forget gate and a position bias mechanism, which can adaptively adjust feature weights according to signal characteristics. This achieves dynamic suppression of noise interference and enhancement of effective spatial features, enabling the model to maintain high DOA estimation accuracy and stability even under low signal-to-noise ratio conditions. Experimental results show that the GOSPA index of the method of this invention is only 6.08° at an extremely low signal-to-noise ratio of -20 dB, which is about 75.7% lower than the traditional GCC-PHAT algorithm and 16.9% lower than the unimproved model. At -10 dB, the angle error is reduced to 0.3°, 94.4% lower than the MUSIC algorithm. Even in the high signal-to-noise ratio range, it maintains a small error on the order of 0.01°, significantly outperforming the traditional algorithm. Therefore, this invention achieves significant improvements in noise suppression, robustness, and estimation accuracy, enabling high-precision and highly stable sound source localization in complex acoustic environments. It provides an efficient and feasible new solution for the engineering application of deep learning in array signal processing. This solution is suitable for direction-of-arrival estimation scenarios using circular microphone arrays in indoor environments, especially exhibiting higher stability and generalization ability in high-noise environments.

[0065] Based on the same inventive concept, the present invention also provides a direction-of-arrival estimation system based on a microphone array, comprising: The data acquisition module is used to acquire target sound source signals in its environment through a circular array of microphones.

[0066] The data conversion module is used to convert the target sound source signal into a cross-correlation-phase transform (GCC-PHAT) matrix and a time-frequency domain (TFD) matrix, and to convert the GCC-PHAT matrix and the TFD matrix into GCC-PHAT feature images and TFD feature images, respectively. The GCC-PHAT feature images and TFD feature images are then weighted and fused to obtain a fused feature image.

[0067] The DOA estimation module is used to input the fused feature image into an improved MLLA network. This improved MLLA network is based on the original MLLA network, introducing a dynamic forget gate mechanism and position bias design into the linear attention mechanism module of each MLABlock in the original MLLA network. The fused feature image is convolutionally embedded and serialized through the convolutional Stem module of the improved MLLA network to generate a one-dimensional embedding sequence. This one-dimensional embedding sequence is then fed into a multi-level BasicLayer module. The multi-level BasicLayer extracts high-dimensional semantic features related to time delay, frequency domain energy, and spatial location from the fused feature image through stacked MLABlocks. The high-dimensional semantic feature sequence output by each BasicLayer is downsampled and channel-expanded in the spatial dimension. The downsampled and channel-expanded features are then converged into a global representation vector through adaptive one-dimensional average pooling. The global representation is processed by a classification head to output the direction-of-arrival (DOA) estimation result.

[0068] The modules in the microphone array-based direction-of-arrival estimation system described above can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in a computer device, or stored in software within the computer device's memory, allowing the processor to invoke and execute the corresponding operations of each module.

[0069] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in the embodiment of the microphone array-based direction-of-arrival estimation method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0070] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in the embodiment of the microphone array-based direction-of-arrival estimation method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0071] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for direction-of-arrival estimation based on a microphone array, characterized in that, include: The target sound source signal in its environment is acquired by a microphone array arranged in a ring. The target sound source signal is converted into a cross-correlation-phase transform (GCC-PHAT) matrix and a time-frequency domain (TFD) matrix. The GCC-PHAT matrix and the TFD matrix are then converted into GCC-PHAT feature images and TFD feature images, respectively. The GCC-PHAT feature image and the TFD feature image are weighted and fused to obtain the fused feature image; The fused feature image is input into an improved MLLA network, which is based on the original MLLA network. The improved MLLA network introduces a dynamic forget gate mechanism and a position bias design into the linear attention mechanism module of each MLABlock of the original MLLA network. The fused feature image is convolutionally embedded and serialized through the convolutional Stem module of the improved MLLA network to generate a one-dimensional embedding sequence. The one-dimensional embedding sequence is fed into a multi-level BasicLayer module. The multi-level BasicLayer extracts high-dimensional semantic features related to time delay, frequency domain energy, and spatial location from the fused feature image through stacked MLABlocks. The high-dimensional semantic feature sequence output by each level of BasicLayer is downsampled and channel-expanded in the spatial dimension. The downsampled and channel-expanded features are converged into a global representation vector through adaptive one-dimensional average pooling. The global representation is processed by a classification head to output the direction of arrival (DOA) estimation result.

2. The method for direction-of-arrival estimation based on a microphone array according to claim 1, characterized in that, Obtaining high-dimensional semantic features through a multi-level BasicLayer module includes the following steps: A one-dimensional embedded sequence is input into the first BasicLayer and normalized by its normalization layer. The normalized sequence is then fed into an MLLABlock stacked from the first BasicLayer. The MLLABlock first normalizes the input and performs a linear transformation to generate Q, K, and V. Kernel functions are applied to Q and K, and positional encoding is applied to introduce positional information. A linear transformation is performed on V to generate a preliminary representation with a forgetting gate. The forgetting coefficient is obtained by combining the positional bias with Sigmoid activation, and the V vector is updated by gating. A learnable positional bias is introduced when calculating the linear attention output, and LePE is superimposed to reinforce local features. The feature extraction of a single MLLABlock is completed through residual connections and MLP nonlinear transformation. After all MLABlocks in the first layer have been processed, the first layer output sequence is generated by downsampling and expanding the channels through PatchMerging. The output sequence is used as the input of the next layer, BasicLayer, and features are extracted layer by layer. After the last BasicLayer completes feature extraction, the final high-dimensional semantic feature sequence is output.

3. The method for direction-of-arrival estimation based on a microphone array according to claim 1, characterized in that, The conversion of the GCC-PHAT matrix into a GCC-PHAT feature image includes the following steps: Normalize the signals in the GCC-PHAT matrix; The time delay is calculated using the GCC-PHAT algorithm on the normalized signal matrix to obtain the time delay feature matrix, which is then converted into a GCC-PHAT feature image.

4. The method for direction-of-arrival estimation based on a microphone array according to claim 1, characterized in that, Converting a TFD matrix into a TFD feature image involves the following steps: Perform a short-time Fourier transform on the TFD matrix to obtain the complex time-frequency representations of each element in the time and frequency dimensions; The covariance matrix of the array output vector is calculated based on each frequency point and time frame, and accumulated and averaged over all frequency and time dimensions to obtain the overall time-frequency domain covariance feature matrix. The real, imaginary, and phase information of the time-frequency domain covariance feature matrix are mapped to the three RGB channels respectively. The phase part is normalized and mapped to the [0,1] interval. The entire three-channel matrix is ​​then globally normalized and finally adjusted to a uniform size using bilinear interpolation to obtain the TFD feature image.

5. The method for direction-of-arrival estimation based on a microphone array according to claim 1, characterized in that, The fused feature image contains temporal delay characteristics and spatial statistical features, and the fusion process is represented as follows: ； In the formula, Represents the fused feature image. Represents the GCC-PHAT feature image. Represents the TFD feature image. and These represent the weights of the GCC-PHAT feature image and the TFD feature image, respectively.

6. The method for direction-of-arrival estimation based on a microphone array according to claim 1, characterized in that, The process of convolutionally embedding and serializing the fused feature image using the convolutional Stem module is as follows: The fused feature image is subjected to three convolutional segments to extract low-level features and then spatial downsampling to obtain the feature map; Flattening and transposing the feature map yields a length of... The sequence representation.

7. A direction-of-arrival estimation system based on a microphone array, characterized in that, include: The data acquisition module is used to acquire target sound source signals in its environment through a circular array of microphones. The data conversion module is used to convert the target sound source signal into a cross-correlation-phase transform (GCC-PHAT) matrix and a time-frequency domain (TFD) matrix, and to convert the GCC-PHAT matrix and the TFD matrix into GCC-PHAT feature images and TFD feature images, respectively. The GCC-PHAT feature image and the TFD feature image are weighted and fused to obtain the fused feature image; The DOA estimation module is used to input the fused feature image into an improved MLLA network. This improved MLLA network is based on the original MLLA network, introducing a dynamic forgetting gate mechanism and positional bias design into the linear attention mechanism module of each MLABlock in the original MLLA network. The fused feature image is convolutionally embedded and serialized through the convolutional Stem module of the improved MLLA network to generate a one-dimensional embedding sequence. This one-dimensional embedding sequence is then fed into a multi-level BasicLayer module. The multi-level BasicLayer extracts high-dimensional semantic features related to time delay, frequency domain energy, and spatial location from the fused feature image through stacked MLABlocks. The high-dimensional semantic feature sequence output by each BasicLayer is downsampled and channel-expanded in the spatial dimension. The downsampled and channel-expanded features are then converged into a global representation vector through adaptive one-dimensional average pooling. The global representation is processed by a classification head to output the direction-of-arrival (DOA) estimation result.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 6.

Images