A multi-sound-source direction of arrival estimation method and related device

Abstract

The application provides a multi-sound source direction of arrival estimation method and related equipment, and belongs to the technical field of sound source positioning. The method comprises the following steps: acquiring a multi-channel time domain observation signal collected and performing time-frequency transformation to obtain a multi-channel frequency domain signal; calculating an observation inter-channel phase difference according to the multi-channel frequency domain signal; constructing a direct path inter-channel phase difference theoretical template according to an array geometric structure and a preset sound source direction search space; matching the observation inter-channel phase difference with the direct path inter-channel phase difference theoretical template of each candidate direction and performing similarity calculation to obtain a steering response power spatial spectrum; generating a time-frequency mask by using the spatial spectrum and applying the time-frequency mask to the multi-channel frequency domain signal to obtain an enhanced frequency domain feature; and performing sound source direction of arrival estimation according to the enhanced frequency domain feature to obtain the direction of arrival of each sound source. The application effectively improves the precision and robustness of multi-sound source direction of arrival estimation in a complex acoustic environment through physical prior-guided time-frequency adaptive enhancement.

Description

Technical Field

[0001] This application relates to the field of sound source localization technology, and in particular to a method and related equipment for estimating the direction of arrival of multiple sound sources. Background Technology

[0002] In the field of speech signal processing, multi-source direction of arrival (DOA) estimation based on microphone arrays is a core pre-processing technology for tasks such as sound source localization, speech enhancement, speech separation, and target sound source extraction. Accurately estimating the spatial location of multiple sound sources can provide crucial spatial prior information for subsequent audio signal processing, and has wide-ranging applications in smart homes, video conferencing, robotic hearing, and security monitoring.

[0003] Existing multi-source DOA estimation methods are mainly divided into two categories. The first category is the traditional signal processing method, represented by Steering Response Power Phase Transform (SRP-PHAT). This method improves robustness in reverberant and noisy environments by traversing possible sound source locations on a spatial grid, performing delay compensation on the microphone array signal, and summing the results. However, the traditional SRP-PHAT method exhibits significant performance degradation in low signal-to-noise ratio and strong reverberation environments, and its spatial resolution is limited by the discretization accuracy of the search grid, making it difficult to meet the high-precision localization requirements in complex acoustic environments.

[0004] The second category is deep learning methods. In recent years, deep learning has been widely used in sound source DOA estimation, mainly divided into end-to-end methods and feature-driven methods. End-to-end methods directly input multi-channel audio signals into a neural network to output the sound source direction, but they are easily affected by interference in multi-source overlap and complex acoustic environments. Feature-driven methods first extract physical or statistical features and then input them into the neural network for prediction, which can improve robustness by utilizing physical prior information. However, most existing deep learning methods operate as "black box" models, lacking effective physical prior guidance mechanisms, especially lacking technical means to explicitly spatially guide the network's feature extraction process, making it difficult for the network to adaptively focus on spatially reliable regions in the time-frequency domain. When multiple sound sources emit sound simultaneously, background noise is strong, or room reverberation is severe, the network's spatial resolution and localization accuracy decrease significantly. Summary of the Invention

[0005] The main objective of this application is to propose a method, device, electronic device, storage medium, and program product for estimating the direction of arrival (DOA) of multiple sound sources based on spatial perception guided by inter-channel phase difference. This aims to address the shortcomings of existing deep learning methods in sound source DOA estimation, such as the lack of physical prior guidance and the difficulty in adaptively focusing on credible spatial regions.

[0006] To achieve the above objectives, one aspect of this application proposes a method for estimating the direction of arrival (DOA) of a multi-source sound source, the method comprising: Acquire multi-channel time-domain observation signals collected by a microphone array, and perform time-frequency transformation on the multi-channel time-domain observation signals to obtain multi-channel frequency-domain signals; Calculate the phase difference between the observation channels based on the multi-channel frequency domain signal; Based on the microphone array geometry and the preset sound source direction search space, a theoretical template for the phase difference between direct path channels is constructed. The phase difference between the observation channels is matched and similarity is calculated with the theoretical template of the phase difference between the direct path channels corresponding to the direction of each candidate sound source to obtain the spatial spectrum of the steering response power. The time-frequency mask is generated using the steering response power spatial spectrum, and then applied to the multi-channel frequency domain signal to obtain enhanced frequency domain features. Based on the enhanced frequency domain characteristics, the direction of arrival (DOA) of the sound sources is estimated to obtain the DOA of each sound source.

[0007] In some embodiments, calculating the phase difference between observation channels based on the multi-channel frequency domain signal includes: All channels of the multi-channel frequency domain signal are paired up to construct complex spectrum pairs, thereby realizing the structural recombination of features between channels. The phase spectrum of each microphone pair is calculated based on the reconstructed complex spectrum. The phase difference of each microphone pair is calculated based on the phase spectrum. The sine and cosine values ​​of the phase difference are then taken to obtain the real-valued phase difference between the observation channels.

[0008] In some embodiments, constructing a theoretical template for the phase difference between direct path channels based on the microphone array geometry and a preset sound source direction search space includes: Discretized direction grids are constructed in spherical coordinates, where the pitch angle is sampled at equal intervals in the range [0, π] and the azimuth angle is sampled at equal intervals in the range [-π, π], forming multiple candidate points for the direction of arrival of the sound source; each candidate point corresponds to a unique combination of pitch and azimuth angles. Based on the far-field plane wave model, calculate the ideal time delay difference for any candidate direction and any pair of microphones: The ideal time delay difference is converted into an inter-channel phase difference in the frequency domain; The complex value of the phase difference between the channels is split into real and imaginary parts and then spliced ​​together in the frequency dimension to obtain the theoretical template of the phase difference between the channels of the direct path.

[0009] In some embodiments, the step of matching and similarity calculation between the phase difference between the observation channels and the theoretical template of the phase difference between the direct path channels corresponding to the directions of each candidate sound source to obtain the steering response power spatial spectrum includes: The theoretical templates for the phase difference between the observation channels and the phase difference between the direct path channels of each candidate direction are reshaped respectively; For each pair of microphones, the inner product of the reshaped phase difference between the observation channels and the theoretical template of the phase difference between the direct path channels is calculated and accumulated in the frequency dimension to obtain the accumulated inner product value of each microphone pair in each candidate direction. For each candidate direction, the directional response power for that candidate direction is obtained by averaging the accumulated inner product values ​​of all microphone pairs in that candidate direction.

[0010] In some embodiments, generating a time-frequency mask using the steering response power spatial spectrum includes: The spatial spectrum of the steering response power is normalized in the azimuth dimension, and the directional information in the frequency dimension is extracted through a frequency mapping network to obtain frequency attention. The normalized steering response power spatial spectrum is averaged along the angle, and the global time weight is extracted through a time mapping network to obtain time attention. Multiplying the frequency attention by the time attention yields time-frequency masking.

[0011] In some embodiments, the step of estimating the direction of arrival (DOA) of the sound sources based on the enhanced frequency domain features to obtain the DOA of each sound source includes: The enhanced frequency domain features and the phase difference between the observation channels are encoded respectively to obtain the first encoded feature and the second encoded feature; The first encoded feature and the second encoded feature are fused using a cross-attention module to obtain a fused feature; The fused features are sequentially modeled in the frequency dimension, time dimension, and channel dimension to obtain deep features; Based on the deep features, multi-label classification is performed, and the prediction results of whether a sound source exists in each discrete angle category are output.

[0012] In some embodiments, the step of sequentially performing frequency dimension modeling, time dimension modeling, and channel dimension modeling on the fused features to obtain deep features includes: The fused features are restructured into a first-dimensional form, and a bidirectional long short-term memory network is used for frequency dimension modeling, with residual connections introduced. The features modeled in the frequency dimension are reorganized into a second-dimensional form, and a bidirectional long short-term memory network is used for time dimension modeling, with residual connections introduced. The compression-excitation module is used to adaptively weight the features modeled in the time dimension in the channel dimension to obtain the deep features modeled in the channel dimension.

[0013] In some embodiments, prior to performing multi-label classification based on the deep features, the method further includes: The deep features are pooled using a hybrid pooling module to obtain a global representation. The hybrid pooling module includes: pooling in the frequency dimension by first averaging in the time dimension, then performing a linear transformation on the channel dimension to generate a frequency score, and finally performing a weighted sum after softmax normalization to obtain the frequency pooling result; pooling in the time dimension by first averaging in the frequency dimension, then performing a linear transformation on the channel dimension to generate a time score, and finally performing a weighted sum after softmax normalization to obtain the time pooling result; and concatenating the frequency pooling result with the time pooling result to obtain the global representation.

[0014] In some embodiments, the step of performing multi-label classification based on the deep features and outputting prediction results for the presence or absence of a sound source in each discrete angle category includes: The deep features or global representation are processed by a fully connected layer and a non-linear activation function to output a multi-label classification score vector, where each dimension of the score vector corresponds to a discrete angle category. The learning objective is to convert the actual sound source angle of arrival of each speech into a multi-label one-hot vector. The network is trained using a weighted combination of binary cross-entropy loss and focusing loss as the total loss function.

[0015] in, For binary classification, cross-entropy loss, To focus on the losses, These are the weighting coefficients.

[0016] In some embodiments, prior to performing time-frequency transformation on the multi-channel time-domain observation signal, the method further includes: The multi-channel time-domain observation signal is normalized using a Laplace normalization method based on a dynamic smoothing factor. The Laplace normalization method includes: calculating a steady-state smoothing factor based on the target smoothing window length; dynamically adjusting the actual smoothing factor according to the current frame number; recursively updating the mean estimate using an exponential moving average method; and dividing the input signal by the dynamically estimated mean sequence to obtain the normalized signal.

[0017] To achieve the above objectives, another aspect of this application provides a multi-source direction-of-arrival estimation device, the device comprising: The time-frequency conversion module is used to acquire multi-channel time-domain observation signals collected by the microphone array, and to perform time-frequency conversion on the multi-channel time-domain observation signals to obtain multi-channel frequency-domain signals. The observation phase difference calculation module is used to calculate the phase difference between observation channels based on the multi-channel frequency domain signal. The theoretical template construction module is used to construct a theoretical template for the phase difference between direct path channels based on the microphone array geometry and the preset sound source direction search space; The spatial spectrum construction module is used to match and calculate the similarity between the phase difference between the observation channels and the theoretical template of the phase difference between the direct path channels corresponding to the direction of each candidate sound source to obtain the spatial spectrum of the steering response power. A spatially aware masking module is used to generate a time-frequency mask using the spatial spectrum of the steering response power, and to apply the time-frequency mask to the multi-channel frequency domain signal to obtain enhanced frequency domain features. The direction-of-arrival estimation module is used to estimate the direction of arrival of the sound sources based on the enhanced frequency domain characteristics, so as to obtain the direction of arrival of each sound source.

[0018] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0019] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0020] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0021] The embodiments of this application include the following beneficial effects: This application provides a method, apparatus, electronic device, storage medium, and program product for estimating the direction of arrival (DOA) of multiple sound sources. This scheme acquires microphone array signals and performs time-frequency transformation on them. On the one hand, it calculates the phase difference between channels from the observation data. On the other hand, it constructs a theoretical template of the direct path phase difference for each candidate direction based on the array geometry. The two are matched to calculate the similarity to construct the steering response power spatial spectrum. The spatial spectrum is then used to generate a time-frequency mask to adaptively enhance the frequency domain features. Based on the enhanced features, the DOA is estimated. This application can achieve explicit modeling of the spatial direction information of the sound source and adaptive focusing of the credible time-frequency region by combining data-driven and physical prior methods. Thus, it can effectively improve the accuracy and robustness of DOA estimation in multi-source scenarios without relying on complex post-processing. Attached Figure Description

[0022] Figure 1 This is a flowchart of a multi-source direction-of-arrival estimation method provided in an embodiment of this application; Figure 2 yes Figure 1 The flowchart of step S102 in the document; Figure 3 yes Figure 1 The flowchart of step S103 in the process; Figure 4 yes Figure 1 The flowchart of step S104 in the process; Figure 5 yes Figure 1 The flowchart of step S105 in the process; Figure 6 yes Figure 1 The flowchart of step S106 in the process; Figure 7 This is a network structure diagram of the multi-source DOA estimation method provided in the embodiments of this application; Figure 8 This is a structural diagram of the spatial sensing module (SAM) and two encoders provided in the embodiments of this application; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0025] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0026] 1) Direction of Arrival (DOA) refers to the azimuth information of the signal source incident on the array. It can effectively characterize the spatial positional relationship between the signal source and the array and plays a very important role in signal processing fields such as speech, antenna, sonar, ultrasound, and radar.

[0027] 2) Steered Response Power with PHAse Transform (SRP-PHAT) is a sound source localization method based on delay and summation beamforming. It traverses possible sound source locations on a spatial grid, compensates for the delay of the microphone array signals according to their corresponding time differences of arrival, and sums the results to obtain the guided response power at each location. The most probable sound source direction is determined by comparing the power magnitudes. To improve robustness in reverberant and noisy environments, SRP-PHAT introduces phase transformation weighting, normalizes the signal's spectral amplitude, and retains only phase information for cross-correlation calculation, thereby suppressing the interference of the amplitude spectrum on the localization results. This method is widely used in multi-channel signal processing scenarios such as speech localization, robot hearing, and video conferencing.

[0028] 3) Inter-channel phase difference (IPD) refers to the phase difference between the signals from the same sound source received by different channels in a microphone array. Due to the time delay in sound wave propagation, the phase of the signal at different microphones will have a specific difference. This difference is closely related to the spatial location of the sound source, the array geometry, and the signal frequency. IPD can effectively characterize the directional information of the sound source and is widely used as an important spatial feature in multi-channel speech processing tasks, such as sound source localization, speech separation, and enhancement.

[0029] Multi-source localization (DOA) aims to estimate the location information of multiple sound sources in acoustic space, providing spatial priors for subsequent tasks such as sound source enhancement, sound source separation, and target sound source extraction. In recent years, deep learning has been widely used in multi-source DOA estimation, with methods mainly falling into two categories: end-to-end methods and feature-driven methods. End-to-end methods directly input multi-channel audio signals into the network and directly output the sound source directions without further processing. However, they are easily affected by interference in environments with low signal-to-noise ratios, multiple overlapping sources, and strong reverberation. Feature-driven methods first extract physical or statistical features (such as IPD, cross-correlation features, etc.) and then input them into the neural network for corresponding predictions. This effectively utilizes physical prior information to improve robustness in complex environments, but requires corresponding post-processing to obtain the final DOA.

[0030] In deep learning sound source localization tasks, common output strategies can be divided into two categories: regression and classification. Regression methods directly predict the angle or coordinates of the sound source. The advantage is that they can obtain continuous DOA estimates, but they are prone to prediction bias and unstable training in environments with multiple overlapping sound sources, noise, and reverberation, as well as label matching problems. Classification methods discretize the sound source direction into several angle intervals, transforming it into a multi-label classification task. This can naturally handle the problem of multiple overlapping sound sources, but its angle resolution is limited by the discretization accuracy. At the same time, it still faces the problem of decreased accuracy in environments with low signal-to-noise ratio and complex reverberation.

[0031] Existing deep learning methods still have the following problems in complex acoustic environments: First, the network is sensitive to noise and reverberation, and the DOA estimation accuracy drops significantly in low signal-to-noise ratio or strong reverberation environments; second, when multiple sound sources overlap, the information in the frequency, time and channel dimensions is not fully modeled, resulting in insufficient spatial resolution and multi-source discrimination ability; third, there is a lack of effective mechanisms to integrate physical priors or spatial features, which limits the network's generalization ability in non-ideal environments, thereby restricting the reliability and accuracy of deep learning methods in practical tasks.

[0032] In view of this, this application provides a method, apparatus, electronic device, storage medium, and program product for estimating the direction of arrival (DOA) of multiple sound sources based on spatial perception guided by inter-channel phase difference (IPD). This scheme constructs a steering response power spatial spectrum as a physical prior, guiding the neural network to perform spatial perception weighting on time-frequency features, thereby improving the accuracy and robustness of multi-source DOA estimation. Specifically, to enhance spatial perception capabilities, this application constructs a steering response power spatial spectrum using observation IPD and spatial direct path IPD templates, and designs a spatial perception attention module based on this spectrum to achieve time-frequency adaptive weighting of STFT features. This allows the network to automatically focus on more reliable time-frequency regions in the spatial direction, thereby improving its modeling capabilities for multi-source scenes. To improve the accuracy and robustness of localization in complex scenes, this application uses SRP spatial spectrum guidance and cross-attention modules, combined with frequency, time, and channel dimension modeling, to suppress the interference of background noise and reverberation on feature extraction. This enables the network to stably and accurately complete the source DOA estimation task even in multi-source overlapping and non-ideal acoustic environments. To achieve efficient end-to-end optimization and robust training, this application introduces multi-label classification supervision into the network structure and employs a weighted combination of binary cross-entropy and focusing loss to balance the learning effects of basic supervision signals and difficult samples. Simultaneously, dynamic positive and negative sample weights are incorporated to gradually enhance the network's spatial sensitivity and discriminative ability during training, thereby achieving more efficient and robust angle-of-arrival estimation in complex multi-source environments.

[0033] The multi-source direction-of-arrival (DOA) estimation method provided in this application relates to the fields of microphone array signal processing technology and sound source localization technology. The multi-source DOA estimation method provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the multi-source DOA estimation method, but is not limited to the above forms.

[0034] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0035] like Figure 1 As shown, this embodiment provides a method for estimating the direction of arrival (DOA) of multiple sound sources, specifically including the following steps: Step S101: Acquire the multi-channel time-domain observation signal collected by the microphone array, and perform time-frequency transformation on the multi-channel time-domain observation signal to obtain the multi-channel frequency-domain signal.

[0036] In one embodiment, before performing time-frequency transformation on the multi-channel time-domain observation signal, the method further includes the following steps: normalizing the multi-channel time-domain observation signal using a Laplace normalization method based on a dynamic smoothing factor. The Laplace normalization method includes: calculating a steady-state smoothing factor based on the target smoothing window length; dynamically adjusting the actual smoothing factor according to the current frame number; recursively updating the mean estimate using an exponential moving average; and dividing the input signal by the dynamically estimated mean sequence to obtain the normalized signal.

[0037] Step S102: Calculate the phase difference between the observation channels based on the multi-channel frequency domain signal.

[0038] In one embodiment, see Figure 2 Step S102 specifically includes steps S201-S202: Step S201: Pair all channels of the multi-channel frequency domain signal together to construct complex spectrum pairs and realize the structural recombination of channel features; Step S202: Calculate the phase spectrum of each microphone pair based on the reconstructed complex spectrum, calculate the phase difference of each microphone pair based on the phase spectrum, and take the sine and cosine values ​​of the phase difference to obtain the phase difference between observation channels in real numerical form.

[0039] Step S103: Based on the microphone array geometry and the preset sound source direction search space, construct a theoretical template for the phase difference between direct path channels.

[0040] In some embodiments, see Figure 3 Step S103 specifically includes steps S301-S304: Step S301: Construct a discretized direction grid in spherical coordinates, wherein the pitch angle is sampled at equal intervals in the range of [0,π], and the azimuth angle is sampled at equal intervals in the range of [-π,π], thus forming multiple candidate points for the direction of arrival of the sound source; Step S302: Based on the far-field plane wave model, calculate the ideal time delay difference for any candidate direction and any pair of microphones: Step S303: Convert the ideal time delay difference into an inter-channel phase difference in the frequency domain; Step S304: Decompose the complex value of the inter-channel phase difference into real and imaginary parts, and splice them in the frequency dimension to obtain the theoretical template of the inter-channel phase difference of the direct path.

[0041] Step S104: Match and calculate the similarity between the phase difference between the observation channels and the theoretical template of the phase difference between the direct path channels corresponding to the direction of each candidate sound source to obtain the spatial spectrum of the steering response power.

[0042] In some embodiments, see Figure 4 Step S104 specifically includes steps S401-S403: Step S401: Reshape the theoretical templates of the phase difference between the observation channels and the phase difference between the direct path channels of each candidate direction respectively; Step S402: For each pair of microphones, calculate the inner product of the reshaped phase difference between the observation channels and the theoretical template of the phase difference between the direct path channels, and accumulate them in the frequency dimension to obtain the accumulated inner product value of each microphone pair in each candidate direction. Step S403: For each candidate direction, average the accumulated inner product value of all microphone pairs in the candidate direction to obtain the directional response power of the candidate direction.

[0043] Step S105: Use the guided response power spatial spectrum to generate a time-frequency mask, and apply the time-frequency mask to the multi-channel frequency domain signal to obtain enhanced frequency domain features.

[0044] In some embodiments, see Figure 5 Step S105 includes the following steps S501-S503: Step S501: Normalize the spatial spectrum of the steering response power in the azimuth dimension, and extract the directional information in the frequency dimension through a frequency mapping network to obtain frequency attention; Step S502: Average the normalized steering response power spatial spectrum along the angle, and extract global time weights through a time mapping network to obtain time attention; Step S503: Multiply the frequency attention with the time attention to obtain time-frequency masking.

[0045] Step S106: Based on the enhanced frequency domain characteristics, estimate the direction of arrival of the sound sources to obtain the direction of arrival of each sound source.

[0046] In some embodiments, see Figure 6 Step S106 includes the following steps S601-S604: Step S601: Encode the enhanced frequency domain features and the phase difference between the observation channels respectively to obtain the first encoded feature and the second encoded feature; Step S602: Use the cross-attention module to fuse the first encoded feature and the second encoded feature to obtain the fused feature; Step S603: Perform frequency dimension modeling, time dimension modeling and channel dimension modeling on the fused features in sequence to obtain deep features; Step S604: Perform multi-label classification based on the deep features, and output the prediction results of whether a sound source exists in each discrete angle category.

[0047] As one implementation method, step S603 specifically includes the following steps A1-A3: Step A1: Reshape the fused features into a first-dimensional form, use a bidirectional long short-term memory network for frequency dimension modeling, and introduce residual connections; Step A2: Reshape the features after frequency dimension modeling into a second dimension form, use a bidirectional long short-term memory network for time dimension modeling, and introduce residual connections; Step A3: Use the compression-excitation module to adaptively weight the features modeled in the time dimension in the channel dimension to obtain the deep features modeled in the channel dimension.

[0048] In one implementation, step S604 specifically includes the following steps B1-B3: Step B1: Process the deep features or global representation through a fully connected layer and a non-linear activation function to output a multi-label classification score vector, where each dimension of the score vector corresponds to a discrete angle category; Step B2: Convert the true sound source angle of arrival of each speech into a multi-label one-hot vector as the learning target; Step B3: Train the network using a weighted combination of binary cross-entropy loss and focusing loss as the total loss function.

[0049] in, For binary classification, cross-entropy loss, To focus on the losses, These are the weighting coefficients.

[0050] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific application examples.

[0051] Figure 7 This is a network structure diagram of the multi-source DOA estimation method proposed in this embodiment. Figure 8 This is a structural diagram of the spatial sensing module (SAM) and two encoders.

[0052] See Figure 7 and Figure 8 The multi-source direction-of-arrival estimation method based on inter-channel phase difference-guided spatial perception provided in this embodiment includes a data processing stage and a network inference stage. The following detailed description of each step is provided in conjunction with specific embodiments.

[0053] (1) Signal acquisition and time-frequency transformation In a noisy, reverberant environment that meets far-field conditions, by including M The multi-channel time-domain observation signal acquired by the microphone array of each element can be expressed as: (1) in, x ( t () represents the observed signal. x dir ( t () represents the direct sound component. x early ( t ) represents the early reverberation component. x late ( t (This refers to the late-stage reverberation component.) n ( t () represents the noise component, and t represents the index of the time sample point. This represents the number of signal sample points.

[0054] The multi-channel time-domain observation signal is subjected to a short-time Fourier transform (STFT) to convert it into a frequency-domain representation. (2) in, T and F These represent the number of time frames and the number of frequency points, respectively.n and f These are indices representing the frame number and frequency, respectively. X represents the complex space. dir ( n , f This can be represented as follows: (3) Where, a( θ q , n , f ) is the direct sound steering vector. θ q For the first q The angle of incidence of a sound source, S q ( n , f ) is the first q One sound source signal, Q This represents the number of sound sources. The purpose of multi-source DOA estimation is to estimate the angle of incidence of each sound source. θ q .

[0055] (2) Full-channel signal reconstruction To effectively extract spatial difference features between microphone arrays and facilitate subsequent calculations of inter-channel features, such as IPD, this embodiment introduces a channel pair reconstruction method. Specifically, it involves pairing all channels of the multi-channel signal after STFT to achieve structured recombination of inter-channel features, enhancing the array's spatial information representation capability. The constructed complex spectrum pair can be represented as: (4) in, i and j Indicates different microphone indices, [ ] T This represents the matrix transpose operation. The reconstructed complex spectrum signal is represented as: (5) in, M ( M -1) / 2 is the number of microphone pairs.

[0056] (3) Laplace normalization To effectively suppress the non-stationarity of speech signals and enhance the robustness of the model to variations in input feature distribution, this embodiment adopts a Laplace normalization method based on a dynamic smoothing factor. The core of this method lies in recursively estimating a time-varying normalization mean factor through an exponential smoothing mechanism. This factor can adaptively track the slow variation trend of the input signal energy while maintaining robustness against sudden changes. The specific implementation details are as follows: 3.1) Initialization and steady-state smoothing factor calculation: Initialize the historical mean =0, and calculate the steady-state smoothing factor based on the target smoothing window length L (set to 192 frames during training) . This factor determines the retention degree of historical information, the closer it is to 1, the greater the smoothing strength, and the calculation method is as follows: (6) 3.2) Dynamic smoothing factor calculation: Dynamically adjust the actual smoothing factor according to the current frame index , for the nth frame feature: When n <L, adopt as the dynamic factor, so that the estimated mean μ can be quickly adjusted from the initial value to a level close to the true average level of the signal, enabling the algorithm to converge rapidly in the initial stage.

[0057] When n ≥ L , perform smoothing using the steady-state factor .

[0058] 3.3) Recursive update of mean estimation: The mean update follows the standard form of Exponential Moving Average (EMA), and the formula is as follows: (7) Wherein, is the instantaneous mean of the current frame; the obtained current estimated value is the weighted sum of the historical mean and the current observation value, which realizes real-time tracking and smoothing of the signal mean.

[0059] 3.4) Normalize the input features: Divide the input full-channel paired complex spectrum by the dynamically estimated mean sequence to obtain normalized features.

[0060] (4) Construction of the theoretical template for the inter-channel phase difference of the direct path To achieve accurate modeling and matching of target sound source direction information, this embodiment proposes to construct a theoretical template for Direct-Path Inter-channel Phase Difference (DPIPD) based on array geometry and physical acoustics principles. This template pre-calculates the ideal phase difference in all candidate sound source directions, providing a physical prior for subsequent sound source localization and feature extraction. The specific implementation is as follows: 4.1) Based on the preset sound source direction search space, construct a discretized direction grid in spherical coordinates. Pitch angle Equal intervals are sampled within the range of [0, π radians]. A point, azimuth angle Equal intervals are sampled within the range of [-π, π] radians. These points together constitute n ele × n azi Candidate points for the direction of arrival of the sound source.

[0061] 4.2) Based on the far-field plane wave model and a given microphone array, for any candidate direction ( , ) and any pair of microphones ( The theoretical time difference (ITD) is calculated using the following formula: (8) in, 3 microphones Spatial coordinates; c r is the speed of sound; , ) 3 Here is the unit direction vector for this candidate direction: (9) 4.3) Convert the ideal time delay difference ITD into the phase difference IPD in the frequency domain. For the th k Frequency points f k The phase difference formula is: (10) in, f k [0, F ], FThis represents the number of frequency points.

[0062] 4.4) The template is fixed with an elevation angle of π / 2, and all candidate values ​​for the azimuth angle are retained. Only non-repeating microphone pairs are kept. The complex values ​​are then split into real and imaginary parts and concatenated in the frequency dimension. The final DPIPD template is as follows: (11) in, N pairs = M ( M -1) / 2 represents the number of microphone pairs, which will be used later. N pairs This is used to represent the number of microphone pairs.

[0063] (5) Observation IPD calculation The full-channel pair complex spectrum obtained using formula (5) The corresponding full-channel phase spectrum X′ was calculated. p For each microphone pair, the phase difference is calculated and its sine and cosine values ​​are taken to obtain a real-valued form. The IPD value is then observed. The calculation is as follows: (12) (13) Where, X′ p For the full-channel phase spectrum, p diff This represents the phase difference between each microphone pair. To facilitate subsequent spatial spectrum calculations, the observed IPD values ​​are reshaped as follows: .

[0064] (6) SRP spatial spectrum estimation based on template matching of observed IPD and spatial DPIPD To illustrate the relationship between the spatial spectrum estimation method based on observation IPD and spatial DPIPD template matching proposed in this embodiment and the traditional SRP-PHAT, we will first explain it from the theoretical derivation.

[0065] The traditional SRP-PHAT method can be obtained by calculating the frame-by-frame GCC-PHAT of all microphone pairs at all frequency points and taking the average value. m , m′ ) in the n Frames and frequencies f The phase-transformed weighted cross-power spectrum on the above is: (14) in, These represent microphones. , Frequency domain representation of the acquired signal, Represents the phase information of the cross power spectrum. For the complex representation of phase information, " " indicates conjugate transpose.

[0066] The GCC-PHAT for each microphone pair per frame can be calculated as follows: (15) in, Is and direction The time delay difference between the relevant microphone pairs, F For frequency points, This indicates the operation of taking the real part.

[0067] A spatial spectrum is constructed by weighting and averaging the results from all microphone pairs. : (16) The above describes the traditional SRP-PHAT algorithm. For a single-source case with no noise and no reverberation, the target sound source DOA is assumed to be... θ k The cross-power spectrum (14) is actually used to calculate the power at the source location. θ k Complex direct path IPD between microphone pairs: (17) in, Is this direction? The time delay difference between the microphone pairs. Here, we propose using the observation IPD of equation (13) instead of the direct path IPD in equation (15). The DPIPD template corresponds to the one in equation (15). The real-valued representation of is equivalent to a direction-dependent time delay compensation factor. Therefore, after reshaping both, the similarity is calculated by taking the inner product and accumulating it in the frequency dimension, and then averaging the results of all microphone pairs. This achieves a spatial spectrum estimation process equivalent to the traditional SRP-PHAT, essentially equivalent to the cumulative response of SRP-PHAT to the energy of the sound source direction. The final constructed SRP spatial spectrum P′ ( θ ) is represented as: (18) Where, r ′ IPDobs For the reshaped observation IPD, r ′ DPIPD ( θ) represents the reshaped DPIPD template, where "1" indicates that the pitch angle dimension is fixed at π / 2.

[0068] (7) Spatial perception masking module The Spatial Aware Module (SAM) uses the obtained SRP spatial spectrum to generate time-frequency masks for STFT features. Specifically, the spatial spectrum is normalized in the azimuth dimension, and the directional information in the frequency dimension is extracted through a frequency mapping network to obtain frequency attention. A f : (19) Among them, W freq {} represents the frequency mapping network, softmax is the normalization operation, and spectrum is the constructed SRP spatial spectrum.

[0069] The global temporal weight is calculated by averaging the normalized spatial spectrum along the angle, thus obtaining the temporal attention. A t : (20) Among them, W time {} represents the time-mapped network, and mean{} represents the angular averaging operation.

[0070] The time-frequency masking is obtained by merging the two. M=A f A t This enables physical prior-guided STFT feature enhancement, allowing it to adaptively focus on a more reliable region in the spatial direction on the time-frequency plane, effectively suppressing noise and interference components.

[0071] (8) Deep feature extraction The mask obtained by SAM M After applying the STFT feature, the enhanced STFT feature and the observed IPD feature are encoded separately to obtain the encoded features: (twenty one) (twenty two) in, Encoder represents the dot product operation. STFT {} and Encoder IPD {} represent the encoder operations for the two features, respectively. Then, the two features are fused using the Cross Attention (CA) module: (twenty three) in, F fusion The shape is B×C×T×F. This feature is first reshaped into (BT)×F×C for frequency modeling; then it is reshaped into (BF)×T×C for time modeling. Both processes are modeled using a bidirectional long short-term memory (Bi-LSTM) network, and residual connections are introduced during the modeling process to avoid information loss. The specific process is as follows: (twenty four) (25) in, g {} represents a combination function of linear mapping and nonlinear activation. F freq and F time These represent the corresponding frequency and time-modeled features, respectively.

[0072] (9) Inter-channel attention module The inter-channel attention module, also known as the Squeeze-and-Excitation Block (SEBlock), acts on features. F time Modeling is performed along the channel dimension. It achieves adaptive channel weighting through a "compression-activation" mechanism to enhance effective features. First, global average pooling is used to compress the frequency and time dimensions, resulting in channel vectors: (26) in, T and F These represent the number of time frames and the number of frequency points, respectively. n and f These are represented as the frame number and frequency index, respectively. The vector is then fed into two convolutional layers and passed through a non-linear activation function to obtain channel weights. These weights are then multiplied channel-by-channel with the input, and residuals are introduced to obtain the enhanced features. (27) Where σ{} represents Sigmoid activation and Conv2d{} represents a convolutional layer.

[0073] (10) Hybrid Pooling Module The Hybrid Pooling Module (HPM) performs pooling simultaneously along both the frequency and time dimensions, and obtains a global representation by concatenating the results of the two pooling operations. Specifically, for the frequency-dimensional pooling operation, it first averages the results along the time dimension, then performs a linear transformation along the channel dimension to generate a frequency score, and finally performs softmax normalization to obtain the frequency weights. (28) Here, `Linear{}` ​​represents the linear layer operation. Finally, the weighted summation along the frequency dimension yields the frequency pooling result: (29) in, F freq-p The shape is B×C. Similarly, for pooling operations in the time dimension, we first average in the frequency dimension, then perform a linear transformation on the channel dimension to generate time scores, and finally perform softmax normalization to obtain time weights. w time Finally, the weighted averages are summed over time to obtain the time pooling result. F time-p Finally, the two results are concatenated and aggregated to obtain the global representation: (30) in, F hybr-p The shape is B×2C. `concat{}` performs the concatenation operation.

[0074] (11) Multi-label classifier The multi-label classifier outputs a multi-label DOA classification score vector by passing the obtained global representation through two fully connected layers and a non-linear activation function. (31) Among them, angle logits The shape is B×180, representing the network's prediction score for 180 discrete angle categories, with each category corresponding to an angle resolution of 2 degrees.

[0075] (12) Learning objectives and loss function The learning objective of the network is to convert the true sound source angle of arrival for each speech into a (0,1) multi-label one-hot vector to indicate whether a sound source exists in each angle category.

[0076] The network employs a weighted combination of binary cross-entropy (BCE) and focal loss for multi-label DOA classification training. BCE provides basic global supervision, ensuring stable convergence across all classes; while Focal Loss focuses on learning from difficult samples, reducing the weight of easily classified samples and increasing the contribution of difficult samples, thus enhancing the model's discriminative ability in complex scenarios. Dynamic algorithms are also introduced. Adjusting the weights of positive and negative samples effectively alleviates the imbalance problem of sound source direction angle categories. Total loss L total Specifically, it can be expressed as: (32) in, w BCE ∈[0,1] is used to control the weights of both, and is set to 0.2 in this embodiment. The BCE loss is calculated based on the network's predicted angle_logits and the ground truth labels: (33) in, a For the predicted angle_logits, b ∈{0,1} is the truth label, and σ{} is the Sigmoid function. Based on BCE, the hard sample scaling factor (1) is used. p c ) γ Adjusting the focus level yields Focal Loss: (34) (35) in, The positive and negative sample weights are dynamically calculated. To emphasize the intensity of attention given to difficult samples, this embodiment sets it to 2.

[0077] (13) Advantages and beneficial effects In summary, compared with the prior art, the method provided in this application has at least the following advantages and beneficial effects: 1) This application obtains comprehensive spatial difference information by reconstructing the signal across all channels to calculate the phase difference between observation channels. Combined with a theoretical template for the phase difference between direct path channels constructed based on array geometry and physical acoustics principles, a high-resolution steering response power spatial spectrum is generated through matching calculations. Based on this, a spatial perception masking module is designed to generate time-frequency masks for STFT features using this spatial spectrum. This achieves adaptive enhancement of frequency domain features guided by physical priors, enabling the network to automatically focus on a more spatially reliable region in the time-frequency plane. This effectively suppresses noise and interference components, significantly improving spatial perception and localization capabilities in multi-source scenarios.

[0078] 2) This application integrates the phase difference between observation channels as a spatial feature with the enhanced frequency domain feature through a cross-attention module, and models it in conjunction with the three dimensions of frequency, time and channel, realizing multi-dimensional interaction of deep features. This further suppresses the interference of background noise and reverberation on feature extraction, enabling the network to maintain stable and accurate angle of arrival estimation performance in multi-source overlapping and non-ideal acoustic scenarios.

[0079] 3) This application introduces multi-label classification supervision into the end-to-end deep neural network and combines a weighted combination of binary cross-entropy and focusing loss to achieve effective learning of difficult samples and sparse positive samples, thereby improving the training stability and localization accuracy of the network.

[0080] This application embodiment also provides a multi-source direction-of-arrival estimation device that can implement the above-described method. The device includes: The time-frequency conversion module is used to acquire multi-channel time-domain observation signals collected by the microphone array, and to perform time-frequency conversion on the multi-channel time-domain observation signals to obtain multi-channel frequency-domain signals. The observation phase difference calculation module is used to calculate the phase difference between observation channels based on the multi-channel frequency domain signal. The theoretical template construction module is used to construct a theoretical template for the phase difference between direct path channels based on the microphone array geometry and the preset sound source direction search space; The spatial spectrum construction module is used to match and calculate the similarity between the phase difference between the observation channels and the theoretical template of the phase difference between the direct path channels corresponding to the direction of each candidate sound source to obtain the spatial spectrum of the steering response power. A spatially aware masking module is used to generate a time-frequency mask using the spatial spectrum of the steering response power, and to apply the time-frequency mask to the multi-channel frequency domain signal to obtain enhanced frequency domain features. The direction-of-arrival estimation module is used to estimate the direction of arrival of the sound sources based on the enhanced frequency domain characteristics, so as to obtain the direction of arrival of each sound source.

[0081] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0082] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0083] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0084] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0085] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0086] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0087] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0088] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0089] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0090] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0091] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0092] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0094] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0095] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0097] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0098] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0099] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for estimating the direction of arrival (DOA) of multiple sound sources, characterized in that, The method includes the following steps: Acquire multi-channel time-domain observation signals collected by a microphone array, and perform time-frequency transformation on the multi-channel time-domain observation signals to obtain multi-channel frequency-domain signals; Calculate the phase difference between the observation channels based on the multi-channel frequency domain signal; Based on the microphone array geometry and the preset sound source direction search space, a theoretical template for the phase difference between direct path channels is constructed. The phase difference between the observation channels is matched and similarity is calculated with the theoretical template of the phase difference between the direct path channels corresponding to the direction of each candidate sound source to obtain the spatial spectrum of the steering response power. The time-frequency mask is generated using the steering response power spatial spectrum, and then applied to the multi-channel frequency domain signal to obtain enhanced frequency domain features. Based on the enhanced frequency domain characteristics, the direction of arrival (DOA) of the sound sources is estimated to obtain the DOA of each sound source.

2. The method according to claim 1, characterized in that, The calculation of the phase difference between observation channels based on the multi-channel frequency domain signal includes: All channels of the multi-channel frequency domain signal are paired up to construct complex spectrum pairs, thereby realizing the structural recombination of features between channels. The phase spectrum of each microphone pair is calculated based on the reconstructed complex spectrum. The phase difference of each microphone pair is calculated based on the phase spectrum. The sine and cosine values ​​of the phase difference are then taken to obtain the real-valued phase difference between the observation channels.

3. The method according to claim 1, characterized in that, The step of constructing a theoretical template for the phase difference between channels along a direct path based on the microphone array geometry and a preset sound source direction search space includes: Discretized direction grids are constructed in spherical coordinates, where the pitch angle is sampled at equal intervals in the range of [0, π] and the azimuth angle is sampled at equal intervals in the range of [-π, π], forming multiple candidate points for the direction of arrival of sound sources; Based on the far-field plane wave model, calculate the ideal time delay difference for any candidate direction and any pair of microphones: The ideal time delay difference is converted into an inter-channel phase difference in the frequency domain; The complex value of the phase difference between the channels is split into real and imaginary parts and then spliced ​​together in the frequency dimension to obtain the theoretical template of the phase difference between the channels of the direct path.

4. The method according to claim 1, characterized in that, The process of matching and similarity calculation between the phase difference between the observation channels and the theoretical template of the phase difference between the direct path channels corresponding to the directions of each candidate sound source to obtain the steering response power spatial spectrum includes: The theoretical templates for the phase difference between the observation channels and the phase difference between the direct path channels of each candidate direction are reshaped respectively; For each pair of microphones, the inner product of the reshaped phase difference between the observation channels and the theoretical template of the phase difference between the direct path channels is calculated and accumulated in the frequency dimension to obtain the accumulated inner product value of each microphone pair in each candidate direction. For each candidate direction, the directional response power for that candidate direction is obtained by averaging the accumulated inner product values ​​of all microphone pairs in that candidate direction.

5. The method according to claim 1, characterized in that, The generation of time-frequency masking using the steering response power spatial spectrum includes: The spatial spectrum of the steering response power is normalized in the azimuth dimension, and the directional information in the frequency dimension is extracted through a frequency mapping network to obtain frequency attention. The normalized steering response power spatial spectrum is averaged along the angle, and the global time weight is extracted through a time mapping network to obtain time attention. Multiplying the frequency attention by the time attention yields time-frequency masking.

6. The method according to claim 1, characterized in that, The step of estimating the direction of arrival (DOA) of the sound sources based on the enhanced frequency domain characteristics to obtain the DOA of each sound source includes: The enhanced frequency domain features and the phase difference between the observation channels are encoded respectively to obtain the first encoded feature and the second encoded feature; The first encoded feature and the second encoded feature are fused using a cross-attention module to obtain a fused feature; The fused features are sequentially modeled in the frequency dimension, time dimension, and channel dimension to obtain deep features; Based on the deep features, multi-label classification is performed, and the prediction results of whether a sound source exists in each discrete angle category are output.

7. The method according to claim 6, characterized in that, The fused features are sequentially modeled in the frequency dimension, time dimension, and channel dimension to obtain deep features, including: The fused features are restructured into a first-dimensional form, and a bidirectional long short-term memory network is used for frequency dimension modeling, with residual connections introduced. The features modeled in the frequency dimension are reorganized into a second-dimensional form, and a bidirectional long short-term memory network is used for time dimension modeling, with residual connections introduced. The compression-excitation module is used to adaptively weight the features modeled in the time dimension in the channel dimension to obtain the deep features modeled in the channel dimension.

8. The method according to claim 1, characterized in that, Before performing time-frequency transformation on the multi-channel time-domain observation signal, the method further includes: The multi-channel time-domain observation signal is normalized using a Laplace normalization method based on a dynamic smoothing factor. The Laplace normalization method includes: calculating a steady-state smoothing factor based on the target smoothing window length; dynamically adjusting the actual smoothing factor according to the current frame number; recursively updating the mean estimate using an exponential moving average method; and dividing the input signal by the dynamically estimated mean sequence to obtain the normalized signal.

9. A multi-source direction-of-arrival estimation device, characterized in that, The device includes: The time-frequency conversion module is used to acquire multi-channel time-domain observation signals collected by the microphone array, and to perform time-frequency conversion on the multi-channel time-domain observation signals to obtain multi-channel frequency-domain signals. The observation phase difference calculation module is used to calculate the phase difference between observation channels based on the multi-channel frequency domain signal. The theoretical template construction module is used to construct a theoretical template for the phase difference between direct path channels based on the microphone array geometry and the preset sound source direction search space; The spatial spectrum construction module is used to match and calculate the similarity between the phase difference between the observation channels and the theoretical template of the phase difference between the direct path channels corresponding to the direction of each candidate sound source to obtain the spatial spectrum of the steering response power. A spatially aware masking module is used to generate a time-frequency mask using the spatial spectrum of the steering response power, and to apply the time-frequency mask to the multi-channel frequency domain signal to obtain enhanced frequency domain features. The direction-of-arrival estimation module is used to estimate the direction of arrival of the sound sources based on the enhanced frequency domain characteristics, so as to obtain the direction of arrival of each sound source.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 8.

Images