Direction of arrival estimation method and system based on graph convolutional network, medium and equipment
By based on the method of convolutional networks, the spherical microphone array is used to obtain acoustic data and extract feature extraction, which solves the deficiencies of the traditional Boda directional estimation algorithm in complex environments, and achieves accurate positioning and robustness of sound source.
Patent Information
- Application Number
- CN202411402862.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-01-24
AI Technical Summary
Traditional wave-distance direction estimation algorithms perform poorly in complex environments and high interference situations, have high computational complexity and limited adaptability to the environment.
A method based on the convolutional network, obtaining acoustics data through the spherical microphone array, converting to the ball harmonic field, extracting feature vectors, and entering the preset convolutional network to determine the three -dimensional space coordinates of the sound source.
The accurate positioning of the sound source has been realized, and the accuracy and robustness of the Boda direction estimation in the complex acoustic environment can be improved, and the stable positioning effect can be maintained in low signal -to -noise ratio and high reverberation.
Smart Images
Figure CN119355632A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of array signal processing, and in particular to a method, system, medium and device for estimating a direction of arrival based on a graph convolutional network. Background Art
[0002] Acoustic signal processing plays a vital role in modern scientific and technological applications. As one of the core technologies in this field, Direction of Arrival (DoA) estimation has a direct and far-reaching impact on the performance of various systems. Direction of Arrival estimation technology determines the direction of the sound source by analyzing the received sound waves, and has been widely used in wireless communications, airborne radar, underwater measurement and other scenarios.
[0003] Traditional wave direction estimation algorithms include two categories. The first category includes beamforming algorithms, which form a directional beam by synthesizing the output of array sensors. Such as Digital Beamforming (DBF) and Minimum Variance Distortionless Response (MVDR). The second category includes subspace-based super-resolution angle estimation algorithms, such as Estimation of Signal Parameters via Rotational Invariance Techniques (ESPRIT) and Multiple Signal Classification (MUSIC). In addition, there are some derivative algorithms. These algorithms usually rely on the structure of traditional arrays and high signal-to-noise ratio conditions, and their computational complexity is high and their adaptability to the environment is limited.
[0004] In recent years, with the development of deep learning technology, researchers have begun to explore the application of neural networks to DOA estimation to solve the problems existing in traditional methods. Deep learning methods show the potential for accurate DOA estimation in complex environments by learning the complex nonlinear relationship between array output and spatial signal source position. Some studies have shown that DOA estimation based on neural networks has faster calculation speed and stronger estimation effect than traditional algorithms, and can adapt to more complex environments and interference situations.
[0005] However, despite some progress, these methods still have some limitations, such as the sensitivity of the model to the data structure and the limited ability to model complex spatial relationships. The introduction of Graph Convolutional Network (GCN) has opened up a new research direction for DOA estimation. In the future, this technology is expected to be applied in a wider range of fields such as intelligent transportation systems, wireless communication network optimization, and even environmental monitoring. Summary of the invention
[0006] In view of the defects in the prior art, the purpose of the present invention is to provide a method, system, medium and device for DOA estimation based on graph convolutional network.
[0007] To achieve the above object, according to one aspect of the present disclosure, a method for estimating a direction of arrival based on a graph convolutional network is provided, comprising:
[0008] Acquiring an acoustic data set using a spherical microphone array, wherein the acoustic data set includes acoustic data;
[0009] Converting the acoustic data into a spherical harmonic domain and determining spherical harmonic coefficients of the acoustic data;
[0010] Performing information extraction processing on the spherical harmonic coefficients of the acoustic data to determine feature vectors of multiple time frames of the acoustic data;
[0011] Averaging the feature vectors of multiple time frames of the acoustic data and combining them by frequency to determine an input feature matrix;
[0012] The input feature matrix is input into a preset graph convolutional network to determine the three-dimensional spatial coordinates of the sound source.
[0013] Optionally, the method further comprises:
[0014] The spherical harmonic coefficients of the acoustic data are normalized to determine the normalized spherical harmonic coefficients.
[0015] Optionally, performing information extraction processing on the spherical harmonic coefficients of the acoustic data to determine feature vectors of multiple time frames of the acoustic data includes:
[0016] Performing information extraction processing on the normalized spherical harmonic coefficients to determine amplitude information, sine phase information, and cosine phase information of the acoustic data;
[0017] Determine feature vectors of a plurality of time frames of the acoustic data according to the amplitude information of the acoustic data, the sine phase information, and the cosine phase information.
[0018] Optionally, the preset graph convolutional network includes a feedforward neural network layer, a graph convolution layer, a pooling layer and a fully connected layer.
[0019] Optionally, inputting the input feature matrix into a preset graph convolutional network to determine the three-dimensional spatial coordinates of the sound source includes:
[0020] Preprocessing the input feature matrix using the feedforward neural network layer to determine preprocessed features;
[0021] Inputting the preprocessed features into the graph convolution layer, using the graph convolution layer to update the features of each node in the graph structure and the K nearest nodes corresponding to each node, and determining the updated node features;
[0022] The updated node features are aggregated using the pooling layer to determine graph-level features;
[0023] The fully connected layer is used to perform regression analysis on the graph-level features, and the three-dimensional spatial coordinates of the sound source are output.
[0024] Optionally, inputting the preprocessed features into the graph convolution layer, using the graph convolution layer to perform feature update on each node in the graph structure and K nearest nodes corresponding to each node, and determining updated node features, includes:
[0025] In the graph convolution layer, a weighted sum combination operation is performed on the features of each node in the graph structure and the features of the K nearest nodes corresponding to each node to determine the node features after the combination operation;
[0026] Aggregate the node features of the K nearest nodes corresponding to each node after the combination operation to determine the updated node feature corresponding to each node.
[0027] Optionally, the step of inputting the preprocessed features into the graph convolution layer, using the graph convolution layer to update features of each node in the graph structure and K nearest nodes corresponding to each node, and determining updated node features, further includes:
[0028] Determine the Euclidean distance between each node of the graph structure according to the preprocessed features;
[0029] According to the Euclidean distance between each node of the graph structure, a K-nearest neighbor algorithm is used to determine the K nearest nodes corresponding to each node.
[0030] Optionally, the method further comprises:
[0031] Convert the three-dimensional spatial coordinates of the sound source into a vector to determine a predicted coordinate vector of the sound source;
[0032] Determining an angle between the predicted coordinate vector of the sound source and the real coordinate vector of the sound source according to the predicted coordinate vector of the sound source and the real coordinate vector of the sound source;
[0033] The model performance of the preset graph convolutional network is determined according to the angle between the predicted coordinate vector of the sound source and the real coordinate vector of the sound source.
[0034] According to a second aspect of the present disclosure, a direction of arrival estimation system based on a graph convolutional network is provided, comprising:
[0035] A data acquisition module, used for acquiring an acoustic data set using a spherical microphone array, wherein the acoustic data set includes acoustic data;
[0036] A data conversion module, used for converting the acoustic data into a spherical harmonic domain and determining the spherical harmonic coefficients of the acoustic data;
[0037] An information extraction module, used to perform information extraction processing on the spherical harmonic coefficients of the acoustic data to determine the feature vectors of multiple time frames of the acoustic data;
[0038] A feature combination module, used for averaging the feature vectors of multiple time frames of the acoustic data and combining them by frequency to determine an input feature matrix;
[0039] The sound source direction estimation module is used to input the input feature matrix into a preset graph convolutional network to determine the three-dimensional spatial coordinates of the sound source.
[0040] According to a third aspect of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method provided in the first aspect of the present disclosure are implemented.
[0041] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:
[0042] a memory having a computer program stored thereon;
[0043] A processor is used to execute the computer program in the memory to implement the steps of the method provided in the first aspect of the present disclosure.
[0044] Compared with the prior art, the embodiments of the present disclosure have at least one of the following beneficial effects:
[0045] Through the above technical solution, a spherical microphone array is used to acquire acoustic data, and a preset graph convolutional network is introduced to integrate the all-round information in the multi-channel signal of the acoustic data, effectively modeling the complex spatial relationship in the spherical microphone array signal, and realizing the precise positioning of the sound source; and, in the face of performance fluctuations in low signal-to-noise ratio and high reverberation, the use of the preset graph convolutional network can show stronger robustness, effectively deal with interference factors in the acoustic signal, and maintain a stable positioning effect in a variety of environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Other features, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:
[0047] Figure 1 The present invention is a flowchart of a method for estimating a direction of arrival based on a graph convolutional network according to an exemplary embodiment.
[0048] Figure 2 It is a schematic diagram of the structure of a preset graph convolutional network according to an exemplary embodiment.
[0049] Figure 3 It is a schematic diagram showing the relationship between the root mean square error and the signal-to-noise ratio of a model according to an exemplary embodiment.
[0050] Figure 4 It is a schematic diagram showing the relationship between the root mean square error and the reverberation time of a model according to an exemplary embodiment.
[0051] Figure 5 It is a block diagram of a direction of arrival estimation system based on a graph convolutional network according to an exemplary embodiment. DETAILED DESCRIPTION
[0052] The present disclosure is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present disclosure, but are not intended to limit the present disclosure in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present disclosure. These all fall within the scope of protection of the present disclosure.
[0053] Figure 1 The present invention is a flowchart of a method for estimating a direction of arrival based on a graph convolutional network according to an exemplary embodiment.
[0054] like Figure 1 As shown, the present disclosure provides a method for estimating a direction of arrival based on a graph convolutional network, including S11 to S15.
[0055] S11, an acoustic dataset is acquired using a spherical microphone array.
[0056] The acoustic data set includes acoustic data, which can be collected in an environment with indoor noise and reverberation.
[0057] As an example, the spherical microphone may be a rigid spherical microphone, and the spherical microphone array is a rigid spherical microphone array configured with 32 microphone elements, wherein each microphone element is distributed on a spherical surface to ensure that sound waves can be acquired in all directions.
[0058] The spherical harmonic order of the spherical microphone array is 4.
[0059] S12, converting the acoustic data into a spherical harmonic domain, and determining the spherical harmonic coefficients of the acoustic data.
[0060] Among them, the acoustic data is a time domain signal. The acoustic data is first converted from the time domain to the frequency domain through Fourier transform, and then the acoustic data in the frequency domain is converted to the harmonic domain.
[0061] The spherical harmonic coefficients are denoted by x nm (τ,k h ), the spherical harmonic coefficients contain the spatial information of the sound field for subsequent feature extraction.
[0062] S13, performing information extraction processing on the spherical harmonic coefficients of the acoustic data to determine feature vectors of multiple time frames of the acoustic data.
[0063] The information extracted by performing information extraction processing on the spherical harmonic coefficients of the acoustic data includes: amplitude information of the acoustic data, phase information of the sine, and phase information of the cosine.
[0064] Feature vectors of multiple time frames of the acoustic data are constructed based on the amplitude information of the acoustic data, the phase information of the sine, and the phase information of the cosine.
[0065] S14, averaging the feature vectors of multiple time frames of the acoustic data and combining them by frequency to determine an input feature matrix.
[0066] The averaging of the feature vectors of multiple time frames of the acoustic data means summing the feature vectors of each time frame and dividing the sum by the total number of time frames.
[0067] The averaged feature vectors of each time frame are combined according to frequency to determine the input feature matrix.
[0068] S15, inputting the input feature matrix into a preset graph convolutional network to determine the three-dimensional spatial coordinates of the sound source.
[0069] Through the above technical solution, a spherical microphone array is used to acquire acoustic data, and a preset graph convolutional network is introduced to integrate the all-round information in the multi-channel signal of the acoustic data, effectively modeling the complex spatial relationship in the spherical microphone array signal, and realizing the precise positioning of the sound source; and, in the face of performance fluctuations in low signal-to-noise ratio and high reverberation, the use of the preset graph convolutional network can show stronger robustness, effectively deal with interference factors in the acoustic signal, and maintain a stable positioning effect in a variety of environments.
[0070] In a possible embodiment, a method for estimating a direction of arrival based on a graph convolutional network further includes S16.
[0071] S16, normalizing the spherical harmonic coefficients of the acoustic data to determine the normalized spherical harmonic coefficients.
[0072] Wherein, step S16 is executed after step S12.
[0073] In a possible embodiment, S13 may include S21 to S22.
[0074] S21, performing information extraction processing on the normalized spherical harmonic coefficients to determine amplitude information, sine phase information, and cosine phase information of the acoustic data.
[0075] S22, determining feature vectors of multiple time frames of the acoustic data according to the amplitude information, sine phase information, and cosine phase information of the acoustic data.
[0076] Specifically, the amplitude information, sine phase information and cosine phase information in each time frame are combined and concatenated to form a feature vector of the time frame, thereby determining feature vectors of multiple time frames of acoustic data.
[0077] Figure 2 It is a schematic diagram of the structure of a preset graph convolutional network according to an exemplary embodiment.
[0078] like Figure 2 As shown, in a possible embodiment, the preset graph convolutional network (GCN) includes a feedforward neural network layer, a graph convolution layer, a pooling layer, and a fully connected layer.
[0079] The preset graph convolutional network also includes an input layer for receiving an input feature matrix, and the dimension of the input layer is 54×72.
[0080] The feedforward neural network layer consists of two fully connected layers. The output dimension of each fully connected layer is 54×256, and the activation function of each layer is ReLU.
[0081] The graph convolution layer consists of three graph convolution layers and a Dropout layer, including graph convolution layer 1, graph convolution layer 2, and graph convolution layer 3. Among them, the type of graph convolution layer 1 is GIN (Graph Isomorphism Network), the output dimension is 54×128, the activation function is GELU, and the expansion factor is 1; the type of graph convolution layer 2 is GIN, the output dimension is 54×64, the activation function is GEIU, and the expansion factor is 2; the type of graph convolution layer 3 is GIN, the output dimension is 54×32, the activation function is GEIU, and the expansion factor is 2; the Dropout rate of the Dropout layer is set to 0.1, and the Dropout layer is used to prevent overfitting of the graph convolution network.
[0082] The output dimension of the pooling layer is set to 32 to obtain the graph-level representation of features, namely, graph-level features.
[0083] The output dimension of the fully connected layer is set to 3, and the activation function uses Tanh to output three-dimensional space coordinates.
[0084] In a possible embodiment, the preset graph convolutional network of the present invention can use a simulated data set for model training, wherein the sampling frequency of the simulated data set is set to 16kHz, a Hanning window is used, a frame field is 256, and a frame shift is 25% of the window length.
[0085] During the training process, AdamW is used as the optimizer, MSELoss is used as the loss function, and the early stopping method is used in the training process to prevent the graph convolutional network from overfitting. The early stopping patience value is set to 30.
[0086] In a possible embodiment, S15 may include S41 to S44.
[0087] S41, using a feedforward neural network layer to preprocess the input feature matrix and determine the preprocessed features.
[0088] Among them, the preprocessing method includes mapping the node features of the original feature matrix of the input feature matrix to a higher-dimensional feature space through the fully connected layer of the feedforward neural network layer, for example, from the original 72 dimensions to 256 dimensions, and combining the nonlinear activation function ReLU to increase the nonlinear expression ability of the model. The preprocessed features are features mapped to a higher-dimensional feature space.
[0089] Through this preprocessing method, the original features are mapped to a high-dimensional feature space that is more suitable for graph convolution processing, providing richer node feature representation for subsequent graph convolution layers.
[0090] S42, input the preprocessed features into the graph convolution layer, use the graph convolution layer to update the features of each node in the graph structure and the K nearest nodes corresponding to each node, and determine the updated node features.
[0091] S43, a pooling layer is used to aggregate the updated node features to determine the graph-level features.
[0092] S44 uses a fully connected layer to perform regression analysis on the graph-level features and output the three-dimensional spatial coordinates of the sound source.
[0093] Traditional direction of arrival estimation methods perform poorly when processing non-Euclidean data. The present invention uses a graph convolutional network to process graph-structured data. It directly uses the connection information between nodes to effectively model the complex spatial relationship of spherical microphone array signals, improves the accuracy and robustness of direction of arrival (DoA) estimation in complex acoustic environments, and provides new perspectives and tools for using non-traditional data structures to process complex acoustic estimation problems.
[0094] In a possible embodiment, S42 may also include S51 to S52.
[0095] S51, determining the Euclidean distance between each node of the graph structure according to the preprocessed features.
[0096] Based on the preprocessed features, a graph structure is constructed with each frequency point as a node, so as to determine the Euclidean distance between each node.
[0097] S52, according to the Euclidean distance between each node of the graph structure, a K-nearest neighbor algorithm is used to determine the K nearest nodes corresponding to each node.
[0098] Among them, the K-nearest neighbor (K-NN) algorithm is used to determine the K nearest neighbors of each node based on the Euclidean distance between nodes to generate the adjacency matrix of the graph, providing topological structure information for the subsequent node feature update in the graph convolution layer.
[0099] In a possible embodiment, S42 may further include S53 to S54.
[0100] S53, in the graph convolution layer, a weighted sum combination operation is performed on the features of each node in the graph structure and the features of the K nearest nodes corresponding to each node to determine the node features after the combination operation.
[0101] Among them, the characteristics of each node and the characteristics of the K nearest nodes corresponding to each node can be determined based on each node in the graph structure and the K nearest nodes corresponding to each node.
[0102] The node features of each node as the central node are updated through the combination operation.
[0103] S54, performing aggregation processing on the node features of the K nearest nodes corresponding to each node after the combination operation, and determining the updated node feature corresponding to each node.
[0104] Among them, through aggregation processing, the node features of each node as the central node are updated again, so as to obtain the updated node features of each node, and the updated node features corresponding to each node are the final node feature representations of each node.
[0105] In the present disclosure, the method of step S42 is performed in the order of steps S51 to S54.
[0106] In a possible embodiment, a method for estimating a direction of arrival based on a graph convolutional network may also include S17 to S19.
[0107] S17, converting the three-dimensional spatial coordinates of the sound source into a vector, and determining the predicted coordinate vector of the sound source.
[0108] S18, determining the angle between the predicted coordinate vector of the sound source and the real coordinate vector of the sound source according to the predicted coordinate vector of the sound source and the real coordinate vector of the sound source.
[0109] S19, determining the model performance of the preset graph convolutional network according to the angle between the predicted coordinate vector of the sound source and the actual coordinate vector of the sound source.
[0110] In the present disclosure, the evaluation indicators of the model performance of the preset graph convolutional network may include mean absolute error (MAE), root mean square error (RMSE) and angle accuracy.
[0111] In a possible embodiment, the mean absolute error (MAE) of the preset graph convolutional network model proposed in the present disclosure is compared with the direction of arrival estimation method using the traditional CNN model, the m-CNN model, and the MUSIC algorithm.
[0112] As shown in Table 1:
[0113]
[0114]
[0115] Table 1
[0116] As shown in Table 1, the m-CNN model is a model that uses the same framework as the graph convolutional network model in the present disclosure but uses graph convolution to replace ordinary convolution, which is used to compare the effectiveness of using graph convolution.
[0117] Experimental results show that in terms of positioning accuracy, the graph convolution network model proposed in this disclosure shows the best performance compared with the traditional CNN model, the same-frame graph convolution replacing the ordinary convolution model, and the MUSIC algorithm.
[0118] Figure 3 It is a schematic diagram showing the relationship between the root mean square error and the signal-to-noise ratio of a model according to an exemplary embodiment. Figure 4 It is a schematic diagram showing the relationship between the root mean square error and the reverberation time of a model according to an exemplary embodiment.
[0119] The root mean square error (RMSE) can be used to measure the robustness of the model under different reverberation and noise conditions.
[0120] like Figure 3 As shown, taking the root mean square error (RMSE) as the evaluation index, for the signal-to-noise ratio (SNR), it can be observed that the root mean square error decreases as the signal-to-noise ratio increases. The graph convolutional network model proposed in the present disclosure maintains the best performance under various signal-to-noise ratio conditions, which means that the method proposed in the present disclosure can effectively resist noise.
[0121] like Figure 4 As shown, for the reverberation time (RT60), the root mean square error increases with the increase of the reverberation time. When the reverberation time reaches a higher level, the graph convolutional network model proposed in the present invention has a significant improvement in performance compared with other models. Therefore, the method proposed in the present invention can also effectively resist reverberation.
[0122] Figure 5 It is a block diagram of a direction of arrival estimation system based on a graph convolutional network according to an exemplary embodiment.
[0123] Based on the same concept, the present disclosure also provides a direction of arrival estimation system 100 based on a graph convolutional network, such as Figure 5 As shown, it includes: a data acquisition module 110, a data conversion module 120, an information extraction module 130, a feature combination module 140, and a sound source direction estimation module 150.
[0124] A data acquisition module 110, configured to acquire an acoustic data set using a spherical microphone array, wherein the acoustic data set includes acoustic data;
[0125] A data conversion module 120, for converting the acoustic data into a spherical harmonic domain and determining spherical harmonic coefficients of the acoustic data;
[0126] An information extraction module 130, configured to perform information extraction processing on the spherical harmonic coefficients of the acoustic data to determine feature vectors of multiple time frames of the acoustic data;
[0127] A feature combination module 140 is used to average the feature vectors of multiple time frames of the acoustic data and combine them by frequency to determine an input feature matrix;
[0128] The sound source direction estimation module 150 is used to input the input feature matrix into a preset graph convolutional network to determine the three-dimensional spatial coordinates of the sound source.
[0129] Through the above technical solution, a spherical microphone array is used to acquire acoustic data, and a preset graph convolutional network is introduced to integrate the all-round information in the multi-channel signal of the acoustic data, effectively modeling the complex spatial relationship in the spherical microphone array signal, and realizing the precise positioning of the sound source; and, in the face of performance fluctuations in low signal-to-noise ratio and high reverberation, the use of the preset graph convolutional network can show stronger robustness, effectively deal with interference factors in the acoustic signal, and maintain a stable positioning effect in a variety of environments.
[0130] Regarding the embodiment of the above system, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0131] Based on the same concept as above, in another embodiment of the present disclosure, there is also provided an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor is used to execute a method for estimating a direction of arrival based on a graph convolutional network when executing the program.
[0132] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.
[0133] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.
[0134] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.
[0135] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.
[0136] In an embodiment of the present disclosure, a non-temporary computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a complex method for estimating the direction of arrival based on a graph convolutional network in any of the above embodiments are implemented.
[0137] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0138] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0139] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0141] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present disclosure.
[0142] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.
Claims
1. A method for estimating direction of arrival based on graph convolutional network, characterized in that: include: Acquiring an acoustic data set using a spherical microphone array, wherein the acoustic data set includes acoustic data; Converting the acoustic data into a spherical harmonic domain and determining spherical harmonic coefficients of the acoustic data; Performing information extraction processing on the spherical harmonic coefficients of the acoustic data to determine feature vectors of multiple time frames of the acoustic data; Averaging the feature vectors of multiple time frames of the acoustic data and combining them by frequency to determine an input feature matrix; The input feature matrix is input into a preset graph convolutional network to determine the three-dimensional spatial coordinates of the sound source.
2. The method according to claim 1, characterized in that The method further comprises: The spherical harmonic coefficients of the acoustic data are normalized to determine the normalized spherical harmonic coefficients.
3. The method according to claim 2, characterized in that The step of performing information extraction processing on the spherical harmonic coefficients of the acoustic data to determine the feature vectors of multiple time frames of the acoustic data includes: Performing information extraction processing on the normalized spherical harmonic coefficients to determine amplitude information, sine phase information, and cosine phase information of the acoustic data; Determine feature vectors of a plurality of time frames of the acoustic data according to the amplitude information of the acoustic data, the sine phase information, and the cosine phase information.
4. The method according to claim 1, characterized in that: The preset graph convolutional network includes a feedforward neural network layer, a graph convolution layer, a pooling layer and a fully connected layer; Inputting the input feature matrix into a preset graph convolutional network to determine the three-dimensional spatial coordinates of the sound source includes: Preprocessing the input feature matrix using the feedforward neural network layer to determine preprocessed features; Inputting the preprocessed features into the graph convolution layer, using the graph convolution layer to update the features of each node in the graph structure and the K nearest nodes corresponding to each node, and determining the updated node features; The updated node features are aggregated using the pooling layer to determine graph-level features; The fully connected layer is used to perform regression analysis on the graph-level features, and the three-dimensional spatial coordinates of the sound source are output.
5. The method according to claim 4, characterized in that The step of inputting the preprocessed features into the graph convolution layer, using the graph convolution layer to update the features of each node in the graph structure and the K nearest nodes corresponding to each node, and determining the updated node features includes: In the graph convolution layer, a weighted sum combination operation is performed on the features of each node in the graph structure and the features of the K nearest nodes corresponding to each node to determine the node features after the combination operation; Aggregate the node features of the K nearest nodes corresponding to each node after the combination operation to determine the updated node feature corresponding to each node.
6. The method according to claim 5, characterized in that The step of inputting the preprocessed features into the graph convolution layer, using the graph convolution layer to perform feature updates on each node in the graph structure and the K nearest nodes corresponding to each node, and determining updated node features, further includes: Determine the Euclidean distance between each node of the graph structure according to the preprocessed features; According to the Euclidean distance between each node of the graph structure, a K-nearest neighbor algorithm is used to determine the K nearest nodes corresponding to each node.
7. The method according to claim 1, characterized in that The method further comprises: Convert the three-dimensional spatial coordinates of the sound source into a vector to determine a predicted coordinate vector of the sound source; Determining an angle between the predicted coordinate vector of the sound source and the real coordinate vector of the sound source according to the predicted coordinate vector of the sound source and the real coordinate vector of the sound source; The model performance of the preset graph convolutional network is determined according to the angle between the predicted coordinate vector of the sound source and the real coordinate vector of the sound source.
8. A direction of arrival estimation system based on graph convolutional network, characterized in that: include: A data acquisition module, configured to acquire an acoustic data set using a spherical microphone array, wherein the acoustic data set includes acoustic data; A data conversion module, used for converting the acoustic data into a spherical harmonic domain and determining the spherical harmonic coefficients of the acoustic data; An information extraction module, used to perform information extraction processing on the spherical harmonic coefficients of the acoustic data to determine the feature vectors of multiple time frames of the acoustic data; A feature combination module, used for averaging the feature vectors of multiple time frames of the acoustic data and combining them by frequency to determine an input feature matrix; The sound source direction estimation module is used to input the input feature matrix into a preset graph convolutional network to determine the three-dimensional spatial coordinates of the sound source.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.
10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.