Acoustic imaging method of spiral microphone array based on multi-task deep learning network

Abstract

The invention discloses an acoustic imaging method of a spiral microphone array based on a multi-task deep learning network. The method comprises the following steps: converting an audio signal captured by a microphone array into a spectrogram through short-time Fourier transform STFT, and extracting a real part and an imaginary part of the spectrogram as feature input; the features are respectively input into a double-attention network of a double-flow structure, and local and global features are fused in a self-adaptive mode. And then the two paths of outputs are fused and are further processed through a convolution layer and a bi-directional gating cycle unit Bi-GRU layer. And respectively calculating a sound source position and sound pressure distribution through the two full connection layers. And finally, the imaging module combines the sound pressure distribution information with a camera image to generate a visual sound image. The real part and imaginary part of the spectrogram are introduced to serve as a double-flow structure of feature input, the double-attention network module is combined, real-time sound source localization and imaging are achieved under the condition of few array elements, and high spatial resolution and high anti-interference capacity are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of acoustic imaging, and in particular relates to an acoustic imaging method of a spiral microphone array based on a multi-task deep learning network. Background Art

[0002] Traditional sound source localization and imaging technologies mainly rely on beamforming algorithms. Although such methods can achieve basic sound source localization functions to a certain extent, they generally have a high dependence on the number of array elements and the shape of array distribution, resulting in low positioning accuracy and slow processing speed. Especially in complex acoustic environments with high noise, the anti-interference ability of these technologies is relatively weak. In recent years, with the rapid development of deep learning technology, the research on sound source localization and imaging using neural network models has gradually increased. Such methods can overcome the limitations of traditional algorithms to a certain extent. However, most of the current sound source localization technologies based on deep learning mainly focus on the extraction of signal position information, but are still insufficient in the visualization of sound pressure distribution, so they cannot fully present the sound field information. Summary of the invention

[0003] To solve the above problems, the present invention discloses an acoustic imaging method of a spiral microphone array based on a multi-task deep learning network, which can realize real-time sound source localization and imaging with fewer array elements and has high spatial resolution and strong anti-interference ability.

[0004] To achieve the above object, the technical solution of the present invention is as follows: The acoustic imaging method of the spiral microphone array based on the multi-task deep learning network comprises the following steps: (1) The audio signal collected by the spiral microphone array is converted into a spectrogram using short-time Fourier transform (STFT), and the real and imaginary components of the spectrogram are extracted as feature inputs. (2) The real and imaginary components are input into their respective dual attention network modules, and the local features and global dependencies are adaptively fused through the residual block, multi-spectral attention module and frame attention module; (3) The feature maps output by the dual attention network modules of the real and imaginary streams are concatenated and fused, and the deep features are further extracted through subsequent convolutional layers and bidirectional gated recurrent unit layers; (4) Extract the sound source position and sound pressure distribution information respectively through two fully connected layers; (5) Finally, the sound pressure distribution information is integrated with the camera image to generate a visualization result.

[0005] Furthermore, the audio signal STFT processing method in step (1) is as follows: First, the spectrogram is extracted from the audio signal obtained from 12 microphones using the short-time Fourier transform (STFT) of 2F points. During the extraction process, a Hamming window of length 2F is used and an overlap rate of 50% is set. The extracted imaginary part and amplitude components are used as feature inputs of the corresponding dual attention network modules in the dual-stream structure, respectively. The dimensions are expressed as T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones.

[0006] Furthermore, the dual attention network module modeling method in step (2) is as follows: The dual attention network module inputs the feature maps processed by the residual block to the multi-spectral attention module and the frame attention module respectively. The multi-spectral attention module is used to model the interaction between different frequency points and the correlation between the signals captured by different microphones; the frame attention module is used to capture the mutual dependence between different time points. Finally, the outputs of the two attention modules are fused together by element-wise summation.

[0007] In the dual attention network module, the structure of the residual block is as follows: First, the number of channels of the input feature map is reduced through the first 1x1 convolution layer; then, features are extracted through a 3x3 convolution layer; then, the number of channels of the feature map is restored through the second 1x1 convolution layer; finally, the processed feature map is fused with the original input feature map by element-wise summation to obtain the final output.

[0008] In the dual attention network module, the structure of the multi-spectral attention module is as follows: The input feature map A has a size of T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones. First, feature map A is passed through a convolutional layer to generate two new feature maps B and C, which are also of the same size of T×F×M. Feature map B is then reshaped into a two-dimensional matrix of T×(F*M), and feature map C is reshaped into a two-dimensional matrix of (F*M)×T. After that, matrix multiplication is performed on B and C and the softmax function is applied to calculate the attention matrix X, which has a size of (F*M)×(F*M). In addition, feature map A is passed through another convolutional layer to generate a new feature map D, which has a size of T×F×M, and then reshaped to T×(F*M). After that, matrix multiplication is performed on feature map D and attention matrix X, and the result is reshaped back to the shape of T×F×M. Subsequently, it is multiplied by a scale parameter β and element-wise added to the original feature map A to obtain the final output feature map E.

[0009] In the dual attention network module, the structure of the frame attention module is as follows: The input feature map A has a size of T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones. First, feature map A directly generates three new feature maps B, C, and D, and the size of these three new feature maps is also T×F×M. Then feature map B is reshaped into a two-dimensional matrix of (F*M)×T, and feature map C is reshaped into a two-dimensional matrix of T×(F*M). After that, matrix multiplication is performed on B and C and the softmax function is applied to calculate the attention matrix X, which has a size of T×T. Then, matrix multiplication is performed on the attention matrix X and the feature map D reshaped to T×(F*M), and the result is reshaped back to the shape of T×F×M. Subsequently, it is multiplied by a scale parameter β and element-by-element addition is performed on the original feature map A to obtain the final output feature map E.

[0010] Furthermore, the feature fusion and extraction method in step (3) is as follows: The dual attention network module of the real stream and the imaginary stream outputs feature maps of size T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones. The two are concatenated and fused, and the deep features are further extracted through the subsequent three convolutional layers and two bidirectional gated recurrent unit layers.

[0011] In the feature fusion and extraction method, the convolution module structure is as follows: The convolution module has three convolutional layers. Each convolutional layer contains P 3×3 filters, which work on the time-frequency-channel axis. Batch Normalization is then used to standardize the output activation values, and Rectified Linear Unit (ReLU) is used as the activation function to capture local translation-invariant features in the spectrogram. Finally, the maximum pooling operation along the frequency axis is used to reduce the dimension while keeping the time series length T unchanged. After three convolutional layers, the output dimension becomes T×2×P, where 2 represents the frequency dimension after maximum pooling and P is the number of filters.

[0012] In the feature fusion and extraction method, the bidirectional gated recurrent unit module structure is as follows: The model uses a two-layer bidirectional gated recurrent unit (Bi-GRU) design to capture complex temporal dependencies. Each layer of Bi-GRU has Q nodes, and the internal hyperbolic tangent (tanh) activation function is used to update the hidden state, and the flow of information is controlled by the gating mechanism (update gate and reset gate). After two layers of Bi-GRU processing, the output dimension becomes T×Q.

[0013] Furthermore, the method for extracting the sound source position information and the sound pressure distribution information in step (4) is as follows: This method uses two fully connected layers to extract the location information and sound pressure distribution information of the sound source. Specifically, for the extraction of the sound source location information, two fully connected layers are used, where the first layer has B nodes and the activation function is Linear, and the second layer has 3 nodes, corresponding to the position coordinates of the x, y, and z axes, respectively, and the output value is mapped to between -1 and 1 through the Tanh activation function, thereby standardizing the position coordinates. For the extraction of sound pressure distribution information, a similar two-layer structure is also used. The first layer also has B nodes and a linear activation function, while the second layer has N nodes, corresponding to the number of acoustic imaging grid points, and the ReLU activation function is used to represent the sound pressure.

[0014] Furthermore, the loss function of the network is as follows: The loss function of the sound source coordinates is: , in, Represent the real coordinates of the sound source in three-dimensional space, They respectively represent the predicted coordinate values ​​of the sound source passing through the model in three-dimensional space; The loss function of the sound pressure distribution is: , Where N is the number of acoustic imaging grid points, is the true value of the sound pressure level at the ith grid point, is the predicted value of the sound pressure level at the i-th grid point; The total loss function is , is the weight of the loss function of the sound source coordinates in the total loss function, is the weight of the loss function of sound pressure distribution in the total loss function.

[0015] Furthermore, the acoustic imaging method in step (5) is as follows: First, ensure the time synchronization of sound pressure data and image data. Then, align the sound pressure heat map with the image space according to the relative position and posture of the microphone array and the camera in the audio-visual acquisition system. Finally, overlay the sound pressure heat map on the image and adjust the transparency to ensure that the sound pressure information is clearly visible without blocking the background details, so as to achieve real-time visualization of the sound pressure distribution of the sound source. The structure of the audio-visual acquisition system is a 3-arm logarithmic spiral microphone array, and the camera is placed at the center of the microphone array; Logarithmic spiral expressed in polar coordinates: , in is the polar diameter, is the polar angle, is a constant coefficient; The length of the spiral arc between two adjacent microphones increases in equal proportion, and the ratio is ; For the maximum pole diameter The logarithmic spiral of When , the microphones on the spiral are evenly distributed; At that time, The polar coordinates of the microphone are: , , in, It is The polar diameter of the microphone in polar coordinates, is the number of microphones on a single cantilever. For a spiral array with three cantilevers, place the array panel as The arc is divided into three equal parts, and the entire array coordinates are obtained by rotating and replicating the microphone coordinate points.

[0016] The beneficial effects of the present invention are: The present invention discloses an acoustic imaging method for a spiral microphone array based on a multi-task deep learning network. By introducing a dual-stream structure with real and imaginary components of a spectrum graph as feature inputs and combining them with dual attention network modules, real-time sound source localization and imaging are achieved with fewer array elements, and the method has higher spatial resolution and stronger anti-interference capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Flow chart for realizing the method of the present invention; Figure 2 is a structural diagram of the residual block in the method of the present invention; Figure 3 is a structural diagram of the multi-spectral attention module in the method of the present invention; Figure 4 It is a structural diagram of the frame attention module in the method of the present invention; Figure 5 Schematic diagram of the acquisition system in the method of the present invention. DETAILED DESCRIPTION

[0018] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0019] like Figure 1 As shown, the acoustic imaging method of a spiral microphone array based on a multi-task deep learning network described in the present invention comprises the following steps: (1) The audio signal collected by the spiral microphone array is converted into a spectrogram using short-time Fourier transform (STFT), and the real and imaginary components of the spectrogram are extracted as feature inputs. (2) The real and imaginary components are input into their respective dual attention network modules, and the local features and global dependencies are adaptively fused through the residual block, multi-spectral attention module and frame attention module; (3) The feature maps output by the dual attention network modules of the real and imaginary streams are concatenated and fused, and the deep features are further extracted through subsequent convolutional layers and bidirectional gated recurrent unit layers; (4) Extract the sound source position and sound pressure distribution information respectively through two fully connected layers; (5) Finally, the imaging module integrates the sound pressure distribution information with the camera image to generate a visualization result.

[0020] Furthermore, the audio signal STFT processing method in step (1) is as follows: First, the spectrogram is extracted from the audio signal obtained from 12 microphones using the short-time Fourier transform (STFT) of 2F points. During the extraction process, a Hamming window of length 2F is used and an overlap rate of 50% is set. The extracted imaginary part and amplitude components are used as feature inputs of the corresponding dual attention network modules in the dual-stream structure, respectively. The dimensions are expressed as T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones.

[0021] The modeling method of the dual attention network module in step (2) is as follows: The dual attention network module inputs the feature maps processed by the residual block to the multi-spectral attention module and the frame attention module respectively. The multi-spectral attention module is used to model the interaction between different frequency points and the correlation between the signals captured by different microphones; the frame attention module is used to capture the mutual dependence between different time points. Finally, the outputs of the two attention modules are fused together by element-wise summation.

[0022] like Figure 2 As shown, in the dual attention network module, the structure of the residual block is as follows: First, the number of channels of the input feature map is reduced through the first 1x1 convolution layer; then, features are extracted through a 3x3 convolution layer; then, the number of channels of the feature map is restored through the second 1x1 convolution layer; finally, the processed feature map is fused with the original input feature map by element-wise summation to obtain the final output.

[0023] like Figure 3 As shown, in the dual attention network module, the structure of the multi-spectral attention module is as follows: The input feature map A has a size of T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones. First, feature map A is passed through a convolutional layer to generate two new feature maps B and C, which are also of the same size of T×F×M. Feature map B is then reshaped into a two-dimensional matrix of T×(F*M), and feature map C is reshaped into a two-dimensional matrix of (F*M)×T. After that, matrix multiplication is performed on B and C and the softmax function is applied to calculate the attention matrix X, which has a size of (F*M)×(F*M). In addition, feature map A is passed through another convolutional layer to generate a new feature map D, which has a size of T×F×M, and then reshaped to T×(F*M). After that, matrix multiplication is performed on feature map D and attention matrix X, and the result is reshaped back to the shape of T×F×M. Subsequently, it is multiplied by a scale parameter β and element-wise added to the original feature map A to obtain the final output feature map E.

[0024] like Figure 4 As shown, in the dual attention network module, the structure of the frame attention module is as follows: The input feature map A has a size of T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones. First, feature map A directly generates three new feature maps B, C, and D, and the size of these three new feature maps is also T×F×M. Then feature map B is reshaped into a two-dimensional matrix of (F*M)×T, and feature map C is reshaped into a two-dimensional matrix of T×(F*M). After that, matrix multiplication is performed on B and C and the softmax function is applied to calculate the attention matrix X, which has a size of T×T. Then, matrix multiplication is performed on the attention matrix X and the feature map D reshaped to T×(F*M), and the result is reshaped back to the shape of T×F×M. Subsequently, it is multiplied by a scale parameter β and element-by-element addition is performed on the original feature map A to obtain the final output feature map E.

[0025] The feature fusion and extraction method in step (3) is as follows: The dual attention network module of the real stream and the imaginary stream outputs feature maps of size T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones. The two are concatenated and fused, and the deep features are further extracted through the subsequent three convolutional layers and two bidirectional gated recurrent unit layers.

[0026] In the feature fusion and extraction method, the convolution module structure is as follows: The convolution module has three convolutional layers. Each convolutional layer contains P 3×3 filters, which work on the time-frequency-channel axis. Batch Normalization is then used to standardize the output activation values, and Rectified Linear Unit (ReLU) is used as the activation function to capture local translation-invariant features in the spectrogram. Finally, the maximum pooling operation along the frequency axis is used to reduce the dimension while keeping the time series length T unchanged. After three convolutional layers, the output dimension becomes T×2×P, where 2 represents the frequency dimension after maximum pooling and P is the number of filters.

[0027] In the feature fusion and extraction method, the bidirectional gated recurrent unit module structure is as follows: The model uses a two-layer bidirectional gated recurrent unit (Bi-GRU) design to capture complex temporal dependencies. Each layer of Bi-GRU has Q nodes, and the internal hyperbolic tangent (tanh) activation function is used to update the hidden state, and the flow of information is controlled by the gating mechanism (update gate and reset gate). After two layers of Bi-GRU processing, the output dimension becomes T×Q.

[0028] The method for extracting the sound source position information and the sound pressure distribution information in step (4) is as follows: This method uses two fully connected layers to extract the location information and sound pressure distribution information of the sound source. Specifically, for the extraction of the sound source location information, two fully connected layers are used, where the first layer has B nodes and the activation function is Linear, and the second layer has 3 nodes, corresponding to the position coordinates of the x, y, and z axes, respectively, and the output value is mapped to between -1 and 1 through the Tanh activation function, thereby standardizing the position coordinates. For the extraction of sound pressure distribution information, a similar two-layer structure is also used. The first layer also has B nodes and a linear activation function, while the second layer has N nodes, corresponding to the number of acoustic imaging grid points, and the ReLU activation function is used to represent the sound pressure.

[0029] The loss function of the network is as follows: The loss function of the sound source coordinates is: , in, Represent the real coordinates of the sound source in three-dimensional space, They respectively represent the predicted coordinate values ​​of the sound source passing through the model in three-dimensional space; The loss function of the sound pressure distribution is: , Where N is the number of acoustic imaging grid points, is the true value of the sound pressure level at the ith grid point, is the predicted value of the sound pressure level at the i-th grid point; The total loss function is , is the weight of the loss function of the sound source coordinates in the total loss function, is the weight of the loss function of sound pressure distribution in the total loss function.

[0030] The acoustic imaging method in step (5) is as follows: First, ensure the time synchronization of sound pressure data and image data. Then, align the sound pressure heat map with the image space according to the relative position and posture of the microphone array and the camera in the audio-visual acquisition system. Finally, overlay the sound pressure heat map on the image and adjust the transparency to ensure that the sound pressure information is clearly visible without blocking the background details, so as to achieve real-time visualization of the sound pressure distribution of the sound source. The structure of the audio-visual acquisition system is a 3-arm logarithmic spiral microphone array, and the camera is placed at the center of the microphone array; Logarithmic spiral expressed in polar coordinates: , in is the polar diameter, is the polar angle, is a constant coefficient; The length of the spiral arc between two adjacent microphones increases in equal proportion, and the ratio is ; For the maximum pole diameter The logarithmic spiral of When , the microphones on the spiral are evenly distributed; At that time, The polar coordinates of the microphone are: , , in, It is The polar diameter of the microphone in polar coordinates, is the number of microphones on a single cantilever. For a spiral array with three cantilevers, place the array panel as The arc is divided into three equal parts, and the entire array coordinates are obtained by rotating and replicating the microphone coordinate points.

[0031] The present invention introduces a dual-stream structure with real and imaginary components of the spectrum graph as feature inputs, and combines them with dual attention network modules respectively, to achieve real-time sound source localization and imaging with fewer array elements, with higher spatial resolution and stronger anti-interference ability.

Claims

1. An acoustic imaging method of a spiral microphone array based on a multi-task deep learning network, characterized in that: The following steps are involved: (1) The audio signal collected by the spiral microphone array is converted into a spectrogram using STFT, and the real and imaginary components of the spectrogram are extracted as feature inputs. (2) The real and imaginary components are input into their respective dual attention network modules, and the local features and global dependencies are adaptively fused through the residual block, multi-spectral attention module and frame attention module; (3) The feature maps output by the dual attention network modules of the real and imaginary streams are concatenated and fused, and the deep features are further extracted through subsequent convolutional layers and Bi-GRU layers; (4) Extract the sound source position and sound pressure distribution information respectively through two fully connected layers; (5) Finally, the sound pressure distribution information is integrated with the image captured by the camera to generate a visualization result.

2. The acoustic imaging method of a spiral microphone array based on a multi-task deep learning network according to claim 1, characterized in that: The audio signal STFT processing method in step (1) is as follows: Firstly, the 2F-point STFT is used to extract the spectrogram from the audio signal obtained from 12 microphones. During the extraction process, a Hamming window with a length of 2F is adopted, and an overlap rate of 50% is set. The extracted imaginary part and amplitude components are used as the feature inputs of their respective dual attention network modules, and their dimensions are expressed as T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones.

3. The acoustic imaging method of a spiral microphone array based on a multi-task deep learning network according to claim 1, characterized in that: The modeling method of the dual attention network module in step (2) is as follows: The dual attention network module inputs the feature maps processed by the residual block to the multi-spectral attention module and the frame attention module respectively; the multi-spectral attention module is used to model the interaction between different frequency points and the correlation between the signals captured by different microphones; The frame attention module is used to capture the interdependencies between different time points; finally, the outputs of the two attention modules are fused together by element-wise summation.

4. The acoustic imaging method of the spiral microphone array based on the multi-task deep learning network according to claim 3, It is characterized in that The structure of the residual block is as follows: First, the number of channels of the input feature map is reduced through the first 1x1 convolution layer; then, features are extracted through a 3x3 convolution layer; then, the number of channels of the feature map is restored through the second 1x1 convolution layer; finally, the processed feature map is fused with the original input feature map by element-wise summation to obtain the final output.

5. The acoustic imaging method of the spiral microphone array based on the multi-task deep learning network according to claim 3, It is characterized in that The structure of the multi-spectral attention module is as follows: The input feature map A has a size of T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones. First, feature map A generates two new feature maps B and C through a convolutional layer, and the size of these two new feature maps is also T×F×M. Then feature map B is reshaped into a two-dimensional matrix of T×(F*M), and feature map C is reshaped into a two-dimensional matrix of (F*M)×T. After that, matrix multiplication is performed on B and C and the softmax function is applied to calculate the attention matrix X, which has a size of (F*M)×(F*M). In addition, feature map A generates a new feature map D through another convolutional layer, which has a size of T×F×M and is then reshaped to T×(F*M). After that, matrix multiplication is performed on feature map D and attention matrix X, and the result is reshaped back to the shape of T×F×M. Subsequently, it is multiplied by a scale parameter β and element-wise added to the original feature map A to obtain the final output feature map E.

6. The acoustic imaging method of a spiral microphone array based on a multi-task deep learning network according to claim 3, It is characterized in that The structure of the frame attention module is as follows: The input feature map A has a size of T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones. First, the feature map A directly generates three new feature maps B, C, and D, and the sizes of these three new feature maps are also T×F×M. Then the feature map B is reshaped into a two-dimensional matrix of (F*M)×T, and the feature map C is reshaped into a two-dimensional matrix of T×(F*M). After that, matrix multiplication is performed on B and C and the softmax function is applied to calculate the attention matrix X, which has a size of T×T. After that, the attention matrix X is matrix multiplied with the feature map D reshaped to T×(F*M), and the result is reshaped back to the shape of T×F×M. Subsequently, it is multiplied by a scale parameter β and element-by-element addition is performed on the original feature map A to obtain the final output feature map E.

7. The acoustic imaging method of a spiral microphone array based on a multi-task deep learning network according to claim 1, characterized in that: The feature fusion and extraction method in step (3) is as follows: The dual attention network module of the real stream and the imaginary stream outputs feature maps of size T×F×M, where T represents the number of time frames, F represents the number of positive frequency components, and M represents the number of microphones. The two are concatenated and fused, and the subsequent convolution module and bidirectional gated recurrent unit module are used to further extract deep features. The convolution module has three convolutional layers; each convolutional layer contains P 3×3 filters, which work on the time-frequency-channel axis; batch normalization technology is then used to standardize the output activation value, and the rectified linear unit ReLU is used as the activation function to capture the local translation invariant features in the spectrogram; finally, the maximum pooling operation along the frequency axis is used to reduce the dimension while keeping the time series length T unchanged; after three convolutional layers, the output dimension becomes T×2×P, where 2 represents the frequency dimension after maximum pooling, P is the number of filters, The bidirectional gated recurrent unit module adopts a two-layer Bi-GRU design to capture complex temporal dependencies. There are Q nodes in each layer of Bi-GRU, which uses a hyperbolic tangent activation function to update the hidden state and controls the flow of information through a gating mechanism. After two layers of Bi-GRU processing, the output dimension becomes T×Q.

8. The acoustic imaging method of a spiral microphone array based on a multi-task deep learning network according to claim 1, characterized in that: The method for extracting the sound source position information and the sound pressure distribution information in step (4) is as follows: This method extracts the location information and sound pressure distribution information of the sound source through two fully connected layers respectively; specifically, for the extraction of the sound source location information, two fully connected layers are used, where the first layer has B nodes and the activation function is Linear, and the second layer has 3 nodes, corresponding to the position coordinates of the x, y, and z axes respectively, and the output value is mapped to between -1 and 1 through the Tanh activation function, thereby standardizing the position coordinates; for the extraction of sound pressure distribution information, a similar two-layer structure is also used, the first layer also has B nodes and a linear activation function, and the second layer has N nodes, corresponding to the number of acoustic imaging grid points, and the ReLU activation function is used to represent the sound pressure.

9. The acoustic imaging method of a spiral microphone array based on a multi-task deep learning network according to claim 1, characterized in that: The loss function of the network is as follows: The loss function of the sound source coordinates is: ， in, Represent the real coordinates of the sound source in three-dimensional space, They respectively represent the predicted coordinate values ​​of the sound source passing through the model in three-dimensional space; The loss function of the sound pressure distribution is: ， Where N is the number of acoustic imaging grid points, is the true value of the sound pressure level at the ith grid point, is the predicted value of the sound pressure level at the i-th grid point; The total loss function is ， is the weight of the loss function of the sound source coordinates in the total loss function, is the weight of the loss function of sound pressure distribution in the total loss function.

10. The acoustic imaging method of a spiral microphone array based on a multi-task deep learning network according to claim 1, characterized in that: The acoustic imaging method in step (5) is as follows: First, ensure the time synchronization of sound pressure data and image data. Then, align the sound pressure heat map with the image space according to the relative position and posture of the microphone array and the camera in the audio-visual acquisition system. Finally, overlay the sound pressure heat map on the image and adjust the transparency to ensure that the sound pressure information is clearly visible without blocking the background details, so as to achieve real-time visualization of the sound pressure distribution of the sound source. The structure of the audio-visual acquisition system is a 3-arm logarithmic spiral microphone array, and the camera is placed at the center of the microphone array; Logarithmic spiral expressed in polar coordinates: ， in is the polar diameter, is the polar angle, is a constant coefficient; The length of the spiral arc between two adjacent microphones increases in equal proportion, and the ratio is ; For the maximum pole diameter The logarithmic spiral of When , the microphones on the spiral are evenly distributed; At that time, The polar coordinates of the microphone are: ， ， in, It is The polar diameter of the microphone in polar coordinates, is the number of microphones on a single cantilever. For a spiral array with three cantilevers, place the array panel as The arc is divided into three equal parts, and the entire array coordinates are obtained by rotating and replicating the microphone coordinate points.

Images