Sound source identification method based on array expansion and SE mobile reverse bottleneck convolutional neural network

Abstract

The invention relates to a sound source identification method based on array expansion and an SE mobile reverse bottleneck convolutional neural network, and the method comprises the steps: obtaining an 18-array sound source distribution diagram employing a sound pressure cross-spectrum matrix to calculate a MUSIC algorithm, inputting the 18-array sound source distribution diagram into an EAG-U-Net data conversion model, converting the 18-array sound source distribution diagram into a 64-array sound source distribution diagram, and carrying out the recognition of the 64-array sound source distribution diagram. According to the model, an EAG mechanism is introduced to optimize model feature selection in the spatial dimension and the channel dimension, the representation capability and accuracy of the model in the data conversion process are improved, and an acoustic imaging graph after data conversion is generated; and inputting the 64 array sound source distribution diagram into a moving reverse bottleneck convolutional neural network model based on an SE attention mechanism to perform feature extraction and image reconstruction, and obtaining a predicted sound source distribution diagram. Through predicting a sound source distribution diagram, local maximum detection is carried out, and sound source localization and intensity are obtained.

Description

Technical Field

[0001] The invention relates to the field of sound source localization and deep learning technology, and proposes a sound source recognition method based on array expansion and SE moving reverse bottleneck convolutional neural network. Background Art

[0002] Sound source localization technology has important applications in mechanical fault diagnosis, environmental noise control, automotive NVH development and other fields. In industrial environments, the sound of equipment failure is often masked by other noises, making it difficult to detect problems in a timely manner. Sound source localization can accurately identify weak fault signals and effectively locate the source of noise, providing support for noise control and equipment maintenance. With the development of microphone arrays and beamforming technology, the accuracy and practicality of sound source localization continue to improve. At the same time, advances in artificial intelligence and deep learning have made sound signal-based positioning and recognition technology show broad prospects in speech recognition, intelligent driving, security monitoring and other fields. However, traditional methods still face challenges such as accuracy, robustness and real-time performance, and new technological innovations are urgently needed to solve these problems.

[0003] Sound source localization technology has made significant progress in recent years, especially in array-based direction estimation methods. The MUSIC (Multiple Signal Classification) algorithm is widely used due to its high-precision positioning capability. The algorithm obtains the spatial information of the sound source by calculating the sound pressure cross-spectral matrix using array sensors. However, in complex environments with high noise or a limited number of arrays, the accuracy and robustness of the MUSIC algorithm are affected. In the MUSIC algorithm, the size of the sound source distribution map usually depends on the number of microphones in the array, and using more microphone arrays to train deep learning models may lead to a decrease in the positioning accuracy of arrays with fewer microphones. Traditional methods improve accuracy by increasing the number of array sensors, but this faces challenges such as hardware cost, array layout complexity, and computational burden. Therefore, how to expand the array reception range through algorithms without increasing the physical array has become an important research direction in the field of sound source localization.

[0004] With the rapid development of deep learning technology, its application in signal processing has gradually shown great potential. In terms of sound source recognition, deep learning-based methods are mainly divided into two categories: grid-based and gridless. The grid-based method divides the sound source plane into multiple grids and identifies the sound source in each grid. For example, Ma et al. applied convolutional neural network (CNN) to sound pressure distribution map, combining deep learning and beamforming algorithm for sound source recognition for the first time; Xu et al. used densely connected convolutional neural network (DenseNet) to generate sound pressure distribution map from cross-spectral matrix (CSM), realizing high-resolution sound source recognition, and can identify up to 25 sound sources at a single frequency. In order to further improve the accuracy, Lee et al. proposed a new objective function that depends on the relative position relationship between grid points and sound sources, and combined with the encoder-decoder structure of the fully convolutional neural network (FCN), to achieve high-resolution and multi-sound source recognition. The combination of these methods not only improves the accuracy of sound source localization, but also enhances the robustness of the system in complex environments. The deep convolutional neural network based on U-Net structure further improves the accuracy of sound source localization. In addition, the combination of self-attention and gated attention has been applied to image enhancement and signal processing tasks, significantly improving performance.

[0005] The size of the sound source distribution map used as the input of deep learning is determined by the number of grid points. When the number of microphones decreases, the sound source features in the sound source distribution map are no longer obvious. Therefore, when the data generated by more microphone arrays is used to train the deep learning model to predict the sound source distribution map of fewer microphone arrays, the positioning accuracy may decrease. In order to overcome this problem and enhance the versatility of the deep learning sound source identification method, the present invention proposes a sound source identification method based on array expansion and SE mobile reverse bottleneck convolutional neural network. Among them, the sound source distribution map predicted by the 18-array MUSIC algorithm is converted into the sound source distribution map predicted by the 64-array MUSIC algorithm through the EAG-U-Net data conversion model, thereby solving the problem that the MUSIC algorithm is sensitive to the array geometry configuration and the number of elements. The SE-MBCNet network combines the SE attention mechanism with the convolutional neural network, which can more accurately extract the key information in the image, especially when processing complex sound source distribution maps, it can significantly improve the accuracy of sound source positioning. Summary of the invention

[0006] The present invention proposes a sound source recognition method based on array expansion and SE mobile reverse bottleneck convolutional neural network. As a new type of data conversion network, EAG-U-Net inherits the U-Net network (Efficient Attention Gate U-Net, EAG-U-Net's basic framework adopts an encoder-decoder structure, effectively extracts and restores the spatial features of the image through downsampling and upsampling operations, and the unique jump connection mechanism transfers the feature map of the encoder part to the decoder layer, thereby improving the detail retention capability and sound source localization accuracy. The EAG-U-Net data conversion model converts the sound source distribution map predicted by the 18-array MUSIC algorithm into the sound source distribution map predicted by the 64-array MUSIC algorithm to compensate for the dependence of the MUSIC algorithm on the array structure, microphone distribution, and the number of microphones to achieve array expansion. The SE-MBCNet network performs image processing on the acoustic image generated by the MUSIC algorithm, improves the robustness of the MUSIC algorithm in noise interference and complex environments, and improves the effectiveness and performance of sound source localization.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A sound source recognition method based on array expansion and SE moving reverse bottleneck convolutional neural network, the method comprising:

[0009] Step 1: Obtain a sound pressure cross-spectrum matrix, and use the sound pressure cross-spectrum matrix to calculate an 18-array sound source distribution diagram and a 64-array sound source distribution diagram of a MUSIC algorithm;

[0010] Step 2: Input the 18-array sound source distribution map into the EAG-U-Net data conversion model to obtain the converted 64-array sound source distribution map; the EAG-U-Net data conversion model is trained with the 18-array sound source distribution map as input and the 64-array sound source distribution map as label;

[0011] Step 3: Input the converted 64-array sound source distribution map into the SE-MBCNet network model to obtain a predicted sound source distribution map; the SE-MBCNet network model is obtained by taking the 64-array sound source distribution map after data conversion as input and the target map as label training;

[0012] Step 4: Perform local maximum detection on the predicted sound source distribution map to obtain the sound source location and intensity.

[0013] Optionally, obtaining the 18-array sound source distribution map and the 64-array sound source distribution map includes:

[0014] Step 1.1: grid the sound source plane, discretize the sound source calculation plane, form a grid surface, i.e., a focus plane, and use each grid point of the focus plane as a potential sound source point;

[0015] Step 1.2: Arrange a plurality of microphone sensors in the sound field formed by the radiation of a plurality of sound sources to form a planar microphone array, i.e., a measurement plane, and make the measurement plane parallel to the sound source plane;

[0016] Step 1.3: receiving frequency sound pressure signal data collected by the sensor array, wherein the sound pressure signal data includes: a sound source signal and a noise signal;

[0017] Step 1.4: pre-process the frequency sound pressure signal data, calculate the sound pressure cross-spectrum matrix, and obtain the 18-array sound source distribution map and the 64-array sound source distribution map according to the MUSIC algorithm.

[0018] Optionally, obtaining the converted 64-array sound source distribution diagram includes:

[0019] Step 2.1: Input the 18-array sound source distribution map into the initial convolution layer of the EAG-U-Net data conversion model, and the initial convolution layer uses two 3×3 convolutions to continuously perform preliminary processing on the input image to extract the basic features of the 18-array sound source distribution map;

[0020] Step 2.2: Input the basic features into the encoder of the EAG-U-Net data conversion model. The encoder consists of three double convolutional layers and a pooling layer. By continuously extracting image features and downsampling, the first high-level features are gradually extracted; the double convolutional layer contains two 3×3 convolutions, a ReLU activation function, and a batch normalization layer. The pooling layer uses maximum pooling for downsampling, reduces the spatial dimension, and increases the receptive field, thereby reducing the amount of calculation and extracting more abstract features, namely, the first high-level features;

[0021] Step 2.3: After the pooling operation in the third encoder block, an ECA attention mechanism is added. The ECA attention mechanism obtains the average value of each channel by performing global average pooling to capture global information. Based on the global information, a one-dimensional convolution operation is performed on the pooled channel information to capture the dependency between channels. Based on the dependency, a weight coefficient is obtained by a Sigmoid activation function, and a weighted operation is performed on each channel using the weight coefficient.

[0022] Step 2.4: Through the AG gated attention mechanism, the encoder feature map Fe and the decoder feature map Fd of the corresponding level are spliced ​​to generate a first fused feature map, and the first fused feature map is subjected to 1×1 convolution to extract higher-level feature information, reduce the number of channels and generate input features of the attention map, and the attention weight map A is generated using the Sigmoid activation function. The attention weight map A is multiplied element by element with the encoder feature map Fe to apply the attention weight to the feature map of the encoder, and the weighting of the feature map is dynamically adjusted. The weighted feature map is fused with the feature map in the decoder to obtain the second fused feature map;

[0023] When restoring the spatial resolution, the decoder will be spliced ​​with the feature map of the encoder to form a first fused feature map, which includes the underlying features of the encoder and the high-level semantic information of the decoder;

[0024] Step 2.5: The output of the third encoder block is upsampled by transposed convolution, and the obtained result and the output of the second encoder block are input into the AG gated attention mechanism for processing, and the obtained feature map is concatenated with the input upsampled result to obtain the output result of the third layer decoder block;

[0025] The output result of the third layer decoder block is upsampled by transposed convolution, and the obtained result and the output of the first encoder block are input into the AG gated attention mechanism for processing, and the obtained feature map is concatenated with the input upsampled result to obtain the output result of the second layer decoder block;

[0026] The output of the second layer decoder block is upsampled by transposed convolution, and the obtained result and the output of the initial convolutional layer are input into the AG gated attention mechanism for processing. The obtained feature map is concatenated with the input upsampled result to obtain the output of the first layer decoder block;

[0027] The output result of the first layer decoder block is used to restore the image details using a double convolutional layer to obtain a 64-array sound source distribution map after data conversion.

[0028] Optionally, obtaining the predicted sound source distribution map includes:

[0029] Step 3.1: Input the 64-array sound source distribution map after data conversion into the double-layer convolution module of the SE-MBCNet network model, and the double-layer convolution module is used to perform preliminary feature extraction on the 64-array sound source distribution map after data conversion; the double-layer convolution module includes two consecutive 3×3 convolution operations, a batch normalization layer BN and an activation function ReLU;

[0030] Step 3.2: Input the extracted preliminary features into the encoder module of the SE-MBCNet network model, and the encoder module is used to perform multi-level semantic information and feature encoding on the feature map after the preliminary feature extraction; the encoder module is composed of four MBConv layers of SE attention mechanism and a maximum pooling layer, and the second high-level features are gradually extracted by continuously extracting image features and downsampling; the MBConv layer of the SE attention mechanism includes: depthwise separable convolution, SE attention mechanism and residual connection;

[0031] The workflow of depthwise separable convolution, SE attention mechanism and residual includes:

[0032] First, the dimension is increased by 1×1 convolution, batch normalization BN and Swish activation function to expand the feature dimension and increase the number of channels. Then, a depth-separable convolution is performed, in which 3×3 depth convolution is combined with batch normalization BN and Swish activation function to process each channel independently to extract local features. Subsequently, the SE channel attention mechanism is introduced, and the global average pooling layer is first used for compression to effectively capture global features and reduce computational complexity. Then, in the excitation stage, weights are adaptively generated for each channel through the fully connected layer and Sigmoid activation function to enhance important features. In the scaling stage, the features of each channel are re-weighted by multiplying the obtained channel weights by each channel of the original feature map element by element. Finally, a 1×1 convolution is used to reduce the weighted feature map to the number of output channels, and a residual connection is added to maintain network stability and accelerate training.

[0033] The expression formula of the MBConv layer of the SE attention mechanism is:

[0034] X exp＝ Swish(BN(Conv1(X)))

[0035] X dw =Swish(BN(DwConv(X exp )))

[0036] X out =X+Conv1(SE(X dw ))

[0037] Among them, Swish represents the self-gated activation function, BN represents the batch normalization layer, Conv1 represents 1×1 convolution, DwConv represents 3×3 convolution for each channel independently, SE represents SE channel attention mechanism, X exp Indicates the result of the dimension-raising operation, X out MBConv module results representing the SE attention mechanism;

[0038] Step 3.3: Input the feature map processed by the encoder module of the SE-MBCNet network model into the decoder to restore the high-resolution feature map; the decoder is divided into four stages, and the decoder block of each stage directly concatenates the feature map with the corresponding layer of the encoder through a jump connection, merges the downsampled feature map with the upsampled feature map, and enhances the features of the restored image; the encoder block is composed of the MBConv layer of the SE attention mechanism and the upsampling layer, which retains the detail information, thereby avoiding the loss of important features during the upsampling process;

[0039] Step 3.4: Through the 1×1 convolutional layer at the end of the network, the final predicted sound source distribution map with sound source position coordinates and intensity information is obtained.

[0040] Optionally, obtaining the sound source positioning and intensity includes:

[0041] Step 4.1: Use the local maximum detection algorithm to identify the local maximum in each row of the data matrix of the predicted sound source distribution map and find the first three maximum values;

[0042] Step 4.2: Verify whether the maximum value is a true local maximum value through the neighborhood, that is, by comparing the target value with the value in its neighborhood, determine whether the maximum value is the largest in the local range. If the maximum value is the largest in the local range, sort the detected local maximum values;

[0043] Step 4.3: Based on the sorting results, use the coordinate transformation technology to map the maximum row and column indexes to the actual coordinates to calculate the source position and its intensity.

[0044] The beneficial effects of the present invention are:

[0045] The present invention proposes a sound source localization method that combines innovative data conversion with a deep learning model. By introducing an innovative data conversion preprocessing method, the sound source localization process is effectively optimized before model training. Specifically, the EAG-U-Net data conversion is used to convert the sound source distribution map generated by the 18-array MUSIC algorithm into the prediction result of the 64-array MUSIC algorithm, which makes up for the sensitivity of the MUSIC algorithm to changes in array geometry and the number of array elements. This method significantly reduces the number of microphone arrays required, allowing low-density arrays to simulate the accuracy of high-density arrays. In addition, the method performs well in sidelobe elimination and mainlobe width reduction, improving the clarity and accuracy of the sound source distribution map.

[0046] The present invention combines deep learning technology and optimizes the sound source distribution map predicted by the MUSIC algorithm through the SE-MBCNet network, effectively improving the resolution of the acoustic image and the accuracy of sound source localization. In the traditional MUSIC algorithm, sound source localization is affected by noise. By introducing the deep learning SE-MBCNet network model, the present invention significantly enhances the accuracy of the imaging results and maintains high-resolution sound source localization capabilities.

[0047] The present invention can accurately predict the location and intensity of the sound source by training a deep learning network. Traditional methods rely on a large number of microphone arrays to achieve high-precision positioning, while the present invention can extract sound source information through limited microphone data, significantly improving the accuracy and robustness of sound source positioning and enhancing the application capability in complex acoustic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 This is a flow chart of a sound source recognition method based on array expansion and SE moving reverse bottleneck convolutional neural network according to an embodiment of the present invention;

[0050] Figure 2 Schematic diagram of the specific network structure of the converter EAG-U-Net of an embodiment of the present invention; wherein (a) is the structure diagram of the converter EAG-U-Net, (b) is the structure diagram of the ECA, and (c) is the structure diagram of the AG;

[0051] Figure 3 Schematic diagram of the specific structure of the SE-MBCNet network of an embodiment of the present invention; wherein (a) is a SE-MBCNet structure diagram, (b) is an MBConv structure diagram, and (c) is a SE attention mechanism structure diagram;

[0052] Figure 4 Schematic diagram of a sound source localization measurement model according to an embodiment of the present invention; wherein (a) is a schematic diagram of a sound source localization measurement model, and (b) is an 18-spiral channel microphone array and a 64-spiral channel microphone array;

[0053] Figure 5The single sound source positioning result diagram of the embodiment of the present invention; (a) is a positioning result diagram when the single sound source is at [-0.31, 0.27, 0], SNR = 20dB, f = 1000Hz, z = 1m, (b) is a positioning result diagram when the single sound source is at [0.17, 0.53, 0], SNR = 20dB, f = 2000Hz, z = 1m, (c) is a positioning result diagram when the single sound source is at [-0.57, -0.15, 0], SNR = 20dB, f = 6000Hz, z = 1m;

[0054] Figure 6 Figure 1 is a diagram showing the positioning result of dual sound source positions according to an embodiment of the present invention; (a) is a diagram showing the positioning result when the dual sound source positions are [-0.21, -0.19, 0], [0.59, -0.19, 0], SNR = 20 dB, f = 1000 Hz, and z = 1 m; (b) is a diagram showing the positioning result when the dual sound source positions are [0.05, -0.27, 0], [-0.25, -0.39, 0], SNR = 20 dB, f = 2000 Hz, and z = 1 m; (c) is a diagram showing the positioning result when the dual sound source positions are [-0.59, -0.13, 0], [0.47, -0.27, 0], SNR = 20 dB, f = 6000 Hz, and z = 1 m;

[0055] Figure 7 The three sound source positioning result diagrams of the embodiment of the present invention; (a) is the positioning result diagram when the three sound source positions are [-0.07, 0.53, 0], [0.35, 0.11, 0], [-0.05, -0.41, 0], SNR = 20dB, f = 1000Hz, z = 1m, (b) is the positioning result diagram when the three sound source positions are [-0.15, 0.19, 0], [-0.39, (a) is the positioning result diagram at the three sound source positions [0.53, 0.37, 0], [0.11, 0.17, 0], [0.45, -0.15, 0], SNR = 20dB, f = 6000Hz, z = 1m. (b) is the positioning result diagram at the three sound source positions [0.53, 0.37, 0], [0.11, 0.17, 0], [0.45, -0.15, 0], SNR = 20dB, f = 6000Hz, z = 1m. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] like Figure 1 As shown, this embodiment discloses a sound source identification method based on array expansion and SE mobile reverse bottleneck convolutional neural network, and the method includes: Step 1: Obtain the sound pressure cross-spectral matrix, and calculate the 18-array sound source distribution map and the 64-array sound source distribution map of the MUSIC algorithm. Step 2: Input the 18-array sound source distribution map into the EAG-U-Net data conversion model to obtain the converted 64-array sound source distribution map. The EAG-U-Net data conversion model is obtained by training with the 18-array sound source distribution map as input and the 64-array sound source distribution map as label. Step 3: Input the 64-array sound source distribution map after the data conversion into the SE-MBCNet network model to obtain the predicted sound source distribution map. The SE-MBCNet network model is obtained by training with the 64-array sound source distribution map after the data conversion as input and the target map as label. Step 4: Perform local maximum detection on the predicted result map to obtain the sound source positioning and intensity.

[0059] Specifically:

[0060] Step 1: Obtain the sound pressure cross-spectrum matrix and calculate the 18-array sound source distribution map and 64-array sound source distribution map of the MUSIC algorithm.

[0061] The step 1 is specifically as follows:

[0062] Step 1.1: Arrange M microphone sensors in the sound field formed by the radiation of K sound sources. The M sensors form a planar microphone array, called the measurement plane. The measurement plane is parallel to the sound source plane, and the distance between the two planes is z. Figure 4 (a)-(b).

[0063] Step 1.2: Grid the sound source plane, and the discrete sound source calculation plane forms a grid surface, called the focus plane. The focus plane contains N grid points, each of which is also called a focus point, and each focus point is a potential sound source point.

[0064] Step 1.3: Receive the frequency sound pressure signal data X collected by the sensor array, including the sound source signal and the noise signal, and then calculate its sound pressure cross-spectrum matrix C.

[0065]

[0066] Where X=[x1,x2,...,x M ] T . H represents the conjugate transpose of the matrix.

[0067] Step 1.4: Perform eigenvalue decomposition on the cross-spectral matrix C. Select the eigenvectors corresponding to the largest K eigenvalues ​​as the signal subspace E, and the other eigenvectors correspond to the noise subspace.

[0068] C=UAU H

[0069] in, is the eigenvector matrix. A=diag(λ1,λ2,...,λ M ) is the diagonal eigenvalue matrix, λ1≥λ2≥...≥λ M .

[0070]

[0071] Step 1.4: Using the core steps of the MUSIC algorithm, calculate the spatial spectrum P based on the sound pressure cross-spectrum matrix C. Use it as the input for subsequent algorithm analysis and processing, the formula is as follows:

[0072]

[0073] Where g is the steering vector, ||r n -r m || represents the distance from the mth microphone to the nth grid point, c represents the speed of sound, and f represents the frequency.

[0074] Step 1.5: Perform the normalization operation on the calculated spatial spectrum P to perform acoustic imaging and obtain a sound source distribution map.

[0075] Step 1.6: A target map is designed as the ground truth map for acoustic imaging, which is calculated based on the location, intensity, and frequency of the sound source:

[0076]

[0077] Among them, q k represents the intensity of the kth sound source, ||r n -r k || represents the distance between the kth sound source and the nth grid point in the target graph, and η and β are two adjustable hyperparameters. η is a very small constant used to avoid the denominator being 0, and β is used to control the decay speed of the main lobe in the target graph.

[0078] Step 2: Input the 18-array sound source distribution map into the EAG-U-Net data conversion model, and convert the data to obtain the 64-array sound source distribution map. The structure of the EAG-U-Net data conversion model is as follows: Figure 2 (a)-(c).

[0079] The step 2 is specifically as follows:

[0080] Step 2.1: Input the 18-array sound source distribution map into the initial convolution layer of the EAG-U-Net data conversion model, and the initial convolution layer uses two 3×3 convolutions to continuously perform preliminary processing on the input image to extract the basic features of the 18-array sound source distribution map;

[0081] Step 2.2: Step 2.2: Input the basic features into the encoder of the EAG-U-Net data conversion model. The encoder is processed layer by layer through three encoder blocks. Each encoder block consists of two 3×3 convolutional layers, ReLU activation function, batch normalization layer and maximum pooling layer. The convolution operation extracts low-level and mid-level features of the image, increases nonlinear capabilities through the ReLU activation function, and the batch normalization layer accelerates training and improves stability. The maximum pooling operation reduces the spatial resolution through downsampling, increases the receptive field, and extracts more abstract features. After these processes, the spatial size of the image is gradually compressed, but its high-level semantic information is effectively retained.

[0082] Step 2.3: In the third encoder block, after the same convolution, activation, batch normalization and pooling operations as the first two blocks, the ECA attention mechanism is added. First, global average pooling is performed to obtain the average value of each channel to capture global information, and then a one-dimensional convolution operation is performed on the pooled channel information to capture the dependency between channels. Based on the dependency, a weight coefficient is obtained through the Sigmoid activation function, and the weight coefficient is used to perform a weighted operation on each channel to further enhance the network's attention to important features.

[0083] The calculation formula of ECA attention mechanism is:

[0084]

[0085] Among them, Pool(·) represents global average pooling. Conv1(·) represents a one-dimensional convolution operation. σ is the Sigmoid activation function, which is used to generate the weight coefficient of each channel.

[0086] The ECA attention mechanism is used to further enhance the channel-level feature representation, adjust the channel weights, and obtain the key features of the image before the upsampling process of the decoder.

[0087] Step 2.4: Step 2.4: Through the AG gated attention mechanism, the encoder feature map Fe and the decoder feature map Fd of the corresponding level are spliced ​​to generate a first fused feature map, and the first fused feature map is applied with 1×1 convolution to extract higher-level feature information, reduce the number of channels and generate input features of the attention map, and use the Sigmoid activation function to generate the attention weight map A, and the attention weight map A is multiplied element by element with the encoder feature map Fe to realize the application of attention weight to the feature map of the encoder, and the weighting of the feature map is dynamically adjusted. The feature map after weighted processing is fused with the feature map in the decoder to obtain the second fused feature map.

[0088] When restoring the spatial resolution, the decoder first concatenates the feature map of the encoder to form the first fused feature map, which contains the underlying features of the encoder and the high-level semantic information of the decoder.

[0089] The calculation formula of the Attention Gate (AG) module is:

[0090] A=σ(Conv[F e ,F d ])

[0091] F AG =A⊙F d

[0092] Where σ is the sigmoid activation function. Conv(·) represents a 1×1 convolution. [·] represents a concatenation operation. ⊙ is an element-wise product operation.

[0093] The AttentionGate module improves the fusion process of feature maps and dynamically adjusts the weights of each area in the input feature map, so that important areas receive more attention and secondary areas are suppressed, thereby improving the efficiency and accuracy of the model in processing image tasks.

[0094] Step 2.5: The output of the third encoder block is upsampled through transposed convolution, and the result is sent to the AG gated attention mechanism together with the output of the second encoder block for processing. The obtained feature map is concatenated with the input upsampled result to obtain the output of the third layer decoder block;

[0095] The output of the third-layer decoder block is upsampled through transposed convolution, and the result is sent to the AG gated attention mechanism together with the output of the first encoder block for processing. The obtained feature map is concatenated with the input upsampled result to obtain the output of the second-layer decoder block;

[0096] The output of the second-layer decoder block is upsampled through transposed convolution, and the result is sent to the AG gated attention mechanism together with the output of the initial convolutional layer for processing. The obtained feature map is concatenated with the input upsampled result to obtain the output of the first-layer decoder block; the specific process of the AG gated attention mechanism refers to process 2.4.

[0097] Finally, the output of the first layer decoder block is used to restore the image details using a double convolutional layer to obtain a 64-array sound source distribution map after data conversion.

[0098] Step 3: Input the 64-array sound source distribution map after data conversion into the SE-MBCNet network model to obtain the predicted sound source distribution map. The SE-MBCNet network model is obtained by training with the 64-array sound source distribution map after data conversion as input and the target map as label. The structure of the SE-MBCNet network model is as follows: Figure 3 (a)-(c).

[0099] The step 3 is specifically as follows:

[0100] Step 3.1: Input the 64-array sound source distribution map after data conversion into the double-layer convolution module of the SE-MBCNet network model for preliminary feature extraction of the 64-array sound source distribution map after data conversion. The double-layer convolution module includes two consecutive 3×3 convolution operations, each followed by a batch normalization layer (BN) and an activation function (ReLU), which aims to capture the low-level features of the input image through local feature extraction and nonlinear transformation, and gradually learn more complex features.

[0101] Step 3.2: The extracted preliminary features are input into the encoder module of the SE-MBCNet network model for multi-level semantic information and feature encoding of the input feature map. The encoder module is composed of four MBConv layers with SE attention mechanism and a maximum pooling layer. It extracts high-level features step by step by continuously extracting image features and downsampling. The MBConv layer with SE attention mechanism includes depthwise separable convolution, SE attention mechanism and residual connection, which is used to enhance the feature expression ability of the network, reduce the amount of computation and improve computational efficiency. The maximum pooling layer performs downsampling, reduces the spatial dimension and increases the receptive field, thereby reducing the amount of computation and extracting more abstract features.

[0102] Step 3.3: The MBConv module of the SE attention mechanism of the SE-MBCNet network model consists of depthwise separable convolution, SE attention mechanism and residual connection. First, the dimension is increased by 1×1 convolution, batch normalization (BN) and Swish activation function to expand the feature dimension and increase the number of channels. Then, a depthwise separable convolution is performed, in which 3×3 depthwise convolution (Depthwise Conv) is used with batch normalization (BN) and Swish activation function to process each channel independently to extract local features. Subsequently, the SE channel attention mechanism is introduced to generate channel weights through point-by-point convolution to weight the feature map. Finally, a 1×1 convolution is used to reduce the weighted feature map to the number of output channels, and a residual connection is added to maintain network stability and accelerate training. The expression formula of the MBConv module of the SE attention mechanism:

[0103] X exp＝ Swish(BN(Conv1(X)))

[0104] X dw =Swish(BN(DwConv(X exp )))

[0105] X out =X+Conv1(SE(X dw ))

[0106] Where Swish is the self-gated activation function. BN represents the batch normalization layer. Conv1 represents a 1×1 convolution. DwConv represents a 3×3 convolution for each channel independently. SE represents the SE channel attention mechanism. X exp Indicates the result of the dimension-raising operation. out MBConv module results showing the SE attention mechanism.

[0107] The SE channel attention mechanism module includes three stages: compression, excitation, and scaling. First, the global average pooling layer is used for compression to effectively capture global features and reduce computational complexity. Then, the excitation stage adaptively generates weights for each channel through the fully connected layer and the Sigmoid activation function to enhance important features. Finally, the scaling stage reweights the features of each channel by multiplying the obtained channel weights element by element with each channel of the original feature map. This process enhances the feature expression of important channels and suppresses the feature expression of unimportant channels.

[0108] Step 3.4: Input the feature map processed by the encoder module of the SE-MBCNet network model into the decoder to restore the high-resolution feature map. The decoder is also divided into four stages. The decoder block of each stage directly concatenates the feature map with the corresponding layer of the encoder through skip connections (SkipConnections), merges the downsampled feature map with the upsampled feature map, and enhances the features of the restored image. The encoder block is composed of the MBConv layer of the SE attention mechanism and the upsampling layer, which helps to retain the detail information and avoid losing important features during the upsampling process.

[0109] Step 3.5: Finally, the 1×1 convolutional layer at the end of the network is used to obtain the final predicted sound source distribution map with the sound source position coordinates and intensity information.

[0110] Step 4: Perform local maximum detection on the prediction result graph to obtain the sound source location and intensity.

[0111] The step 4 is specifically as follows:

[0112] Step 4.1: Use the local maximum detection algorithm to identify the local maximum in each row of the data matrix of the predicted sound source distribution map and find the first three maximum values.

[0113] Step 4.2: Verify whether these values ​​are true local maxima through the neighborhood. By comparing the target value with the values ​​in its neighborhood, determine whether the value is the largest in the local range. If a local maximum is identified, sort the detected local maximums.

[0114] Step 4.3: Based on the sorting results, use coordinate transformation technology to map the maximum row and column indexes to the actual coordinates to calculate the location and intensity of the source

[0115] This process combines local maximum detection, coordinate transformation and sorting to improve the accuracy of source location and intensity prediction.

[0116] Figure 5 (a) is the positioning result diagram when the single sound source position is [-0.31, 0.27, 0], SNR = 20dB, f = 1000Hz, and z = 1m. Figure 5 (b) is the positioning result diagram when the single sound source is at the position [0.17, 0.53, 0], SNR = 20dB, f = 2000Hz, and z = 1m. Figure 5 (c) is the positioning result diagram when the single sound source position is [-0.57, -0.15, 0], SNR = 20dB, f = 6000Hz, and z = 1m.

[0117] Figure 6(a) is the positioning result diagram when the dual sound source positions are [-0.21, -0.19, 0], [0.59, -0.19, 0], SNR = 20dB, f = 1000Hz, and z = 1m. Figure 6 (b) is the positioning result diagram when the dual sound source positions are [0.05, -0.27, 0], [-0.25, -0.39, 0], SNR = 20dB, f = 2000Hz, and z = 1m. Figure 6 (c) is the positioning result diagram when the dual sound source positions are [-0.59, -0.13, 0], [0.47, -0.27, 0], SNR = 20dB, f = 6000Hz, and z = 1m.

[0118] Figure 7 (a) is the positioning result diagram at the three sound source positions [-0.07, 0.53, 0], [0.35, 0.11, 0], [-0.05, -0.41, 0], SNR = 20dB, f = 1000Hz, z = 1m. Figure 7 (b) is the positioning result diagram at the three sound source positions [-0.15, 0.19, 0], [-0.39, 0.05, 0], [-0.31, -0.63, 0], SNR = 20dB, f = 2000Hz, z = 1m. Figure 7 (c) is the positioning result diagram at the three sound source positions [0.53, 0.37, 0], [0.11, 0.17, 0], [0.45, -0.15, 0], SNR = 20dB, f = 6000Hz, z = 1m.

[0119] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A sound source recognition method based on array expansion and SE moving reverse bottleneck convolutional neural network, characterized in that: Methods include: Step 1: Obtain a sound pressure cross-spectrum matrix, and use the sound pressure cross-spectrum matrix to calculate an 18-array sound source distribution diagram and a 64-array sound source distribution diagram of a MUSIC algorithm; Step 2: Input the 18-array sound source distribution map into the EAG-U-Net data conversion model to obtain the converted 64-array sound source distribution map; the EAG-U-Net data conversion model is trained with the 18-array sound source distribution map as input and the 64-array sound source distribution map as label; Step 3: Input the converted 64-array sound source distribution map into the SE-MBCNet network model to obtain a predicted sound source distribution map; the SE-MBCNet network model is obtained by taking the 64-array sound source distribution map after data conversion as input and the target map as label training; Step 4: Perform local maximum detection on the predicted sound source distribution map to obtain the sound source location and intensity.

2. The sound source identification method based on array expansion and SE moving reverse bottleneck convolutional neural network according to claim 1 is characterized in that: Acquiring the 18-array sound source distribution diagram and the 64-array sound source distribution diagram comprises: Step 1.1: grid the sound source plane, discretize the sound source calculation plane, form a grid surface, i.e., a focus plane, and use each grid point of the focus plane as a potential sound source point; Step 1.2: Arrange a plurality of microphone sensors in the sound field formed by the radiation of a plurality of sound sources to form a planar microphone array, i.e., a measurement plane, and make the measurement plane parallel to the sound source plane; Step 1.3: receiving frequency sound pressure signal data collected by the sensor array, wherein the sound pressure signal data includes: a sound source signal and a noise signal; Step 1.4: pre-process the frequency sound pressure signal data, calculate the sound pressure cross-spectrum matrix, and obtain the 18-array sound source distribution map and the 64-array sound source distribution map according to the MUSIC algorithm.

3. The sound source recognition method based on array expansion and SE moving reverse bottleneck convolutional neural network according to claim 1 is characterized in that: Obtaining the converted 64-array sound source distribution diagram includes: Step 2.1: Input the 18-array sound source distribution map into the initial convolution layer of the EAG-U-Net data conversion model, and the initial convolution layer uses two 3×3 convolutions to continuously perform preliminary processing on the input image to extract the basic features of the 18-array sound source distribution map; Step 2.2: Input the basic features into the encoder of the EAG-U-Net data conversion model. The encoder consists of three double convolutional layers and a pooling layer. By continuously extracting image features and downsampling, the first high-level features are gradually extracted; the double convolutional layer contains two 3×3 convolutions, a ReLU activation function, and a batch normalization layer. The pooling layer uses maximum pooling for downsampling, reduces the spatial dimension, and increases the receptive field, thereby reducing the amount of calculation and extracting more abstract features, namely, the first high-level features; Step 2.3: After the pooling operation in the third encoder block, an ECA attention mechanism is added. The ECA attention mechanism obtains the average value of each channel by performing global average pooling to capture global information. Based on the global information, a one-dimensional convolution operation is performed on the pooled channel information to capture the dependency between channels. Based on the dependency, a weight coefficient is obtained by a Sigmoid activation function, and a weighted operation is performed on each channel using the weight coefficient. Step 2.4: Through the AG gated attention mechanism, the encoder feature map Fe and the decoder feature map Fd of the corresponding level are spliced ​​to generate a first fused feature map, and the first fused feature map is subjected to 1×1 convolution to extract higher-level feature information, reduce the number of channels and generate input features of the attention map, and the attention weight map A is generated using the Sigmoid activation function. The attention weight map A is multiplied element by element with the encoder feature map Fe to apply the attention weight to the feature map of the encoder, and the weighting of the feature map is dynamically adjusted. The weighted feature map is fused with the feature map in the decoder to obtain the second fused feature map; When restoring the spatial resolution, the decoder will be spliced ​​with the feature map of the encoder to form a first fused feature map, which includes the underlying features of the encoder and the high-level semantic information of the decoder; Step 2.5: The output of the third encoder block is upsampled by transposed convolution, and the obtained result and the output of the second encoder block are input into the AG gated attention mechanism for processing, and the obtained feature map is concatenated with the input upsampled result to obtain the output result of the third layer decoder block; The output result of the third layer decoder block is upsampled by transposed convolution, and the obtained result and the output of the first encoder block are input into the AG gated attention mechanism for processing, and the obtained feature map is concatenated with the input upsampled result to obtain the output result of the second layer decoder block; The output of the second layer decoder block is upsampled by transposed convolution, and the obtained result and the output of the initial convolutional layer are input into the AG gated attention mechanism for processing. The obtained feature map is concatenated with the input upsampled result to obtain the output of the first layer decoder block; The output result of the first layer decoder block is used to restore the image details using a double convolutional layer to obtain a 64-array sound source distribution map after data conversion.

4. The sound source identification method based on array expansion and SE moving reverse bottleneck convolutional neural network according to claim 3 is characterized in that: Acquiring the predicted sound source distribution map includes: Step 3.1: Input the 64-array sound source distribution map after data conversion into the double-layer convolution module of the SE-MBCNet network model, and the double-layer convolution module is used to perform preliminary feature extraction on the 64-array sound source distribution map after data conversion; the double-layer convolution module includes two consecutive 3×3 convolution operations, a batch normalization layer BN and an activation function ReLU; Step 3.2: Input the extracted preliminary features into the encoder module of the SE-MBCNet network model, and the encoder module is used to perform multi-level semantic information and feature encoding on the feature map after the preliminary feature extraction; the encoder module is composed of four MBConv layers of SE attention mechanism and a maximum pooling layer, and the second high-level features are gradually extracted by continuously extracting image features and downsampling; the MBConv layer of the SE attention mechanism includes: depthwise separable convolution, SE attention mechanism and residual connection; The workflow of depthwise separable convolution, SE attention mechanism and residual includes: First, the dimension is increased by 1×1 convolution, batch normalization BN and Swish activation function to expand the feature dimension and increase the number of channels. Then, a depth-separable convolution is performed, in which 3×3 depth convolution is combined with batch normalization BN and Swish activation function to process each channel independently to extract local features. Subsequently, the SE channel attention mechanism is introduced, and the global average pooling layer is first used for compression to effectively capture global features and reduce computational complexity. Then, in the excitation stage, weights are adaptively generated for each channel through the fully connected layer and Sigmoid activation function to enhance important features. In the scaling stage, the features of each channel are re-weighted by multiplying the obtained channel weights by each channel of the original feature map element by element. Finally, a 1×1 convolution is used to reduce the weighted feature map to the number of output channels, and a residual connection is added to maintain network stability and accelerate training. Step 3.3: Input the feature map processed by the encoder module of the SE-MBCNet network model into the decoder to restore the high-resolution feature map; the decoder is divided into four stages, and the decoder block of each stage directly concatenates the feature map with the corresponding layer of the encoder through a jump connection, merges the downsampled feature map with the upsampled feature map, and enhances the features of the restored image; the encoder block is composed of the MBConv layer of the SE attention mechanism and the upsampling layer, which retains the detail information, thereby avoiding the loss of important features during the upsampling process; Step 3.4: Through the 1×1 convolutional layer at the end of the network, the final predicted sound source distribution map with sound source position coordinates and intensity information is obtained.

5. The sound source identification method based on array expansion and SE moving reverse bottleneck convolutional neural network according to claim 1 is characterized in that: Acquiring the sound source positioning and intensity includes: Step 4.1: Use the local maximum detection algorithm to identify the local maximum in each row of the data matrix of the predicted sound source distribution map and find the first three maximum values; Step 4.2: Verify whether the maximum value is a true local maximum value through the neighborhood, that is, by comparing the target value with the value in its neighborhood, determine whether the maximum value is the largest in the local range. If the maximum value is the largest in the local range, sort the detected local maximum values; Step 4.3: Based on the sorting results, use the coordinate transformation technology to map the maximum row and column indexes to the actual coordinates to calculate the source position and its intensity.

Images