Lightweight convolutional network sound source identification method based on dependency graph pruning and channel distillation

Abstract

The invention provides a lightweight convolutional network sound source identification method based on dependency graph pruning and channel distillation. Comprising the following steps: calculating a 64 array sound source distribution diagram under a function beam forming algorithm by using a sound pressure cross-spectrum matrix, and jointly inputting the sound source distribution diagram and a sound source position label into a dense connection full convolutional neural network for pre-training; components in the pre-trained network model are grouped through a dependency graph, a pruning part is accurately identified and cut off, and a network lightweight target is achieved on the premise of keeping the overall architecture of the network stable; taking a dense connection full convolutional neural network as a teacher network, taking the pruned network as a student network, and extracting intermediate features of the two networks to implement channel distillation to obtain a model after distillation; and detecting the number of sound sources by predicting the sound source distribution diagram, and determining the specific position of the sound source according to the local maximum value. According to the method, the positioning performance of the original network model is basically maintained, and meanwhile, the parameter quantity and the calculation quantity of the network model are effectively reduced.

Description

Technical Field

[0001] This invention relates to the fields of sound source localization and deep learning technology, and proposes a lightweight convolutional network sound source recognition method based on dependency graph pruning and channel distillation. Background Technology

[0002] Sound source localization technology has important applications in mechanical fault diagnosis, environmental noise control, and automotive NVH development. In industrial environments, equipment fault sounds are often masked by other noises, making it difficult to detect problems in a timely manner. Sound source localization can accurately identify weak fault signals and effectively locate the noise source, providing support for noise control and equipment maintenance. With the development of microphone array and beamforming technologies, the accuracy and practicality of sound source localization are constantly improving. Meanwhile, advancements in artificial intelligence and deep learning have enabled sound signal-based localization and recognition technologies to show broad prospects in areas such as speech recognition, autonomous driving, and security monitoring. However, traditional methods still face problems such as insufficient accuracy, high hardware dependence, and high computational costs, urgently requiring new technological innovations to solve these problems.

[0003] Sound source localization technology has made significant progress in recent years, with Delayed Summation (DAS) beamforming being a classic method. It estimates the sound source intensity in a specific direction by compensating for the time delay of the microphone array signals and summing them, generating a beamforming map reflecting the spatial distribution of the sound sources. DAS is fast and insensitive to sensor position errors, but it has a wide main lobe and low spatial resolution. Function beamforming (FBF) utilizes the eigenvalues ​​of the array signal covariance matrix and improves the resolution of the beamforming map through weighted processing, outperforming DAS. However, FBF still has a wide main lobe at low frequencies, easily misclassifying nearby sound sources as a single entity. To address the problem of blurred beamforming maps caused by limited array aperture and insufficient microphone quantity, deconvolution-based sound source localization techniques have emerged. Deconvolutional Sound Source Localization (DAMAS) is a commonly used method. It treats the DAS beamforming map as a convolution of the true sound source intensity distribution and the point spread function (PSF), transforming the localization task into the inverse problem of recovering the true sound source distribution from the DAS map. Brooks and Humphreys solved the problem using the Gauss-Seidel iterative method, but it was computationally expensive. Lylloff et al. proposed the FFT-NNLS and FFT-FISTA algorithms, assuming that the PSF has translation invariance and transforming convolution into multiplication to reduce complexity. However, this assumption only holds when the sound source region is much smaller than the array-sound source distance. Furthermore, sparse signal recovery techniques have also been used to solve the DAMAS inverse problem. Yardibi et al. first introduced the assumption of sparse sound source distribution to improve reconstruction accuracy, and this method has since been widely used; for example, Padois and Berry solved it using the Orthogonal Matching Pursuit (OMP) algorithm. The norm regularization problem (OMP-DAMAS) was solved by Zhang et al. using the Fast Iterative Shrink Thresholding Algorithm (FISTA). Norm regularization problems (DAMAS-FISTA-LASSO). The sparsity assumption can reduce the solution space, but these methods are sensitive to the regularization parameter, are prone to convergence to local optima, and may also produce spurious sources.

[0004] With the rapid development of deep learning technology, its application in signal processing has gradually shown great potential. In sound source identification, deep learning-based methods are mainly divided into two categories: grid-based and gridless. In the research of Kujawski et al., a convolutional neural network (CNN)-based model was developed to estimate the intensity and two-dimensional coordinates of a single sound source. In the research of Castellini et al., the real and imaginary parts of the array signal covariance matrix were jointly used as input features, and a multilayer perceptron (MLP) model was employed to estimate the intensity and coordinates of three sound sources. In the research of Kujawski and Sarradj, a Transformer model was used to encode the eigenvectors of the array signal covariance matrix, and then the encoded results were input into an MLP layer to predict the coordinates and intensity of the sound source. However, these methods require training separate models for different frequencies, resulting in poor flexibility.

[0005] To address this issue, Lee et al. incorporated frequencies of interest as prior information into the loss function of a DNN to improve the localization performance of a single sound source across different frequencies. However, this method has limited generalization ability in multi-source scenarios. In the research of Feng et al., a two-stage method, Two-Step Meshless (DSGF), was proposed for sound source localization. This method first uses a classification CNN to estimate the number of sound sources, and then uses a regression DNN to estimate the corresponding coordinates and intensities. DSGF is trained on a large dataset containing multiple frequencies to improve its generalization ability across different frequencies. However, DSGF requires a predefined maximum number of sound sources. Summary of the Invention

[0006] To address the problems of insufficient accuracy and high computational cost of traditional methods in existing technologies, and the poor flexibility, limited generalization ability, and difficulty in balancing lightweight and high performance of deep learning methods, this invention proposes a lightweight convolutional network sound source recognition method based on dependency graph pruning and channel distillation.

[0007] To achieve the above objectives, the present invention provides the following solution: A lightweight convolutional network-based sound source identification method based on dependency graph pruning and channel distillation, the method comprising: Based on the frequency sound pressure signal data collected by the sensor array, the sound pressure cross spectrum matrix is ​​obtained, and the 64 array sound source distribution map of the FBF algorithm is calculated using the sound pressure cross spectrum matrix. The 64-array sound source distribution map and sound source location labels are input into the densely connected fully convolutional neural network DCFCN for pre-training to obtain the DCFCN pre-trained network model. The components of the DCFCN pre-trained network model are divided into different pruning groups. Based on the importance factor ranking results of the pruning groups, channels in the network components that do not meet the preset importance requirements are pruned to obtain the pruned network model DCFCN-P, while maintaining the overall structure of DCFCN. Based on using the DCFCN pre-trained network model as the teacher network and the pruned network model DCFCN-P as the student network, the intermediate features of the two networks are extracted to perform CWD channel distillation, resulting in the distilled network model DCFCN-P-CWD. Based on the distilled network model DCFCN-P-CWD, a predicted sound source distribution map is obtained. Local maxima detection is performed on the predicted sound source distribution map to obtain the number and location of sound sources.

[0008] Preferably, obtaining the 64-array sound source distribution map includes: The sound source plane is meshed, the sound source calculation plane is discretized to form a mesh surface, i.e. a focusing plane, and each mesh point of the focusing plane is used as a potential sound source point. Multiple microphone sensors are arranged in the sound field formed by the radiation of several sound sources to form a planar microphone array, i.e., a measurement plane, which is parallel to the sound source plane. The system receives frequency sound pressure signal data collected by a planar microphone array, wherein the sound pressure signal data includes: sound source signal and noise signal; The frequency sound pressure signal data is preprocessed to calculate the sound pressure cross spectrum matrix, and the distribution map of the 64 array sound sources is obtained according to the FBF algorithm.

[0009] Preferably, obtaining the DCFCN pre-trained network model includes: The initial convolutional layer processes the 64-array sound source distribution map and sound source location label map to extract the basic features of the input data. The encoder consists of four encoder blocks, each of which includes a densely connected module with dilated convolution, a dimension-reduced convolution, and a max-pooling layer. The encoder extracts the first high-level features stepwise through downsampling, and the fourth encoder block adds a convolutional layer after pooling to double the channels to obtain deeper features. The decoder consists of four decoder blocks. It combines transposed convolutional upsampling with densely connected convolutional layers and dimensionality reduction convolutions to restore spatial resolution. When this is done, the blocks are concatenated with the enhanced features to fuse the low-level features of the encoder and the high-level semantic information of the decoder, forming a fused feature map containing multi-scale information. In the fourth decoder block, a convolution is added after the dimensionality reduction convolution to make the output size the same as the input size, thus obtaining a pre-trained network model.

[0010] Preferably, the method for obtaining the pruned network model DCFCN-P includes: The components in the DCFCN pre-trained network model are split into inputs and outputs, a dependency graph is constructed based on the input-output relationship, and pruning groups are divided based on the dependency graph. The 64-array sound source distribution map and sound source location label map are subjected to regularized training to obtain the importance factor corresponding to each channel of the batch normalization layer in the DCFCN pre-trained network model. Sort all the importance factors of the pruned groups from largest to smallest, and based on the preset pruning ratio, complete the channel pruning of the DCFCN pre-trained network model to obtain the pruned network model DCFCN-P.

[0011] Preferably, the method for obtaining the distilled network model DCFCN-P-CWD includes: The DCFCN pre-trained network model is used as the teacher network, and the pruned network model DCFCN-P is used as the student network. The intermediate feature maps of the two networks are extracted. The channels of the intermediate feature map are normalized using a temperature-sparse softmax function, and the loss of the intermediate feature map is obtained by measuring the difference in channel probability distribution between the teacher network and the student network using KL divergence. The student network is trained using the task loss of the student network and the intermediate feature map loss of the teacher network and the student network at a preset ratio, resulting in the distilled network model DCFCN-P-CWD.

[0012] Preferably, obtaining the sound source localization includes: The local maximum detection algorithm is used to identify the local maxima in each row of the data matrix of the predicted sound source distribution map and find the top three candidate values ​​sorted from largest to smallest. For each of the three candidate values, by comparing it with the neighborhood values ​​within a preset local range, it is determined whether the candidate value is the largest within the preset local range. If the candidate value is the largest within the preset local range, the detected local maxima are sorted to obtain a sorting result. Based on the sorting results, the number of sound sources is obtained, and coordinate transformation technology is used to map the row and column indices of local maxima to actual coordinates to obtain the sound source locations.

[0013] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention proposes a lightweight convolutional network sound source recognition method based on dependency graph pruning and channel distillation. By introducing an innovative lightweight network model strategy, the performance level of sound source recognition is effectively maintained while significantly reducing the number of network model parameters and computational cost. Specifically, firstly, a 64-array sound source distribution map under the beamforming (FBF) algorithm based on the sound pressure cross-spectrum matrix calculation function is used. This distribution map and sound source location labels are input into a densely connected fully convolutional neural network (DCFCN) for pre-training. Then, the pre-trained model components are grouped and regularized through dependency graphs to accurately identify and prune redundant parts, achieving network lightweighting while maintaining network architecture stability. Finally, DCFCN is used as the teacher network and the pruned DCFCN-P is used as the student network. Intermediate features are extracted through CWD channel distillation to enhance the performance of the student network.

[0014] This invention combines pruning and distillation techniques to precisely process traditional deep learning network models, effectively solving the problem of high network localization accuracy but high computational cost. In methods combining sound source recognition and deep learning, complex networks can guarantee accuracy but incur high computational costs, while simple networks, although lightweight, suffer from poor performance. This invention removes redundant parameters from the network model through targeted pruning and then improves its localization performance using distillation techniques. This method effectively reduces the number of network model parameters and computational cost while essentially maintaining the original network model's localization performance. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a sound source localization measurement model according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the specific structure of the DCFCN network according to an embodiment of the present invention; wherein, (a) is a DCFCN structure diagram and (b) is a residual dense block structure diagram; Figure 4 This is a schematic diagram of the pruning structure according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the CWD distillation structure according to an embodiment of the present invention; Figure 6The following are comparison charts of positioning accuracy and position error in embodiments of the present invention; wherein, (a) is a comparison chart of positioning accuracy of DCFCN, DCFCN-P and DCFCN-P-CWD at frequencies of 1000 Hz, 2000 Hz, 4000 Hz, 6000 Hz and 8000 Hz respectively, and (b) is a comparison chart of position error of DCFCN, DCFCN-P and DCFCN-P-CWD at frequencies of 1000 Hz, 2000 Hz, 4000 Hz, 6000 Hz and 8000 Hz respectively. Figure 7 This is a qualitative analysis diagram of an embodiment of the present invention; wherein, (a) represents the position of the three sound sources at [-0.09, -0.41, 1], [-0.39, -0.25, 1], [0.25, 0.49, 1], SNR =15 dB f =1000 Hz z The label image at =1 m, (b) shows the labels at the three sound source positions [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR =15 dB f =4000 Hz z The label image at =1 m, (c) shows the labels at the three sound source positions [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39, 0.41,1], SNR =15 dB f =8000 Hz z =1 m, (d) is the label image at the three sound source positions [-0.09,-0.41,1], [-0.39,-0.25,1], [0.25,0.49,1], SNR =15 dB f =1000 Hz z =1 m, FBF position localization result diagram, (e) is the position at the three sound source locations [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR =15dB f =4000 Hz z =1 m, FBF position localization result diagram, (f) is the position at the three sound source locations [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39, 0.41,1], SNR =15 dB f =8000 Hz z=1 m, FBF position localization result diagram, (g) is the position at the three sound source locations [-0.09,-0.41,1], [-0.39,-0.25,1], [0.25,0.49,1], SNR =15 dB f =1000 Hz z =1 m, DCFCN location positioning result diagram, (h) is at the three sound source positions [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR =15 dB f =4000 Hz z =1 m, the DCFCN location results are shown in the figure. (i) shows the location of the three sound sources at [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39, 0.41,1], SNR =15 dB f =8000 Hz z =1 m, DCFCN location positioning result diagram, (j) is the location at the three sound source positions [-0.09,-0.41,1], [-0.39,-0.25,1], [0.25,0.49,1], SNR =15 dB f =1000 Hz z =1 m, DCFCN-P location positioning result diagram, (k) is the position of the three sound sources at [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR =15dB f =4000 Hz z =1 m, the DCFCN-P position positioning result diagram, (l) is the position of the three sound sources at [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39, 0.41,1], SNR =15 dB f =8000 Hz z =1 m, DCFCN-P position localization result diagram, (m) is the position of the three sound sources at [-0.09,-0.41,1], [-0.39,-0.25,1], [0.25,0.49,1], SNR =15 dB f =1000 Hz z=1 m, DCFCN-P-CWD location positioning result diagram, (n) is the location of the three sound sources at [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR =15 dB f =4000 Hz z =1 m, the DCFCN-P-CWD position positioning result diagram, (o) is the position of the three sound sources at [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39, 0.41,1], SNR =15 dB f =8000 Hz z The location positioning result of DCFCN-P-CWD when =1 m. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] Example 1: This embodiment provides a lightweight convolutional network sound source recognition method based on dependency graph pruning and channel distillation, including: S1: Based on the frequency sound pressure signal data collected by the sensor array, obtain the sound pressure cross spectrum matrix, and use the sound pressure cross spectrum matrix to calculate the 64 array sound source distribution map of the FBF algorithm; S2: Input the 64-array sound source distribution map and sound source location labels into the densely connected fully convolutional neural network DCFCN for pre-training to obtain the DCFCN pre-trained network model; S3: Divide the components of the DCFCN pre-trained network model into different pruning groups. Based on the importance factor ranking results of the pruning groups, remove the channels in the network components that do not meet the preset importance requirements to obtain the pruned network model DCFCN-P, while maintaining the overall structure of DCFCN. S4: Based on using the DCFCN pre-trained network model as the teacher network and the pruned network model DCFCN-P as the student network, the intermediate features of the two networks are extracted to perform CWD channel distillation, resulting in the distilled network model DCFCN-P-CWD. S5: Based on the distilled network model DCFCN-P-CWD, a predicted sound source distribution map is obtained. Local maxima detection is performed on the predicted sound source distribution map to obtain the number and location of sound sources.

[0020] S1: Obtaining the sound source distribution map of the 64-array includes: The sound source plane is meshed, the sound source calculation plane is discretized to form a mesh surface, i.e. a focusing plane, and each mesh point of the focusing plane is used as a potential sound source point. Multiple microphone sensors are arranged in the sound field formed by the radiation of several sound sources to form a planar microphone array, i.e., a measurement plane, which is parallel to the sound source plane. The system receives frequency sound pressure signal data collected by a sensor array, the sound pressure signal data including: sound source signal and noise signal; The frequency sound pressure signal data is preprocessed to calculate the sound pressure cross spectrum matrix, and the distribution map of the 64 array sound sources is obtained according to the FBF algorithm.

[0021] S2: Obtaining the DCFCN pre-trained network model includes: The initial convolutional layer processes the 64 array sound source distribution map and sound source location label map to extract the basic features of the input data; The encoder consists of four encoder blocks: a densely connected module with dilated convolution (three layers of convolution), a dimension reduction convolution, and a max pooling layer. It extracts the first high-level features step by step through downsampling, and the fourth encoder block adds a convolutional layer after pooling to double the channels to deepen the features. A convolutional layer is added after the pooling operation in the fourth encoder block to double the number of channels and further extract deep features.

[0022] The decoder consists of four decoder blocks, which combine transposed convolutional upsampling with densely connected convolutional layers and dimensionality-reducing convolutions to restore spatial resolution. When the spatial resolution is restored, the blocks are concatenated with the encoder feature map to fuse low-level features and high-level semantic information, forming a fused feature map containing multi-scale information. After the dimensionality reduction convolution in the fourth decoder block, a convolution is added to make the output size the same as the input size, thus obtaining the pre-trained network model.

[0023] S3: Methods for obtaining the pruned network model DCFCN-P include: The components in the DCFCN pre-trained network model are split into inputs and outputs. A dependency graph is constructed based on the input-output relationship, with mutual dependence being 1 and non-mutual dependence being 0. The model is then divided into different pruning groups based on the dependency graph. The 64-array sound source distribution map and sound source location label map are input into the DCFCN pre-trained network model for regularization training. By adding a penalty term to the weight gradient of the batch normalization (BN) layer, the small weights are made sparse quickly. After regularized training, the importance factor corresponding to each channel in the batch normalization (BN) layer is obtained. Global pruning is then enabled, and the importance factors of all pruned groups are sorted from largest to smallest. Channels to be retained are selected according to the pruning ratio, and the remaining unimportant channels are pruned, resulting in the pruned network model DCFCN-P.

[0024] S4: Methods for obtaining the distilled network model DCFCN-P-CWD include: The DCFCN pre-trained network model is used as the teacher network, and the pruned network model DCFCN-P is used as the student network. The intermediate feature maps of the two networks are extracted. The channels of the intermediate feature map are normalized using a temperature-sparse softmax function to eliminate the scale difference between the teacher and student networks. The loss of the intermediate feature map is obtained by measuring the difference in the channel probability distribution of the teacher and student networks using KL divergence. The student network is trained using the task loss of the student network and a certain proportion of the intermediate feature map loss of the teacher and student networks, resulting in the distilled network model DCFCN-P-CWD.

[0025] S5: Obtaining the sound source localization includes: The local maximum detection algorithm is used to identify the local maxima in each row of the data matrix of the predicted sound source distribution map and find the top three candidate values ​​sorted from largest to smallest. For each of the three candidate values, by comparing it with the neighborhood values ​​within a preset local range, it is determined whether the candidate value is the largest within the preset local range. If the candidate value is the largest within the preset local range, the detected local maxima are sorted to obtain a sorting result. Based on the sorting results, the number of sound sources is obtained, and coordinate transformation technology is used to map the row and column indices of local maxima to actual coordinates to obtain the sound source locations.

[0026] Example 2: like Figure 1 As shown, a lightweight convolutional network-based sound source recognition method based on dependency graph pruning and channel distillation includes: S1: Obtain the sound pressure cross-spectrum matrix and calculate the 64-array sound source distribution map of the FBF algorithm; such as Figure 2As shown, a further embodiment is that the sound source localization measurement model includes: exist K Arranged in the sound field formed by the radiation of each sound source M One microphone sensor. M The sensors form a planar microphone array, called the measurement plane, which is parallel to the sound source plane, and the distance between the two planes is z.

[0027] The sound source plane is meshed, and the discrete sound source computation plane forms a mesh surface called the focusing plane. The focusing plane contains N There are 10 grid points, each of which is also called a focal point, and each focal point serves as a potential sound source point.

[0028] Receive frequency sound pressure signal data collected by sensor array p It contains the sound source signal and the noise signal, and then its sound pressure cross-spectral matrix C is calculated. in, H denotes the conjugate transpose of a matrix.

[0029] Function beamforming involves eigenvalue decomposition of the cross-spectral matrix. The resulting cross-spectral matrix after eigenvalue decomposition can be expressed as: in, This is the eigenvector matrix. It is a diagonal eigenvalue matrix. .

[0030] use and Construct the cross-spectral matrix function: in, It refers to numerical values, generally .

[0031] The formula for calculating acoustic power in function beamforming is: in, It is a direction vector. Indicates the distance from the nth microphone to the nth microphone. j The distance between grid points This represents a weighted vector.

[0032] Sound power B Normalization is performed to perform acoustic imaging, resulting in a sound source distribution map.

[0033] A target image was designed as the ground truth map for acoustic imaging, which was calculated based on the location, intensity, and frequency of the sound source. in, Indicates the first k The intensity of each sound source, Indicates the first in the target graph k The sound source and the first n The distance between grid points η and β These are two adjustable hyperparameters. η It is a very small constant used to avoid a denominator of 0. β Used to control the decay rate of the main lobe in the target image.

[0034] S2: Based on the 64-array sound source distribution map and sound source location label map, a densely connected fully convolutional neural network (DCFCN) is pre-trained to obtain a pre-trained network model; such as... Figure 3 (a) and Figure 3 As shown in (b), a further implementation method is that the structure of the pre-trained network model includes: The initial convolutional layer inputs the 64-array sound source distribution map and the sound source location label map into the DCFCN's initial convolutional layer. The initial convolutional layer uses a 1×1 convolution to perform preliminary processing on the input image and extract the basic features of the 64-array sound source distribution map. The encoder consists of three layers: a convolutional layer, a dimensionality-reducing convolutional layer, and a pooling layer. It extracts image features sequentially and then downsamples them to progressively extract high-level features. Each convolutional layer uses dilated convolutions, which increases the receptive field without changing the kernel size. Each layer includes a 3×3 convolution, a ReLU activation function, and a batch normalization layer. The three convolutional layers are densely connected, with the output of the densely connected module being the original input plus the outputs of all preceding convolutional layers. The dimensionality-reducing convolutional layer includes a 1×1 ordinary convolution, a ReLU activation function, and a batch normalization layer, compressing the number of channels to reduce computation. The pooling layer uses max pooling for downsampling, reducing spatial dimensionality, increasing the receptive field, thereby reducing computation and extracting more abstract features. A convolutional layer is added after the pooling operation in the fourth encoder block. This layer contains a 3×3 convolution, a ReLU activation function, and a batch normalization layer, which doubles the number of channels and further extracts deep features.

[0035] The decoder consists of a transposed convolutional layer, three convolutional layers, and a dimensionality-reducing convolutional layer. The transposed convolutional layer performs upsampling, containing a transposed convolutional layer, a ReLU activation function, and a batch normalization layer, doubling the feature map size and halving the number of channels. The three convolutional layers are also densely connected, with each layer containing a regular convolution, a ReLU activation function, a batch normalization layer, and a dimensionality-reducing convolutional layer. The dimensionality-reducing convolutional layer contains a 1×1 regular convolution, a ReLU activation function, and a batch normalization layer, compressing the number of channels and reducing computational cost. The kernel sizes of the four decoders are 4, 4, 5, and 4, respectively.

[0036] When restoring spatial resolution, the decoder concatenates the feature map of the encoder with the feature map of the encoder to form a first fused feature map. The first fused feature map contains the low-level features of the encoder and the high-level semantic information of the decoder. After the dimensionality reduction convolution in the fourth decoder, a 1×1 convolution is added to make the output size the same as the input size, thus obtaining the pre-trained network model.

[0037] S3: Divide the components of the model into different pruning groups, find the unimportant channels in the network components, and use pruning to remove the unimportant channels to obtain the pruned DCFCN-P model while maintaining the overall structure of DCFCN.

[0038] A further implementation method is, such as Figure 4 As shown, methods for pruning pre-trained network models include: The components in the pre-trained network model are split into inputs and outputs. A dependency graph is constructed based on the input-output relationship, with mutual dependence being 1 and non-dependency being 0. The model is then divided into different pruning groups based on the dependency graph. The formula for constructing the dependency graph is: in, L This indicates that the entire network is divided into L Components, Components Indicates the input feature dimension, component Indicates the output feature dimension. Indicates inter-layer dependency. Indicates intra-layer dependency; in, Represents a dependency graph matrix, if and If a connection exists, the value is 1, or The input and output pruning methods are both 1; The workflow for dividing pruning groups includes: First, the entire neural network is broken down into basic components, each labeled with its input and output feature dimensions. Then, dependencies are determined according to rules: direct connections between different components are considered inter-layer dependencies, and identical pruning rules for the input and output of the same component are considered intra-layer dependencies. Finally, a matrix is ​​used to record these dependencies, forming a dependency graph. Next, pruning groups are found based on this dependency graph. First, the input and output dimensions of all components are placed into an "unvisited" set. Then, one of these sets is selected as a starting point, and all components that depend on it are found through the dependency graph. These components are grouped into a pruning group, and then removed from the "unvisited" set. This process is repeated until no more dependent components can be found. The dimensions in each resulting pruning group are pruned synchronously to avoid disrupting the network structure. Step 3.2: Input the 64 array sound source distribution map and sound source location label map into the pre-trained network model for regularization training. By adding a penalty term to the weight gradient of the batch normalization (BN) layer, the small weights are made sparse quickly. The regularization training formula is as follows: in, Represents the weight gradient. Indicates the importance factor of the BN layer. This indicates the regularization strength, i.e., the penalty term; The regularization workflow includes: First, obtain the importance factor of each channel. The importance factor is the standard for measuring the importance of the channel. Perform regularization training and add a regularization strength multiplied by the positive or negative sign of the importance factor itself to this weight gradient. This will allow the weight value to gradually approach 0, making it easier to prune unimportant channels later. After regularized training, the importance factor corresponding to each channel in the batch normalization (BN) layer is obtained. Global pruning is enabled, and the importance factors of all pruned groups are sorted from largest to smallest. Channels to be retained are selected according to the pruning ratio, and the remaining unimportant channels are pruned to obtain the pruned lightweight network model. S4: Using the pre-trained DCFCN network model as the teacher network and the pruned DCFCN-P model as the student network, the intermediate features of the two networks are extracted for CWD channel distillation, resulting in the DCFCN-P-CWD model. A schematic diagram of the CWD distillation structure is shown below. Figure 5 .

[0039] A further implementation method is, such as Figure 5 As shown, methods for distilling network models include: The pre-trained DCFCN network model is used as the teacher network, and the pruned DCFCN-P network model is used as the student network. The intermediate feature maps of the two networks are extracted. The channels of the intermediate feature map are normalized using a temperature-sparse softmax function to eliminate the scale difference between the teacher and student networks. The loss of the intermediate feature map is obtained by measuring the difference in the channel probability distribution of the teacher and student networks using KL divergence. The formula for the KL divergence is: in, This represents the activation map of a single channel. Indicates the first Channel 1 The activation value of each spatial location. Indicates the temperature coefficient. This represents the predicted output of the student network. This represents the output of the teacher network. Indicates KL divergence; C Represents the number of channels. W The width of the feature map. H This represents the height of the feature map.

[0040] The student network is trained using the task loss of the student network and a certain proportion of the intermediate feature map loss of the teacher and student networks to obtain the DCFCN-P-CWD model.

[0041] The loss function formula is as follows: in, Indicates mission loss. Indicates the true label, This represents the balance coefficient.

[0042] S5: Based on the sound source identification model, a predicted sound source distribution map is obtained, and local maxima detection is performed on the predicted sound source distribution map to obtain the number and location of sound sources.

[0043] A further implementation method includes: obtaining the number and location of sound sources. The local maximum detection algorithm is used to identify the local maxima in each row of the data matrix of the predicted sound source distribution map and find the top three candidate values ​​sorted from largest to smallest. For each of the three candidate values, by comparing it with the neighborhood values ​​within a preset local range, it is determined whether the candidate value is the largest within the preset local range. If the maximum value is the largest within the preset local range, the detected local maxima are sorted to obtain the sorting result. Based on the sorting results, the number of sound sources is obtained, and coordinate transformation technology is used to map the row and column indices of local maxima to actual coordinates to obtain the sound source locations. This process comprehensively applies local maximum detection, coordinate transformation, and sorting processing to improve the accuracy of source localization prediction.

[0044] Table 1 is a comparison table of the number of parameters and computational cost of DCFCN, DCFCN-P and DCFCN-P-CWD; Table 1

[0045] Figure 6 (a) is a comparison chart of the positioning accuracy of DCFCN, DCFCN-P and DCFCN-P-CWD at frequencies of 1000 Hz, 2000 Hz, 4000 Hz, 6000 Hz and 8000 Hz respectively; (b) is a comparison chart of the position error of DCFCN, DCFCN-P and DCFCN-P-CWD at frequencies of 1000 Hz, 2000 Hz, 4000 Hz, 6000 Hz and 8000 Hz respectively. Figure 7 (a) shows the positions of the three sound sources [-0.09, -0.41, 1], [-0.39, -0.25, 1], [0.25, 0.49, 1], ... SNR =15 dB f =1000 Hz z The label image at =1 m, (b) shows the labels at the three sound source positions [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR =15 dB f =4000 Hz z =1 m, (c) is the label image at the three sound source positions [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39, 0.41,1], SNR =15 dB f =8000Hz z =1 m, (d) is the label image at the three sound source positions [-0.09,-0.41,1], [-0.39,-0.25,1], [0.25,0.49,1], SNR =15 dB f =1000 Hz z =1 m, FBF position localization result diagram, (e) is the position at the three sound source locations [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR=15 dB f =4000 Hz z =1 m, FBF position localization result diagram, (f) is the position at the three sound source locations [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39, 0.41,1], SNR =15 dB f =8000 Hz z =1 m, FBF position localization result diagram, (g) is the position at the three sound source locations [-0.09,-0.41,1], [-0.39,-0.25,1], [0.25,0.49,1], SNR =15 dB f =1000 Hz z =1 m, DCFCN location positioning result diagram, (h) is at the three sound source positions [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR =15 dB f =4000 Hz z =1 m, the DCFCN location results are shown in the figure. (i) shows the location of the three sound sources at [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39, 0.41,1], SNR =15 dB f =8000Hz z =1 m, DCFCN location positioning result diagram, (j) is the location at the three sound source positions [-0.09,-0.41,1], [-0.39,-0.25,1], [0.25,0.49,1], SNR =15 dB f =1000 Hz z =1 m, DCFCN-P location positioning result diagram, (k) is the position of the three sound sources at [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR =15 dB f =4000 Hz z =1 m, the DCFCN-P position positioning result diagram, (l) is the position of the three sound sources at [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39, 0.41,1], SNR =15 dB f =8000 Hz z=1 m, DCFCN-P position localization result diagram, (m) is the position of the three sound sources at [-0.09,-0.41,1], [-0.39,-0.25,1], [0.25,0.49,1], SNR =15 dB f =1000 Hz z =1 m, DCFCN-P-CWD location positioning result diagram, (n) is the location of the three sound sources at [-0.03,-0.45,1], [-0.33,-0.03,1], [0.39,0.17,1], SNR =15 dB f =4000 Hz z =1 m, DCFCN-P-CWD position positioning result diagram, (o) is at the three sound source positions [0.35,-0.35,1], [-0.41,-0.09,1], [0.39,0.41,1], [0.39,0.41,1], SNR =15 dB f =8000 Hz z The location positioning result of DCFCN-P-CWD when =1 m.

[0046] The method of this invention can effectively reduce the number of network model parameters and computational cost without additional changes to the network model structure, while basically maintaining the localization performance of the original network model. It not only solves the contradiction between accuracy and computational cost in traditional methods, but also overcomes the problem of "high performance depends on high complexity" in deep learning models, providing an efficient solution for high-precision sound source localization in resource-constrained scenarios.

[0047] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A lightweight convolutional network sound source recognition method based on dependency graph pruning and channel distillation, characterized in that, The methods include: Based on the frequency sound pressure signal data collected by the sensor array, the sound pressure cross spectrum matrix is ​​obtained, and the 64 array sound source distribution map of the FBF algorithm is calculated using the sound pressure cross spectrum matrix. The 64-array sound source distribution map and sound source location labels are input into the densely connected fully convolutional neural network DCFCN for pre-training to obtain the DCFCN pre-trained network model. The components of the DCFCN pre-trained network model are divided into different pruning groups. Based on the importance factor ranking results of the pruning groups, channels in the network components that do not meet the preset importance requirements are pruned to obtain the pruned network model DCFCN-P, while maintaining the overall structure of DCFCN. Based on using the DCFCN pre-trained network model as the teacher network and the pruned network model DCFCN-P as the student network, the intermediate features of the two networks are extracted to perform CWD channel distillation, resulting in the distilled network model DCFCN-P-CWD. Based on the distilled network model DCFCN-P-CWD, a predicted sound source distribution map is obtained. Local maxima detection is performed on the predicted sound source distribution map to obtain the number and location of sound sources.

2. The method according to claim 1, characterized in that, Obtaining the 64-array sound source distribution map includes: The sound source plane is meshed, the sound source calculation plane is discretized to form a mesh surface, i.e. a focusing plane, and each mesh point of the focusing plane is used as a potential sound source point. Multiple microphone sensors are arranged in the sound field formed by the radiation of several sound sources to form a planar microphone array, i.e., a measurement plane, which is parallel to the sound source plane. The system receives frequency sound pressure signal data collected by a planar microphone array, wherein the sound pressure signal data includes: sound source signal and noise signal; The frequency sound pressure signal data is preprocessed to calculate the sound pressure cross spectrum matrix, and the distribution map of the 64 array sound sources is obtained according to the FBF algorithm.

3. The method according to claim 1, characterized in that, Obtaining the DCFCN pre-trained network model includes: The initial convolutional layer processes the 64-array sound source distribution map and sound source location label map to extract the basic features of the input data. The encoder consists of four encoder blocks, each of which includes a densely connected module with dilated convolution, a dimension-reduced convolution, and a max-pooling layer. The encoder extracts the first high-level features stepwise through downsampling, and the fourth encoder block adds a convolutional layer after pooling to double the channels to obtain deeper features. The decoder consists of four decoder blocks. It combines transposed convolutional upsampling with densely connected convolutional layers and dimensionality reduction convolutions to restore spatial resolution. When this is done, the blocks are concatenated with the enhanced features to fuse the low-level features of the encoder and the high-level semantic information of the decoder, forming a fused feature map containing multi-scale information. In the fourth decoder block, a convolution is added after the dimensionality reduction convolution to make the output size the same as the input size, thus obtaining a pre-trained network model.

4. The method according to claim 1, characterized in that, Methods for obtaining the pruned network model DCFCN-P include: The components in the DCFCN pre-trained network model are split into inputs and outputs, a dependency graph is constructed based on the input-output relationship, and pruning groups are divided based on the dependency graph. The 64-array sound source distribution map and sound source location label map are subjected to regularized training to obtain the importance factor corresponding to each channel of the batch normalization layer in the DCFCN pre-trained network model. Sort all the importance factors of the pruned groups from largest to smallest, and based on the preset pruning ratio, complete the channel pruning of the DCFCN pre-trained network model to obtain the pruned network model DCFCN-P.

5. The method according to claim 1, characterized in that, Methods for obtaining the distilled network model DCFCN-P-CWD include: The DCFCN pre-trained network model is used as the teacher network, and the pruned network model DCFCN-P is used as the student network. The intermediate feature maps of the two networks are extracted. The channels of the intermediate feature map are normalized using a temperature-sparse softmax function, and the loss of the intermediate feature map is obtained by measuring the difference in channel probability distribution between the teacher network and the student network using KL divergence. The student network is trained using the task loss of the student network and the intermediate feature map loss of the teacher network and the student network at a preset ratio, resulting in the distilled network model DCFCN-P-CWD.

6. The method according to claim 1, characterized in that, Obtaining the sound source localization includes: The local maximum detection algorithm is used to identify the local maxima in each row of the data matrix of the predicted sound source distribution map and find the top three candidate values ​​sorted from largest to smallest. For each of the three candidate values, by comparing it with the neighborhood values ​​within a preset local range, it is determined whether the candidate value is the largest within the preset local range. If the candidate value is the largest within the preset local range, the detected local maxima are sorted to obtain a sorting result. Based on the sorting results, the number of sound sources is obtained, and coordinate transformation technology is used to map the row and column indices of local maxima to actual coordinates to obtain the sound source locations.

Images