A Debris Flow Infrasound Recognition Method Based on Mel Spectrograms and Lightweight Neural Networks

Abstract

This invention discloses a debris flow infrasound recognition method based on Mel spectrograms and a lightweight neural network, belonging to the field of image data processing. The method includes the following steps: collecting infrasound signals generated by debris flows and other events to generate a Mel spectrogram dataset; constructing a channel-aligned fusion bridge; constructing an improved MBConv block; constructing a debris flow infrasound recognition network based on the channel-aligned fusion bridge and the improved MBConv block; training a debris flow infrasound recognition model using the Mel spectrogram dataset, and then using it to identify the infrasound signals to be identified. This invention combines the characteristics of debris flow infrasound, constructing infrasound Mel spectrograms from samples in the dataset, and designing a debris flow infrasound recognition network based on a multi-stage lightweight feature extraction module to classify and recognize the infrasound Mel spectrograms. This effectively reduces computational complexity and the number of parameters, making it suitable for low-power devices, and improves classification accuracy, making it particularly suitable for real-time debris flow early warning scenarios.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a method for identifying infrasound in debris flows based on Mel spectrograms and lightweight neural networks. Background Technology

[0002] Debris flows have complex causes, cause enormous damage once they occur, and are difficult to manage comprehensively. Therefore, effective early warning of debris flows has become a research hotspot and challenge. Methods for debris flow identification mainly include the mud level method, ground acoustic method, video monitoring, and infrasound method. The infrasound method captures infrasound waves from debris flows in the air. Its advantage lies in the fact that infrasound travels faster in the air than the movement speed of debris flows (the propagation speed of infrasound waves in debris flows is tens of times faster) and has strong penetrating power (the dominant frequency range is concentrated in the 0-20Hz range). Therefore, the infrasound method has become a major research focus in debris flow monitoring and early warning both domestically and internationally. Furthermore, with the application of deep learning to acoustic signal identification, more identification methods combining infrasound signals with deep learning have emerged.

[0003] Since infrasound audio is one-dimensional signal data, it is impossible to obtain the temporal and spatial distribution characteristics of the signal. Therefore, infrasound audio is converted into a time-spectrum image before feature extraction and recognition. For example, in 2023, Yuan Li et al. proposed converting the infrasound audio of debris flows into a time-spectrum image after noise reduction and filtering, using the AlexNet network for feature extraction, and then using machine learning methods for classification and recognition. This method has achieved certain results in the classification of debris flow infrasound signals. However, the dominant frequency characteristics of debris flow infrasound are concentrated in the 0-20Hz range, while the frequency resolution of the time-spectrum image varies in different frequency bands (lower frequency bands have lower frequency resolution). This leads to the loss of some details in the lower frequency bands of the debris flow infrasound signal, resulting in insufficient robustness and extraction of important features when processing data in noisy environments. Furthermore, the recognition process often uses traditional machine learning methods such as K-nearest neighbors, random forests, decision trees, and support vector machines, which lead to high latency and high computational resource consumption in real-time debris flow monitoring. Therefore, they are not suitable for real-time prediction of debris flow infrasound recognition.

[0004] In 2024, Hu Meichen et al. constructed a debris flow recognition model based on the dual-stream neural network and TSN model framework, using ResNet-50 as the underlying architecture. Experimental results showed that the model improved the accuracy of debris flow recognition. However, for specific scenarios like debris flow recognition, it is usually used under conditions of limited hardware resources, especially in edge devices or embedded systems. Due to the high complexity of deep learning networks such as ResNet-50, they consume significant computational and memory resources. Deploying them on resource-constrained devices may prevent real-time identification and processing of infrasound signals, which is a disadvantage in scenarios requiring rapid response to disasters such as debris flows. Furthermore, the complex deep network model increases data transmission latency, potentially reducing early warning efficiency.

[0005] Therefore, there is a need for a debris flow infrasound recognition method that can be adapted to low-power devices by reducing computational complexity and the number of parameters without significantly reducing the model recognition accuracy. This is crucial in debris flow early warning and helps to respond in a timely manner at an early stage.

[0006] Definitions:

[0007] The MBConv (Mobile Inverted Bottleneck Convolution) block was first proposed in MobileNetV2 and later widely used in the EfficientNet series. It consists of a main computation branch and a residual branch. The main computation branch sequentially performs 1×1 dimensionality upscaling, depthwise separable convolution, SE channel attention, and 1×1 dimensionality reduction on the input features. The residual branch is only activated when stride=1 and the number of input channels equals the number of output channels; it is used to directly add the input features to the output of the main computation branch. stride is the step size.

[0008] The Ghost Module is a plug-and-play lightweight convolutional unit designed to generate feature maps that are approximately equivalent to ordinary convolutions with fewer FLOPs and parameters. The Ghost Module consists of a primary convolutional branch (main convolution) and a cheap operation branch (cheap operation convolution). The main convolutional branch is used to generate the intrinsic feature maps, while the cheap operation convolution is used to generate the ghost feature maps. The ratio is the ratio of the number of channels in the intrinsic feature maps to the number of channels in the ghost feature maps. Assuming ratio = s, the main convolutional branch generates m intrinsic feature maps from the input feature map, forming a set of convolutional feature maps Y. prim The cheap convolution derives s-1 ghost feature maps from each intrinsic feature map, for a total of m(s-1) ghost feature maps, forming the cheap feature map set Y. cheap Then, concatenate the two along the channel dimension to obtain the output output=[Y]. prim Ycheap ].

[0009] The SE module, or Squeeze-and-Excitation Module, learns the dependencies between channels through two steps: compression (Squeeze) and excitation (Excitation). This strengthens the weights of key feature channels, suppresses unimportant feature channels, and improves the model's feature extraction capabilities.

[0010] The ECA module, or Efficient Channel Attention Module, aims to achieve an efficient channel attention mechanism with lower computational complexity. Summary of the Invention

[0011] The purpose of this invention is to provide a debris flow infrasound recognition method based on Mel spectrograms and lightweight neural networks that solves the above-mentioned problems, has higher recognition capability in the extraction of infrasound signal features of debris flows, can significantly reduce the number of model parameters and computational load, and thus achieves faster response speed and more accurate recognition results even when running in resource-constrained environments.

[0012] To achieve the above objectives, the technical solution adopted by this invention is as follows: a debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks, comprising the following steps:

[0013] S1 generates the Mel spectrogram dataset D, including S11~S13;

[0014] S11, collects infrasound signals generated by debris flows and other events at the scene;

[0015] S12, for each infrasound signal, preprocess it first and then convert it into an infrasound Mel spectrum;

[0016] S13, Classify the infrasound Mel spectrograms to obtain samples. All samples constitute the Mel spectrogram dataset D, where the categories include debris flow and noise.

[0017] S2, construct a channel alignment and fusion bridge;

[0018] It includes a channel-aligned convolutional layer, a basal feature extraction layer, a control signal generation layer, a channel attention layer, and a residual mixing and fusion layer arranged sequentially.

[0019] The channel-aligned convolutional layer is used to perform convolution operations on the input features with a stride of 1 or 2, aligning the channels to a preset number of channels, and then normalizing the process to obtain the aligned features.

[0020] The base feature extraction layer is used to identify the stride of the channel-aligned convolutional layer. When the stride is 2, the alignment features are downsampled to obtain the base feature X_base. When the stride is 1, the alignment features are used as the base feature X_base.

[0021] The control signal generation layer is used to perform global average pooling and Sigmoid function operations on X_base to generate control signal α;

[0022] The channel attention layer is used to perform channel attention on X_base and generate channel attention features Z;

[0023] The residual fusion layer is used to generate the output feature Y of the channel-aligned fusion bridge according to the formula Y=X_base+γ(Z-X_base), where γ is the residual fusion factor;

[0024] S3, construct an improved MBConv block;

[0025] An improved MBConv block is obtained by adding a lightweight gating unit at the front end of the main computation branch of the MBConv block. The lightweight gating unit is used to control the input features of the input main computation branch. Generate a gated signal β and connect it to the gated signal β. The gated features are obtained by multiplying element by element and then fed into the main computation branch.

[0026] S4, construct a debris flow infrasound recognition network, including a convolutional leader layer, a multi-stage lightweight feature extraction module, and a classification output layer;

[0027] The convolutional leader layer is used to perform convolution and batch normalization operations on the samples sequentially to obtain the first feature F1.

[0028] The multi-stage lightweight feature extraction module is used to input F1 and output the second feature F2. It consists of 10 layers from front to back, where the 2nd, 8th and 9th layers are Ghost modules, the 3rd and 10th layers are improved MBConv blocks, and the remaining layers are MBConv blocks.

[0029] When the layer above the Ghost module is an MBConv block, the MBConv block and the Ghost module are labeled A1 and A2 respectively. They are connected by a channel alignment fusion bridge. The output features of A1 are used as the input features of A2. The control signal α of A1 is used to adjust the main convolutional feature map set Y in the Ghost module. prim and cheap feature map set Y cheap The proportion of [a certain number of layers] is such that the remaining adjacent layers are directly connected;

[0030] The classification output layer is used as input to F2 and outputs the predicted category of the corresponding sample;

[0031] S5. Train the debris flow infrasound recognition network using the Mel spectrogram dataset D to obtain the debris flow infrasound recognition model.

[0032] S6: Obtain the infrasound signal to be identified, convert it into an infrasound Mel spectrum according to S12, and send it into the debris flow infrasound identification model to obtain the predicted output.

[0033] Preferably, in S11, the infrasound signal is acquired by an infrasound acquisition instrument, and the other events include water flow, insect chirping, bird calls, airplanes, explosions, flowing water, lightning, cars, and / or wind.

[0034] Preferably, the preprocessing in S12 involves cutting the infrasound signal into infrasound segments of a preset duration, filtering each infrasound segment to retain only the signal of 3-20Hz, and then using wavelet soft thresholding to denoise the filtered infrasound segments to obtain the preprocessed infrasound signal.

[0035] As a preferred embodiment, the conversion to infrasonic Mel spectrum in S12 is specifically as follows:

[0036] The preprocessed infrasound signal is processed by analog-to-digital conversion, framing, windowing, and short-time Fourier transform to obtain the spectrum and power spectrum of each frame. The power spectrum of each frame is then processed by a Mel filter bank, logarithmic transformation, and normalization to generate the Mel spectrum of each frame. Finally, the Mel spectra of all frames are stacked along the time axis to obtain the Mel spectrogram.

[0037] Preferably, the convolutional leader layer includes a convolutional layer and a batch normalization layer. The convolutional layer is based on convolutional operations on samples using the Conv2dStaticSamePadding function, wherein the convolutional kernel size is 3×3, the stride is 2, the number of input channels is 3, and the number of output channels is 32. The features are then processed by a batch normalization layer to obtain... The first feature F1, where H and W are the height and width of the sample, respectively. This represents the number of channels in F1.

[0038] Preferably, the lightweight gating unit includes a global average pooling layer, a first two-dimensional convolutional layer, a SiLU function layer, a second two-dimensional convolutional layer, and a Sigmoid function layer arranged sequentially.

[0039] Features of input lightweight gating units The signal is sequentially processed through a global average pooling layer, a first two-dimensional convolutional layer for dimensionality reduction, a SiLU function layer for nonlinear activation, a second two-dimensional convolutional layer for dimensionality increase, and a Sigmoid function layer for normalization to the range [0,1] to obtain the gated signal β. Then, β is compared with... The gating feature is obtained by multiplying each element.

[0040] Preferably, the control signal α of A1 is used to adjust the main convolutional feature map set Y in the Ghost module. prim and cheap feature map set Y cheap The proportions are as follows:

[0041] The control signal α is virtually extended to Y via broadcast. prim The internal feature maps are of the same size, resulting in signal α'. These are then concatenated according to the following formula to obtain the output feature of A2: output = Concat[α'⊙Y]. prim ,(1-α')⊙Y cheap ], Concat[∙] concatenates along the channel dimension, and ⊙ multiplies element by element.

[0042] Preferably, the improved MBConv block includes a lightweight gating unit, a main computation branch, and a residual branch;

[0043] The lightweight gating unit is used to improve the input characteristics of the MBConv block based on the input. Generate corresponding gating features ;

[0044] The main computation branch includes, in sequence, an extended convolutional layer, a channel-wise convolutional layer, a channel attention module, and a linear projection convolutional layer. The extended convolutional layer, channel-wise convolutional layer, channel attention module, and linear projection convolutional layer sequentially... Perform 1×1 up-dimensional convolution, channel-wise convolution, and then perform channel attention followed by 1×1 down-dimensional convolution to obtain the output of the improved main computation branch. ;

[0045] The residual branch is used to, when the step size is 1, convert... and Identity residual connection as output of improved MBConv block When the step size is 2, As .

[0046] Preferably, the channel attention module in the MBConv block is an ECA module.

[0047] The present invention is based on the following approach: converting the infrasound signal of debris flow into an infrasound Mel spectrum to form a dataset D, and then constructing a debris flow infrasound identification network. The debris flow infrasound identification model is trained using dataset D to classify and identify the infrasound signals in the debris flow area.

[0048] Regarding infrasound Mel spectra: Since energy changes over time are observed in Mel spectra, and based on the spectral characteristics of debris flows, this invention finds that after converting the infrasound of noise and the infrasound of debris flows into infrasound Mel spectra, there are significant differences between the two, making them more distinguishable.

[0049] Regarding the debris flow infrasound recognition network: the main improvements include a channel alignment fusion bridge, a lightweight gating unit, and a multi-stage lightweight feature extraction module.

[0050] (1) Regarding the channel alignment fusion bridge:

[0051] As the feature flow moves from the MBConv block to the Ghost module, a channel-aligned fusion bridge is established between them. This channel-aligned convolutional layer uses convolution and batch normalization operations to align the features output from the MBConv block with the Ghost module's entry scale in terms of channel dimension and statistical distribution, reducing distribution drift and covariate offset. The statistical distribution consists of the mean and variance of the feature map in each channel (typically obtained by the moving average of the preceding batch normalization layer). The base feature extraction layer generates base features X_base based on the stride of the convolution operation in the channel-aligned convolutional layer. The control signal generation layer and the channel attention layer generate control signals α and channel attention features Z based on X_base, respectively. The control signal α adjusts the ratio of the main convolution and the cheap convolution. It allocates the proportion within the group according to the upstream features. α is virtually expanded to the same shape as the features through broadcasting to obtain signal α'. The main convolution branch features are scaled by α', and the cheap convolution branch features are scaled by 1-α'. Then, they are concatenated along the channel dimension to obtain the output. The scaling effect is to change the contribution ratio of the main convolution branch and the cheap convolution branch from a fixed value to a learnable and controllable dynamic weight, while maintaining the stability of the feature tensor shape and the computation graph. Without changing the number and arrangement of the Ghost module's output channels, the model adaptively determines the contribution size between the main convolution branch and the cheap convolution branch. When α' is 1, the output comes entirely from the main convolution branch; when α' is 0, the output comes entirely from the cheap convolution branch. Generally, the two contributions are adjusted according to α':(1-α'). The role of channel attention features Z is to assign weights to each channel, allowing the model to automatically learn which channels should be strengthened or suppressed. Compared to the original base features, channel attention features Z semantically highlight key features and suppress redundant features, improving the quality of information flow. X_base and channel attention features Z are combined through a residual mixing fusion layer to generate the output feature Y of the channel-aligned fusion bridge. During residual mixing, γ is introduced for interpolation fusion; γ is a residual fusion factor that gradually introduces enhancement information during training. Y retains the original structural information of X_base while gradually incorporating the attention weighting effect of channel attention features Z, making the network more discriminative and generalizing, and allowing the downstream Ghost module to receive more informative input.

[0052] (2) Regarding the improved MBConv block: When the feature flow moves from the Ghost module to the MBConv block, the improved MBConv block is used, characterized by the addition of lightweight gating units to the existing MBConv block. Although the output features of the Ghost module have expanded the number of channels, they still contain some noise information. If directly input into the MBConv block, there will be problems of wasted computational resources and insufficient feature discrimination. Therefore, before entering the main branch of MBConv, it is necessary to perform lightweight channel filtering and dynamic modulation on the input features to highlight semantically significant channels, suppress noise and redundancy, and improve the modeling efficiency of the MBConv block.

[0053] The global average pooling layer performs a global average of the input features over the spatial dimension H×W to obtain the global statistics for each channel, forming a channel description vector. This operation compresses high-dimensional spatial features into one-dimensional channel features, preserving channel-level global semantic information and reducing computational complexity.

[0054] The channel description vector is linearly mapped (1×1 convolution) in the first two-dimensional convolutional layer to achieve channel dimensionality reduction. This reduces the number of channels, the number of parameters and the amount of computation, while also compressing information and extracting features.

[0055] The SiLU function layer introduces nonlinearity after the output of the previous convolutional layer, retaining some negative gradients, which can provide smoother gradient flow, avoid premature loss of feature information, and improve gating fitting ability and stability.

[0056] The second 2D convolutional layer maps the reduced intermediate channel count back to the original input channel count, restoring it to the same dimension as the input. It generates a corresponding weight vector U for each channel, ensuring the gating signal is dimensionally aligned with the original input. The Sigmoid function layer compresses the output of the previous convolutional layer to the [0,1] interval, according to the formula... The gate signal β is obtained. The gate signal β obtained from the Sigmoid function is then compared with the input feature... Element-wise multiplication yields the backbone input features of the MBConv block. The input is fed into the extended convolutional layer. This operation enables weighted modulation of each channel, enhancing important channels and reducing redundant and noisy channels. The residual branch of the MBConv block, when satisfying a stride of 1 and the number of input channels equals the number of output channels, modulates the unmodulated input features... Add back the output. The existence of the identity residual ensures that information is not lost and is transmitted stably. Before entering the main branch of the MBConv block, the lightweight gating unit dynamically modulates the input features with extremely low computational cost, highlighting semantically significant channels and suppressing redundant and noisy channels in the Ghost module output, concentrating computational power on the high-value subspace. When processing these high-value features, the convolutional layers in the MBConv block more efficiently model and discriminate patterns, improving feature representation and generalization capabilities.

[0057] (3) Regarding the multi-stage lightweight feature extraction module: This module is constructed based on the channel alignment fusion bridge, lightweight gating unit, and the existing MBConv block and Ghost module. The output size of the convolutional leader structure is 112×112×32. Considering that the energy of the debris flow infrasound signal in the Mel spectrum is mainly concentrated in the low frequency band, different module types are used in different stages to balance feature extraction accuracy and lightweight requirements: Initial layer (convolutional leader layer, multi-stage lightweight feature extraction module layer 1): Focuses on basic patterns such as low frequency energy distribution, requiring high-precision convolution to capture subtle changes and provide reliable input for subsequent deep analysis, so the MBConv block is retained. Intermediate layer (layers 2-9): Extracts texture features, with low dependence on high-precision convolution. To reduce the risk of computation and overfitting, GhostModule is used at specific locations. Later layer (layer 10 and classification output layer): Extracts high-level abstract features, which directly affect classification accuracy. The MBConv block and subsequent high-dimensional convolution are retained to ensure expressive power.

[0058] The Ghost module consists of two parts: the main convolutional branch is a 1×1 convolution, and the low-cost convolutional branch is a 3×3 depthwise separable convolution. The number of parameters and computational cost of the Ghost module can be calculated from this. The number of parameters in the main branch convolution is Params1 = (C...). in ⋅m⋅K 2 ), C in Here, m is the number of input channels, m is the number of principal feature maps, and K is the size of the principal branch convolution kernel. The parameter count for a low-cost convolution branch is Params2 = m(r-1)k. 2 Where r is the expansion ratio and k is the inexpensive kernel size. The final total number of parameters can be obtained as Params = (C... in ⋅m⋅K 2 )+ m(r-1)k 2 The formula for the computational quantity is: H out and W out These are the height and width of the feature map output by the main convolutional branch, respectively. and These are the height and width of the feature map output by the inexpensive convolution branch, respectively.

[0059] The MBConv block contains expanded convolution (1×1), channel-wise convolution (k×k), and projective convolution (1×1). Therefore, the formula for calculating the number of parameters is: t is the expansion factor. The formula for computational complexity is: H and W are the height and width of the input feature map. and This is used to output the height and width of the feature map.

[0060] Taking the second layer of multi-stage lightweight feature extraction as an example, the input feature map size is 112×112×16. Under the same conditions, the total computation of the Ghost module is 7.41 MMACs (where the first M represents 10). 6 The Ghost module (MACs, short for Multiply-Accumulate Operations) has 1.67k parameters. The MBConv module has a total computational cost of 60.81 MMACs and 13.97k parameters. The comparison shows that the Ghost module significantly reduces both computational and parameter count compared to the MBConv module, substantially lowering the model's complexity. Therefore, in the intermediate stages of the model, i.e., at levels with lower dependence on high-precision convolutions, the Ghost module can safely replace some of the more computationally intensive MBConv blocks, achieving a lightweight model while maintaining a certain improvement in overall recognition accuracy and precision.

[0061] (4) Replace the SE module in the MBConv block with the ECA module. The SE module has two fully connected layers, which increases the number of parameters and computation. The ECA module does not require two fully connected layers, significantly reducing the number of parameters and computation. It avoids the information bottleneck of dimensionality increase to dimensionality decrease in SE, making information flow smoother, more lightweight, and more suitable for the input of spectrum type images.

[0062] Compared with the prior art, the advantages of the present invention are as follows:

[0063] (1) By introducing a channel alignment fusion bridge before the Ghost module, the model can highlight the discriminative information in the main features, thereby improving the relevance and utilization of the extended features of the Ghost module, and making the features generated by the inexpensive convolutional branch maintain stronger semantic consistency with the main branch convolution. The whole process significantly improves the expressive power and discriminability of the features, while maintaining the original lightweight advantage of the Ghost module.

[0064] (2) An improved MBConv block was designed, in which a lightweight gating unit was added to the beginning of the MBConv block. This unit only relies on global average pooling and two 1×1 convolutions, resulting in extremely low computational overhead. By assigning weights to each channel through the gating signal β, the input features are adaptively adjusted, effectively suppressing invalid or redundant channels. This allows the MBConv block to no longer process all features, but instead concentrates computing power on the high-value features that have been filtered and modulated. Thus, while ensuring computational efficiency, the discriminative and generalization capabilities of the module are enhanced.

[0065] (3) Based on the channel alignment fusion bridge and the improved MBConv block, a multi-stage lightweight feature extraction module is designed, which can effectively reduce the computational complexity and the number of parameters to adapt to low-power devices, especially suitable for debris flow early warning scenarios.

[0066] (4) Combining the characteristics of debris flow infrasound, the data samples are constructed into infrasound Mel spectra, and a debris flow infrasound recognition network is designed based on a multi-stage lightweight feature extraction module to classify and recognize the infrasound Mel spectra. This not only improves classification efficiency but also improves accuracy.

[0067] (5) Replacing the SE module with ECA reduces the computational cost of the original model by approximately 13.8%, allowing the improved MBConv module to maintain lower computational complexity. The improved MBConv module can effectively preserve the neighborhood correlation of spectral images and has better adaptability and discrimination ability in spectral tasks.

[0068] In summary, the model and method of this invention have high classification accuracy, low computational complexity, and a small number of parameters, making them particularly suitable for real-time prediction of debris flows. Attached Figure Description

[0069] Figure 1 This is a flowchart of the present invention;

[0070] Figure 2 Diagram of the infrasound identification network structure for debris flows;

[0071] Figure 3 Diagram of the channel alignment and fusion bridge structure;

[0072] Figure 4 This is a structural diagram of a lightweight gating unit;

[0073] Figure 5 To improve the MBConv block structure diagram;

[0074] Figure 6 The infrasound Mel spectrum corresponding to a debris flow event;

[0075] Figure 7 This is the infrasound mellogram corresponding to an explosion event. Detailed Implementation

[0076] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0077] Example 1: See Figures 1-5 ,

[0078] A debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks includes the following steps:

[0079] S1 generates the Mel spectrogram dataset D, including S11~S13;

[0080] S11, collects infrasound signals generated by debris flows and other events at the scene;

[0081] S12, for each infrasound signal, preprocess it first and then convert it into an infrasound Mel spectrum;

[0082] S13, Classify the infrasound Mel spectrograms to obtain samples. All samples constitute the Mel spectrogram dataset D, where the categories include debris flow and noise.

[0083] S2, construct a channel alignment and fusion bridge;

[0084] It includes a channel-aligned convolutional layer, a basal feature extraction layer, a control signal generation layer, a channel attention layer, and a residual mixing and fusion layer arranged sequentially.

[0085] The channel-aligned convolutional layer is used to perform convolution operations on the input features with a stride of 1 or 2, aligning the channels to a preset number of channels, and then normalizing the process to obtain the aligned features.

[0086] The base feature extraction layer is used to identify the stride of the channel-aligned convolutional layer. When the stride is 2, the alignment features are downsampled to obtain the base feature X_base. When the stride is 1, the alignment features are used as the base feature X_base.

[0087] The control signal generation layer is used to perform global average pooling and Sigmoid function operations on X_base to generate control signal α;

[0088] The channel attention layer is used to perform channel attention on X_base and generate channel attention features Z;

[0089] The residual fusion layer is used to generate the output feature Y of the channel-aligned fusion bridge according to the formula Y=X_base+γ(Z-X_base), where γ is the residual fusion factor;

[0090] S3, construct an improved MBConv block;

[0091] An improved MBConv block is obtained by adding a lightweight gating unit at the front end of the main computation branch of the MBConv block. The lightweight gating unit is used to control the input features of the input main computation branch. Generate a gated signal β and connect it to the gated signal β. The gated features are obtained by multiplying element by element and then fed into the main computation branch.

[0092] S4, construct a debris flow infrasound recognition network, including a convolutional leader layer, a multi-stage lightweight feature extraction module, and a classification output layer;

[0093] The convolutional leader layer is used to perform convolution and batch normalization operations on the samples sequentially to obtain the first feature F1.

[0094] The multi-stage lightweight feature extraction module is used to input F1 and output the second feature F2. It consists of 10 layers from front to back, where the 2nd, 8th and 9th layers are Ghost modules, the 3rd and 10th layers are improved MBConv blocks, and the remaining layers are MBConv blocks.

[0095] When the layer above the Ghost module is an MBConv block, the MBConv block and the Ghost module are labeled A1 and A2 respectively. They are connected by a channel alignment fusion bridge. The output features of A1 are used as the input features of A2. The control signal α of A1 is used to adjust the main convolutional feature map set Y in the Ghost module. prim and cheap feature map set Y cheap The proportion of [a certain number of layers] is such that the remaining adjacent layers are directly connected;

[0096] The classification output layer is used as input to F2 and outputs the predicted category of the corresponding sample;

[0097] S5. Train the debris flow infrasound recognition network using the Mel spectrogram dataset D to obtain the debris flow infrasound recognition model.

[0098] S6: Obtain the infrasound signal to be identified, convert it into an infrasound Mel spectrum according to S12, and send it into the debris flow infrasound identification model to obtain the predicted output.

[0099] In S11, the infrasound signal is acquired by an infrasound acquisition instrument, and the other events include water flow, insect chirping, bird calls, airplanes, explosions, flowing water, lightning, cars, and / or wind.

[0100] In S12, the preprocessing involves cutting the infrasound signal into infrasound segments of a preset duration, filtering each infrasound segment to retain only the signal in the 3-20Hz range, and then using wavelet soft thresholding to denoise the filtered infrasound segments to obtain the preprocessed infrasound signal.

[0101] The conversion from S12 to infrasonic Mel spectrum is as follows:

[0102] The preprocessed infrasound signal is processed by analog-to-digital conversion, framing, windowing, and short-time Fourier transform to obtain the spectrum and power spectrum of each frame. The power spectrum of each frame is then processed by a Mel filter bank, logarithmic transformation, and normalization to generate the Mel spectrum of each frame. Finally, the Mel spectra of all frames are stacked along the time axis to obtain the Mel spectrogram.

[0103] The convolutional leader layer includes a convolutional layer and a batch normalization layer. The convolutional layer is based on convolutional operations on samples using the Conv2dStaticSamePadding function, where the kernel size is 3×3, the stride is 2, the number of input channels is 3, and the number of output channels is 32. The features are then processed by a batch normalization layer to obtain... The first feature F1, where H and W are the height and width of the sample, respectively. This represents the number of channels in F1.

[0104] The lightweight gating unit includes a global average pooling layer, a first two-dimensional convolutional layer, a SiLU function layer, a second two-dimensional convolutional layer, and a Sigmoid function layer arranged sequentially.

[0105] Features of input lightweight gating units The signal is sequentially processed through a global average pooling layer, a first two-dimensional convolutional layer for dimensionality reduction, a SiLU function layer for nonlinear activation, a second two-dimensional convolutional layer for dimensionality increase, and a Sigmoid function layer for normalization to the range [0,1] to obtain the gated signal β. Then, β is compared with... The gating feature is obtained by multiplying each element.

[0106] The control signal α of A1 is used to adjust the main convolutional feature map set Y in the Ghost module. prim and cheap feature map set Y cheap The proportions are as follows:

[0107] The control signal α is virtually extended to Y via broadcast. prim The internal feature maps are of the same size, resulting in signal α'. These are then concatenated according to the following formula to obtain the output feature of A2: output = Concat[α'⊙Y]. prim ,(1-α')⊙Y cheap ], Concat[∙] concatenates along the channel dimension, and ⊙ multiplies element by element.

[0108] The improved MBConv block includes a lightweight gating unit, a main computation branch, and a residual branch;

[0109] The lightweight gating unit is used to improve the input characteristics of the MBConv block based on the input. Generate corresponding gating features ;

[0110] The main computation branch includes, in sequence, an extended convolutional layer, a channel-wise convolutional layer, a channel attention module, and a linear projection convolutional layer. The extended convolutional layer, channel-wise convolutional layer, channel attention module, and linear projection convolutional layer sequentially... Perform 1×1 up-dimensional convolution, channel-wise convolution, and then perform channel attention followed by 1×1 down-dimensional convolution to obtain the output of the improved main computation branch. ;

[0111] The residual branch is used to, when the step size is 1, convert... and Identity residual connection as output of improved MBConv block When the step size is 2, As .

[0112] The channel attention module in the MBConv block is an ECA module.

[0113] Example 2: See Figures 1-7 Based on Example 1, the operation method is given in more detail:

[0114] The preprocessing described in S12 involves: first, data slicing, dividing the infrasound signal into 6-second segments to expand the dataset; then, filtering using a Chebyshev Type II filter to remove signals with frequencies greater than or equal to 20Hz and less than or equal to 3Hz, retaining only signals between 3 and 20Hz; zero-phase-shift filtering is used to avoid phase distortion; finally, wavelet soft thresholding is applied for noise reduction, and multi-scale analysis is used to preserve key features and reduce noise in the debris flow infrasound. The preprocessed infrasound signal is then obtained.

[0115] The conversion to infrasound Mel-spectrum described in S12 is as follows: Set the sampling rate f... s The parameters were: 100Hz, FFT window size of 128 or 256, frame length of 0.32s, frame shift of 0.02s, and 80 Mel filter banks. Since a larger FFT window results in higher resolution but also increased computation, 128 was chosen to ensure high Mel spectrum resolution and relatively low computational cost. For each preprocessed infrasound signal, analog-to-digital conversion was performed, followed by framing to divide the infrasound signal into very short segments, each of which could be approximated as a stationary signal. A Hamming window was applied to each frame of the audio to ensure smooth transitions at signal boundaries. A short-time Fourier transform was then used to obtain the spectrum and power spectrum of each frame. The power spectrum of each frame was mapped to the Mel scale using Mel filter banks, followed by logarithmic transformation and normalization to generate the Mel spectrum of each frame. The Mel spectra of all frames were then stacked along the time axis to obtain the infrasound Mel spectrum. The infrasound Mel spectra corresponding to debris flow events and noise events are significantly different; see [link to relevant documentation]. Figure 6 and Figure 7 . Figure 6 There are distinct and continuous bright areas in the 0-20 Hz range, representing the sustained energy release of debris flow signals. Figure 7 The corresponding region is intermittent, and there are many gaps in the energy distribution, indicating that the noise signal is usually short-lived and transient, and the energy fluctuates irregularly.

[0116] Example 3: To illustrate the effectiveness of the present invention, we designed a comparative experiment:

[0117] Dataset: A self-made dataset containing 1207 infrasound Mel spectra, including 607 infrasound Mel spectra of debris flows and 600 infrasound Mel spectra of other noises, divided into training and testing sets.

[0118] Experimental environment: The hardware environment is an NVIDIA RTX 4060 GPU, and the software environment is PyTorch: 1.12.1 + cu116.

[0119] Experimental Groups: The experimental group used the debris flow infrasound recognition model of this invention, while the control group used the classic lightweight deep learning recognition model EfficientNet-b0. In the debris flow infrasound recognition model of this invention, the convolutional layers in the MBConv block use the SiLU activation function, the convolutional layers in the Ghost module use the ReLU activation function, and the pooling layers use average pooling. During training, the cross-entropy loss function and the Adam optimization algorithm are used, with an initial learning rate of 0.0001 and the learning rate scheduler being ReduceLROnPlateau, which is dynamically adjusted during training.

[0120] Tables 1 and 2 are obtained by comparing the model of this invention with EfficientNet-b0;

[0121] Table 1. Performance metrics for different methods:

[0122] Model Name Accuracy Recall rate F1 score accuracy This invention model 96.00% 97.56% 96.77% 96.69% EfficientNet-b0 92.97% 96.74% 94.82% 94.63%

[0123] Table 2. Parameter quantity, computational cost, and average inference time for different methods

[0124] Model Name Parameters computational load Average inference time This invention model 2.68M 178.8MMac 1.2240s EfficientNet-b0 5.29M 401.67MMac 1.4553s

[0125] Data Analysis: The model proposed in this invention demonstrates excellent performance in precision, recall, F1 score, and accuracy, achieving accuracy and recall of 96.69% and 97.56% respectively, significantly outperforming the comparative model EfficientNet-b0. This is attributed to the effective collaboration between the Ghost module and MBConv. Regarding parameter and computational costs, the proposed model has 2.68M parameters and 178.8MMac FLOPs, significantly lower than the original EfficientNet-b0's 5.29M parameters and 401.67MMac. It also shows an improvement in average inference time, indicating that the improved model method of this invention significantly reduces computational costs while maintaining high performance, enabling effective infrasound image recognition tasks.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying infrasound in debris flows based on Mel spectrograms and lightweight neural networks, characterized in that, Includes the following steps: S1 generates the Mel spectrogram dataset D, including S11~S13; S11, collects infrasound signals generated by debris flows and other events at the scene; S12, for each infrasound signal, preprocess it first and then convert it into an infrasound Mel spectrum; S13, Classify the infrasound Mel spectrograms to obtain samples. All samples constitute the Mel spectrogram dataset D, where the categories include debris flow and noise. S2, construct a channel alignment and fusion bridge; It includes a channel-aligned convolutional layer, a basal feature extraction layer, a control signal generation layer, a channel attention layer, and a residual mixing and fusion layer arranged sequentially. The channel-aligned convolutional layer is used to perform convolution operations on the input features with a stride of 1 or 2, aligning the channels to a preset number of channels, and then normalizing the process to obtain the aligned features. The base feature extraction layer is used to identify the stride of the channel-aligned convolutional layer. When the stride is 2, the alignment features are downsampled to obtain the base feature X_base. When the stride is 1, the alignment features are used as the base feature X_base. The control signal generation layer is used to perform global average pooling and Sigmoid function operations on X_base to generate control signal α; The channel attention layer is used to perform channel attention on X_base and generate channel attention features Z; The residual fusion layer is used to generate the output feature Y of the channel-aligned fusion bridge according to the formula Y=X_base+γ(Z-X_base), where γ is the residual fusion factor; S3, construct an improved MBConv block; An improved MBConv block is obtained by adding a lightweight gating unit at the front end of the main computation branch of the MBConv block. The lightweight gating unit is used to control the input features of the input main computation branch. Generate a gated signal β and connect it to the gated signal β. The gated features are obtained by multiplying element by element and then fed into the main computation branch. S4, construct a debris flow infrasound recognition network, including a convolutional leader layer, a multi-stage lightweight feature extraction module, and a classification output layer; The convolutional leader layer is used to perform convolution and batch normalization operations on the samples sequentially to obtain the first feature F1. The multi-stage lightweight feature extraction module is used to input F1 and output the second feature F2. It consists of 10 layers from front to back, where the 2nd, 8th and 9th layers are Ghost modules, the 3rd and 10th layers are improved MBConv blocks, and the remaining layers are MBConv blocks. When the layer above the Ghost module is an MBConv block, the MBConv block and the Ghost module are labeled A1 and A2 respectively. They are connected by a channel alignment fusion bridge. The output features of A1 are used as the input features of A2. The control signal α of A1 is used to adjust the main convolutional feature map set Y in the Ghost module. prim and cheap feature map set Y cheap The proportion of [a certain number of layers] is such that the remaining adjacent layers are directly connected; The classification output layer is used as input to F2 and outputs the predicted category of the corresponding sample; S5. Train the debris flow infrasound recognition network using the Mel spectrogram dataset D to obtain the debris flow infrasound recognition model. S6: Obtain the infrasound signal to be identified, convert it into an infrasound Mel spectrum according to S12, and send it into the debris flow infrasound identification model to obtain the predicted output.

2. The debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks according to claim 1, characterized in that, In S11, the infrasound signal is acquired by an infrasound acquisition instrument, and the other events include water flow, insect chirping, bird calls, airplanes, explosions, flowing water, lightning, cars, and / or wind.

3. The debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks according to claim 1, characterized in that, In S12, the preprocessing involves cutting the infrasound signal into infrasound segments of a preset duration, filtering each infrasound segment to retain only the signal in the 3-20Hz range, and then using wavelet soft thresholding to denoise the filtered infrasound segments to obtain the preprocessed infrasound signal.

4. The debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks according to claim 1, characterized in that, The conversion from S12 to infrasonic Mel spectrum is as follows: The preprocessed infrasound signal is processed by analog-to-digital conversion, framing, windowing, and short-time Fourier transform to obtain the spectrum and power spectrum of each frame. The power spectrum of each frame is then processed by a Mel filter bank, logarithmic transformation, and normalization to generate the Mel spectrum of each frame. Finally, the Mel spectra of all frames are stacked along the time axis to obtain the Mel spectrogram.

5. The debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks according to claim 1, characterized in that, The convolutional leader layer includes a convolutional layer and a batch normalization layer. The convolutional layer is based on convolutional operations on samples using the Conv2dStaticSamePadding function, where the kernel size is 3×3, the stride is 2, the number of input channels is 3, and the number of output channels is 32. The features are then processed by a batch normalization layer to obtain... The first feature F1, where H and W are the height and width of the sample, respectively. This represents the number of channels in F1.

6. The debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks according to claim 1, characterized in that, The lightweight gating unit includes a global average pooling layer, a first two-dimensional convolutional layer, a SiLU function layer, a second two-dimensional convolutional layer, and a Sigmoid function layer arranged sequentially. Features of input lightweight gating units The signal is sequentially processed through a global average pooling layer, a first two-dimensional convolutional layer for dimensionality reduction, a SiLU function layer for nonlinear activation, a second two-dimensional convolutional layer for dimensionality increase, and a Sigmoid function layer for normalization to the range [0,1] to obtain the gated signal β. Then, β is compared with... The gating feature is obtained by multiplying each element.

7. The debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks according to claim 1, characterized in that, The control signal α of A1 is used to adjust the main convolutional feature map set Y in the Ghost module. prim and cheap feature map set Y cheap The proportions are as follows: The control signal α is virtually extended to Y via broadcast. prim The internal feature maps are of the same size, resulting in signal α'. These are then concatenated according to the following formula to obtain the output feature of A2: output = Concat[α'⊙Y]. prim ,(1-α')⊙Y cheap ], Concat[∙] concatenates along the channel dimension, and ⊙ multiplies element by element.

8. The debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks according to claim 1, characterized in that: The improved MBConv block includes a lightweight gating unit, a main computation branch, and a residual branch; The lightweight gating unit is used to improve the input characteristics of the MBConv block based on the input. Generate corresponding gating features ; The main computation branch includes, in sequence, an extended convolutional layer, a channel-wise convolutional layer, a channel attention module, and a linear projection convolutional layer. The extended convolutional layer, channel-wise convolutional layer, channel attention module, and linear projection convolutional layer sequentially... Perform 1×1 up-dimensional convolution, channel-wise convolution, and then perform channel attention followed by 1×1 down-dimensional convolution to obtain the output of the improved main computation branch. ; The residual branch is used to, when the step size is 1, convert... and Identity residual connection as output of improved MBConv block When the step size is 2, As .

9. The debris flow infrasound identification method based on Mel spectrograms and lightweight neural networks according to claim 1, characterized in that: The channel attention module in the MBConv block is an ECA module.

Images