Mel spectrum and lightweight neural network-based debris flow infrasound identification method

Abstract

The invention discloses a debris flow infrasound recognition method based on a Mel spectrogram and a lightweight neural network, and belongs to the field of image data processing, and the method comprises the steps: collecting infrasonic wave signals generated by on-site debris flow and other events, and generating a Mel spectrogram data set; constructing a channel alignment fusion bridge; constructing an improved MBConv block; constructing a debris flow infrasound recognition network based on the channel alignment fusion bridge and the improved MBConv block; and training by using the Mel spectrogram data set to obtain a debris flow infrasound identification model, and identifying the infrasound signal to be identified. According to the method, the characteristics of debris flow infrasonic waves are combined, samples in the data set are constructed into the infrasonic mel spectrogram, the debris flow infrasonic recognition network is designed based on the multi-stage lightweight feature extraction module, the infrasonic mel spectrogram is classified and recognized, the calculation complexity can be effectively reduced, the parameter quantity can be reduced, low-power-consumption equipment can be adapted, and the method is suitable for popularization and application. The method can improve the classification accuracy, and is especially suitable for the real-time early warning scene of debris flow.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method for identifying debris flow infrasound based on a mel-spectrogram and a lightweight neural network. Background Art

[0002] Debris flows have complex causes, and once they occur, they cause enormous losses and are difficult to comprehensively manage. Therefore, effective early warning of debris flows has become a hot topic and a challenge in current research. Debris flow identification methods mainly include mud level analysis, geoacoustic analysis, video monitoring, and infrasound analysis. Infrasound analysis relies on capturing airborne debris flow infrasound waves. Its advantages lie in the fact that infrasound propagates faster than the speed of a debris flow (tens of times faster than the flow velocity) and has strong penetrating power (with a dominant frequency range of 0-20Hz). Therefore, infrasound analysis has become a major research focus in debris flow monitoring and early warning, both domestically and internationally. Furthermore, with the application of deep learning to acoustic signal recognition, more recognition methods combining infrasound signals with deep learning have emerged.

[0003] Because infrasound audio is a one-dimensional signal, its temporal and spatial distribution characteristics cannot be captured. Therefore, it is converted into a time-spectrogram for feature extraction and recognition. For example, in 2023, Yuan Li et al. proposed converting debris flow infrasound audio into a time-spectrogram after noise reduction and filtering. The AlexNet network was then used for feature extraction, followed by machine learning classification and recognition. This method has achieved some success in classifying debris flow infrasound signals. However, the dominant frequency characteristics of debris flow infrasound are concentrated between 0 and 20 Hz, while the frequency resolution of the time-spectrogram varies across frequency bands (lower resolution in the lower frequency bands). This results in a loss of detail in the lower frequency bands of the debris flow infrasound signal, resulting in insufficient robustness and extraction of important features when processing data in noisy environments. Traditional machine learning methods, such as the K-nearest neighbor algorithm, random forests, decision trees, and support vector machines, are often used in the recognition process. These methods, however, result in high latency and consume significant computational resources in real-time debris flow monitoring, making them unsuitable for real-time prediction of debris flow infrasound.

[0004] In 2024, Hu Meichen et al. constructed a debris flow recognition model based on a dual-stream neural network and TSN model framework, using ResNet-50 as the underlying architecture. Experimental results showed that the model improved the accuracy of debris flow recognition. However, for specific scenarios such as debris flow recognition, it is usually used with limited hardware resources, especially in edge devices or embedded systems. Due to the high complexity of deep learning networks such as ResNet-50, they consume a lot of computing resources and memory. When deployed on resource-constrained devices, they may not be able to recognize and process infrasound signals in real time. This is a disadvantage in scenarios that require a rapid response to disasters such as debris flows. At the same time, complex deep network models increase data transmission latency, which may reduce early warning efficiency.

[0005] Therefore, a debris flow infrasound identification method is needed that can adapt to low-power devices by reducing computational complexity and reducing the number of parameters without significantly reducing the model recognition accuracy. This is very critical in debris flow early warning and helps to respond in a timely manner in the early stages.

[0006] Glossary: MBConv (Mobile Inverted Bottleneck Convolution) blocks were first introduced in MobileNetV2 and later widely used in the EfficientNet series. They consist of a main computation branch and a residual branch. The main computation branch sequentially performs 1×1 dimension increase, depthwise separable convolution, SE channel attention, and 1×1 dimension reduction on the input features. The residual branch is enabled only when stride = 1 and the number of input channels equals the number of output channels. It directly adds the input features to the output of the main computation branch. stride is the stride length.

[0007] The Ghost Module is a plug-and-play lightweight convolution unit that is used to generate feature maps that are approximately equivalent to ordinary convolutions with lower FLOPs and parameters. The Ghost Module consists of a primary convolution branch (main convolution) and a cheap operation branch (cheap convolution). The main convolution branch is used for intrinsic feature maps, and the cheap convolution is used to generate ghost feature maps. The ratio is the ratio of the number of channels of the intrinsic feature map to the ghost feature map. Assuming ratio = s, the main convolution branch generates m intrinsic feature maps based on the input feature map, forming the convolution feature map set Y. prim , the cheap convolution derives s-1 ghost feature maps based on each intrinsic feature map, a total of m(s-1) ghost feature maps, forming a cheap feature map set Y cheap , and then concatenate the two along the channel dimension to get the output output=[Y prim , Y cheap ].

[0008] The SE module (Squeeze-and-Excitation Module) learns the dependencies between channels through the two steps of compression (Squeeze) and excitation (Excitation), thereby increasing the weights of key feature channels and suppressing unimportant feature channels, thereby improving the model's feature extraction capabilities.

[0009] ECA module: Efficient Channel Attention Module, which aims to achieve an efficient channel attention mechanism with lower computational complexity. Summary of the Invention

[0010] The purpose of the present invention is to provide a method for debris flow infrasound recognition based on Mel-spectrogram and lightweight neural network to solve the above problems, which has higher recognition ability in the extraction of infrasound signal features of debris flow, can significantly reduce the number of parameters and calculation amount of the model, and thus achieve faster response speed and more accurate recognition results in a resource-constrained environment.

[0011] To achieve the above-mentioned object, the present invention adopts the following technical solution: a method for identifying debris flow infrasound based on mel-spectrogram and lightweight neural network, comprising the following steps: S1, generates the Mel-spectrogram dataset D, including S11~S13; S11, collects infrasound signals generated by debris flows and other events on site; S12, for each infrasound signal, preprocessing and then converting it into an infrasound mel spectrogram; S13, classifying the infrasound mel spectrograms to obtain samples. All samples constitute a mel spectrogram dataset D, where the categories include debris flow and noise. S2, construct a channel-aligned fusion bridge; It includes the channel alignment convolution layer, base feature extraction layer, control signal generation layer, channel attention layer, and residual mixing fusion layer set in sequence; The channel alignment convolution layer is used to perform a convolution operation on the input features with a step size of 1 or 2, align the channels to a preset number of channels, and then normalize them to obtain aligned features; The base feature extraction layer is used to identify the step size of the channel alignment convolution layer. When the step size is 2, the alignment feature is downsampled to obtain the base feature X_base. When the step size is 1, the alignment feature is used as the base feature X_base. The control signal generation layer is used to perform global average pooling and Sigmoid function operations on X_base to generate a control signal α; The channel attention layer is used to perform channel attention on X_base to generate channel attention features Z; The residual mixing fusion layer is used to generate the output feature Y of the channel-aligned fusion bridge according to the formula Y=X_base+γ(Z-X_base), where γ is the residual fusion factor; S3, construct an improved MBConv block; A lightweight gating unit is set at the front end of the main computing branch of the MBConv block to obtain an improved MBConv block. The lightweight gating unit is used to input the input features of the main computing branch. Generate a gate signal β and The gated features are obtained by element-by-element multiplication and sent to the main calculation branch; S4, construct a debris flow infrasound recognition network, including a convolutional leading layer, a multi-stage lightweight feature extraction module and a classification output layer; The convolution leading layer is used to perform convolution and batch normalization operations on the samples in sequence to obtain the first feature F1; The multi-stage lightweight feature extraction module is used to input F1 and output the second feature F2. It includes a 10-layer structure from front to back, of which the 2nd, 8th, and 9th layers are Ghost modules, the 3rd and 10th layers are improved MBConv blocks, and the remaining layers are MBConv blocks; When the previous layer of the Ghost module is an MBConv block, the MBConv block and the Ghost module are marked as A1 and A2 respectively. The two are connected by a channel alignment fusion bridge. The output features of A1 are used as the input features of A2. The control signal α of A1 is used to adjust the main convolution feature map set Y in the Ghost module. prim and the cheap feature map set Y cheap The remaining adjacent layers are directly connected; The classification output layer is used to input F2 and output the predicted category of the corresponding sample; S5, using the mel-spectrogram dataset D to train the debris flow infrasound recognition network to obtain the debris flow infrasound recognition model; S6, obtaining the infrasound signal to be identified, converting it into an infrasound mel-spectrogram according to S12, and sending it into the debris flow infrasound identification model to obtain a predicted output.

[0012] Preferably, in S11, the infrasound signal is collected by an infrasound collector, and the other events include water flow, insect sounds, bird calls, airplanes, explosions, running water, lightning, cars and / or wind.

[0013] Preferably, the preprocessing in S12 is to cut the infrasound signal into infrasound segments of preset duration, filter each infrasound segment to retain only the 3-20 Hz signal, and then denoise the filtered infrasound segments using wavelet soft thresholding to obtain a preprocessed infrasound signal.

[0014] As a preference, the conversion into the infrasound mel spectrogram in S12 is specifically as follows: The preprocessed infrasound signal is subjected to analog-to-digital conversion, framing, windowing, and short-time Fourier transform to obtain the frequency spectrum and power spectrum of each frame. The power spectrum of each frame is subjected to a Mel filter bank, logarithmic conversion, and normalization to generate the Mel spectrum of each frame. The Mel spectra of all frames are then stacked along the time axis to obtain a Mel spectrogram.

[0015] Preferably, the convolution leading layer includes a convolution layer and a batch normalization layer, wherein the convolution layer is based on performing a convolution operation on the sample through the Conv2dStaticSamePadding function, wherein the convolution kernel size is 3×3, the step size is 2, the number of input channels is 3, and the number of output channels is 32, and the obtained The features of are then batch normalized by the batch normalization layer to obtain The first feature F1, where H and W are the height and width of the sample respectively, is the number of channels of F1.

[0016] Preferably, the lightweight gating unit includes a global average pooling layer, a first two-dimensional convolutional layer, a SiLU function layer, a second two-dimensional convolutional layer, and a Sigmoid function layer arranged in sequence; Input characteristics of lightweight gating unit The global average pooling layer is used for global average pooling, the first two-dimensional convolution layer is used for dimensionality reduction, the SiLU function layer is used for nonlinear activation, the second two-dimensional convolution layer is used for dimensionality increase, and the Sigmoid function layer is used for normalization to [0, 1] to obtain the gate signal β. Then β is compared with Element-wise multiplication is performed to obtain the gated features.

[0017] Preferably, the control signal α of A1 is used to adjust the main convolution feature map set Y in the Ghost module prim and the cheap feature map set Y cheap The ratio is: The control signal α is virtually expanded to Y by broadcasting prim The inner feature maps have the same size, and the signal α' is obtained, and the output feature of A2 is obtained by splicing according to the following formula output=Concat[α'⊙Y prim , (1-α')⊙Y cheap ], Concat[∙] is concatenation along the channel dimension, and ⊙ is element-wise multiplication.

[0018] Preferably, the improved MBConv block includes a lightweight gating unit, a main computation branch and a residual branch; The lightweight gating unit is used to improve the input features of the MBConv block according to the input Generate corresponding gating features ; The main computing branch includes an extended convolution layer, a channel-by-channel convolution layer, a channel attention module, and a linear projection convolution layer connected in sequence. The extended convolution layer, the channel-by-channel convolution layer, the channel attention module, and the linear projection convolution layer are sequentially connected. Perform 1×1 dimensionality increase convolution, channel-by-channel convolution, perform channel attention and then 1×1 dimensionality reduction convolution to obtain the output of the improved main computation branch ; The residual branch is used to convert and Identity residual connection as the output of the improved MBConv block , when the step size is 2, As .

[0019] Preferably, the channel attention module in the MBConv block is an ECA module.

[0020] The idea of ​​the present invention is to convert the infrasound signals of debris flow into infrasound mel-spectrograms to form a data set D, then construct a debris flow infrasound recognition network and train it with data set D to obtain a debris flow infrasound recognition model for classifying and identifying infrasound signals in debris flow areas.

[0021] Regarding infrasound Mel-spectrograms: Since the change of energy over time is observed in the Mel-spectrograms, based on the spectral characteristics of debris flows, the present invention found that after converting the infrasound waves of noise and debris flow into infrasound Mel-spectrograms, there are significant differences between the two, making them more recognizable.

[0022] Regarding the debris flow infrasound recognition network: the main improvements include channel alignment fusion bridge, lightweight gating unit, and multi-stage lightweight feature extraction module.

[0023] (1) About channel alignment fusion bridge: When the feature stream flows from the MBConv block to the Ghost module, a channel alignment fusion bridge is set between the two. The channel alignment convolution layer uses convolution and batch normalization operations to align the features output by the MBConv block with the input scale of the Ghost module in terms of channel dimension and statistical distribution, reducing distribution drift and covariate shift. The statistical distribution is the mean and variance of the feature map on each channel (usually obtained by the sliding average statistics of the preceding batch normalization layer). The base feature extraction layer generates base features X_base based on the stride of the convolution operation in the channel-aligned convolution layer. The control signal generation layer and the channel attention layer generate control signals α and channel attention features Z, respectively, based on X_base. The control signal α regulates the ratio of the main and cheap convolutions. The ratio is assigned within the group based on the upstream features. α is virtually expanded to the same shape as the features through broadcasting to generate a signal α'. The main convolution branch features are scaled by α', and the cheap convolution branch features are scaled by 1-α'. The output is then concatenated along the channel dimension. The scaling transforms the contribution ratio of the main and cheap convolution branches from a fixed one to a learnable and controllable dynamic weight, while maintaining the stability of the feature tensor shape and computation graph. This allows the model to adaptively determine the contribution between the main and cheap convolution branches without changing the number and arrangement of the Ghost module's output channels. When α' is 1, the output comes entirely from the main convolution branch; when α' is 0, the output comes entirely from the cheap convolution branch. Generally, the contributions of the two paths are adjusted according to α':(1-α'). The role of the channel attention feature Z is to assign weights to each channel, allowing the model to automatically learn which channels should be strengthened or suppressed. Compared to the original base features, the channel attention feature Z semantically highlights key features, suppresses redundant features, and improves the quality of information flow. X_base and the channel attention feature Z are combined through a residual mixing layer to generate the output feature Y of the channel-aligned fusion bridge. During residual mixing, γ is introduced for interpolation fusion. γ is a residual fusion factor that gradually introduces enhanced information during training. Y retains the original structural information of X_base while gradually incorporating the attention weighting effect of the channel attention feature Z, making the network more discriminative and generalizable, and allowing the downstream Ghost module to receive more informative input.

[0024] (2) Regarding the improved MBConv block: When the feature flow flows from the Ghost module to the MBConv block, the improved MBConv block is used. Its characteristic is that a lightweight gating unit is added to the existing MBConv block. Although the output features of the Ghost module have expanded the number of channels, they still contain some noise information. If they are directly input to the MBConv block, there will be problems such as wasted computing resources and insufficient feature discrimination. Therefore, it is necessary to perform lightweight channel screening and dynamic modulation on the input features before entering the main branch of MBConv to highlight semantically significant channels, suppress noise and redundancy, and improve the modeling efficiency of the MBConv block.

[0025] The global average pooling layer performs a global average on the input features over the spatial dimensions H×W to obtain global statistics for each channel, forming a channel description vector. This operation compresses high-dimensional spatial features into one-dimensional channel features, preserving global semantic information at the channel level and reducing computational complexity.

[0026] The channel description vector is linearly mapped (1×1 convolution) in the first two-dimensional convolution layer to achieve channel dimensionality reduction, which reduces the number of channels, the number of parameters and the amount of calculation, while compressing the information and extracting features.

[0027] The SiLU function layer introduces nonlinearity after the output of the previous convolution layer, retains some negative gradients, and can provide smoother gradient flow, avoid premature discarding of feature information, and retain and improve gate fitting ability and stability.

[0028] The function of the second two-dimensional convolution layer is to map the number of intermediate channels after dimensionality reduction back to the number of original input channels, restore the dimension consistent with the input, and generate a corresponding weight expression vector U for each channel to ensure that the gated signal is aligned with the original input in dimension. The Sigmoid function layer compresses the output of the previous convolution layer to the interval [0,1]. According to the formula Get the gate signal β. The gate signal β obtained by Sigmoid is combined with the input feature Multiply element by element to get the backbone input features of the MBConv block , input to the expanded convolution layer. This operation can perform weighted modulation on each channel, strengthen important channels, and reduce redundant channels and noise channels. The residual branch of the MBConv block converts the unmodulated input features into The output is added back. The existence of the identity residual ensures stable information transmission without loss. Before entering the main branch of the MBConv block, the lightweight gating unit dynamically modulates the input features with extremely low computational effort, highlighting semantically significant channels and suppressing redundant and noisy channels output by the Ghost module, concentrating computing power on high-value subspaces. When processing these high-value features, the convolutional layers in the MBConv block more efficiently model discriminative patterns, improving feature expression and generalization capabilities.

[0029] (3) Multi-stage lightweight feature extraction module: This module is constructed based on the channel alignment fusion bridge, lightweight gating unit, and existing MBConv blocks and Ghost modules. The output size of the convolutional leader structure is 112×112×32. Considering that the energy of the debris flow infrasound signal in the mel-spectrogram is mainly concentrated in the low-frequency band, different module types are used at different stages to balance the feature extraction accuracy and lightweight requirements: Initial layer (convolutional leader layer, multi-stage lightweight feature extraction module layer 1): Focusing on basic patterns such as low-frequency energy distribution, high-precision convolution is required to capture subtle changes and provide reliable input for subsequent deep analysis, so the MBConv block is retained. Intermediate layer (layer 2-9): Extracting texture features, with low dependence on high-precision convolution. To reduce the risk of calculation and overfitting, GhostModule is used at specific locations. Later layer (layer 10 and classification output layer): Extracting high-level abstract features, which directly affects the classification accuracy. The MBConv block and subsequent high-dimensional convolution are retained to ensure expression ability.

[0030] The Ghost module is divided into two parts: the main convolution branch is a 1×1 convolution, and the cheap convolution branch is a 3×3 depth-separable convolution. From this, the parameters and computational complexity of the Ghost module can be calculated. The main branch convolution parameter Params1 = (C in ⋅m⋅K 2 ), C in is the number of input channels, m is the number of main feature maps, and K is the size of the main branch convolution kernel. The number of cheap convolution branch parameters Params2=m(r-1)k 2 , r is the expansion ratio, k is the size of the cheap convolution kernel. Finally, the total parameter amount Params=(C in ⋅m⋅K 2 )+ m(r-1)k 2 The formula for calculating the amount is , H out and W out are the height and width of the output feature map of the main convolution branch, and are the height and width of the output feature map of the cheap convolution branch, respectively.

[0031] MBConv block contains extended convolution (1×1), channel-by-channel convolution (k×k), and projection convolution (1×1). It can be seen that the parameter calculation formula is , t is the expansion factor. The formula for calculating the amount is . H and W are the height and width of the input feature map, and is the height and width of the output feature map.

[0032] Taking the second layer of multi-stage lightweight feature extraction as an example, the input feature map size is 112×112×16. Under the same conditions, the total computational cost of the Ghost module is 7.41MMACs (where the first M represents 10 6 MACs (Multiply–Accumulate Operations), with a parameter count of 1.67k. The MBConv module has a total computational load of 60.81 MMACs and 13.97k parameters. Comparison results show that the Ghost module significantly reduces both computational load and parameter count compared to the MBConv module, significantly reducing model complexity. Therefore, Ghost modules can be safely used to replace some of the more parameter-intensive and computationally intensive MBConv blocks in the intermediate stages of the model, at layers that are less dependent on high-precision convolution. This achieves a lightweight model while ensuring a certain improvement in overall recognition accuracy and precision.

[0033] (4) Replace the SE module in the MBConv block with the ECA module. The SE module has two fully connected layers, which increases the number of parameters and computation. The ECA module does not require two fully connected layers, significantly reducing the number of parameters and computation. This avoids the information bottleneck from dimensionality increase to dimensionality reduction in SE, resulting in smoother information flow, lighter weight, and better suited for spectral image input.

[0034] Compared with the prior art, the advantages of the present invention are: (1) By introducing a channel alignment fusion bridge before the Ghost module, the model can highlight the discriminative information in the backbone features, thereby improving the relevance and utilization of the extended features of the Ghost module, and making the features generated by the cheap convolution branch more semantically consistent with the main branch convolution. The entire process significantly improves the expressiveness and distinguishability of the features while maintaining the original lightweight advantage of the Ghost module.

[0035] (2) An improved MBConv block is designed. A lightweight gating unit is added to the beginning of the MBConv block. This unit relies only on global average pooling and two 1×1 convolutions, with extremely low computational overhead. By assigning weights to each channel through the gating signal β, adaptive adjustment of input features is achieved, effectively suppressing invalid or redundant channels. The MBConv block no longer processes all features, but instead focuses on high-value features that have been screened and modulated. This ensures computational efficiency while enhancing the module's discriminability and generalization capabilities.

[0036] (3) Based on the channel alignment fusion bridge and the improved MBConv block, a multi-stage lightweight feature extraction module is designed, which can effectively reduce the computational complexity and the number of parameters to adapt to low-power devices, especially for debris flow warning scenarios.

[0037] (4) Based on the characteristics of debris flow infrasound, the samples in the dataset are constructed into infrasound Mel-spectrograms, and a debris flow infrasound recognition network is designed based on a multi-stage lightweight feature extraction module. The infrasound Mel-spectrograms are classified and recognized, which can not only improve the classification efficiency but also improve the accuracy.

[0038] (5) The use of ECA instead of SE module reduces the computational complexity of the original model by about 13.8%, making the improved MBConv module maintain lower computational complexity. The improved MBConv module can effectively preserve the neighborhood correlation of spectral images and has better adaptability and discrimination ability in spectral tasks.

[0039] In summary, the model and method of the present invention have the characteristics of high classification accuracy, low computational complexity and parameter reduction, and are particularly suitable for real-time prediction of debris flows. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Flowchart of the present invention; Figure 2 This is the network structure diagram for debris flow infrasound identification; Figure 3 Figure 1. Fusion bridge structure diagram for channel alignment; Figure 4 This is a structural diagram of the lightweight gate control unit; Figure 5 To improve the MBConv block structure diagram; Figure 6 This is the infrasound Mel-spectrogram corresponding to a debris flow event; Figure 7 This is the infrasound Mel-spectrogram corresponding to an explosion event. DETAILED DESCRIPTION

[0041] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0042] Example 1: See Figures 1 to 5 , A method for identifying debris flow infrasound based on mel-spectrogram and lightweight neural network includes the following steps: S1, generates the Mel-spectrogram dataset D, including S11~S13; S11, collects infrasound signals generated by debris flows and other events on site; S12, for each infrasound signal, preprocessing and then converting it into an infrasound mel spectrogram; S13, classifying the infrasound mel spectrograms to obtain samples. All samples constitute a mel spectrogram dataset D, where the categories include debris flow and noise. S2, construct a channel-aligned fusion bridge; It includes the channel alignment convolution layer, base feature extraction layer, control signal generation layer, channel attention layer, and residual mixing fusion layer set in sequence; The channel alignment convolution layer is used to perform a convolution operation on the input features with a step size of 1 or 2, align the channels to a preset number of channels, and then normalize them to obtain aligned features; The base feature extraction layer is used to identify the step size of the channel alignment convolution layer. When the step size is 2, the alignment feature is downsampled to obtain the base feature X_base. When the step size is 1, the alignment feature is used as the base feature X_base. The control signal generation layer is used to perform global average pooling and Sigmoid function operations on X_base to generate a control signal α; The channel attention layer is used to perform channel attention on X_base to generate channel attention features Z; The residual mixing fusion layer is used to generate the output feature Y of the channel-aligned fusion bridge according to the formula Y=X_base+γ(Z-X_base), where γ is the residual fusion factor; S3, construct an improved MBConv block; A lightweight gating unit is set at the front end of the main computing branch of the MBConv block to obtain an improved MBConv block. The lightweight gating unit is used to input the input features of the main computing branch. Generate a gate signal β and The gated features are obtained by element-by-element multiplication and sent to the main calculation branch; S4, construct a debris flow infrasound recognition network, including a convolutional leading layer, a multi-stage lightweight feature extraction module and a classification output layer; The convolution leading layer is used to perform convolution and batch normalization operations on the samples in sequence to obtain the first feature F1; The multi-stage lightweight feature extraction module is used to input F1 and output the second feature F2. It includes a 10-layer structure from front to back, of which the 2nd, 8th, and 9th layers are Ghost modules, the 3rd and 10th layers are improved MBConv blocks, and the remaining layers are MBConv blocks; When the previous layer of the Ghost module is an MBConv block, the MBConv block and the Ghost module are marked as A1 and A2 respectively. The two are connected by a channel alignment fusion bridge. The output features of A1 are used as the input features of A2. The control signal α of A1 is used to adjust the main convolution feature map set Y in the Ghost module. prim and the cheap feature map set Y cheap The remaining adjacent layers are directly connected; The classification output layer is used to input F2 and output the predicted category of the corresponding sample; S5, using the mel-spectrogram dataset D to train the debris flow infrasound recognition network to obtain the debris flow infrasound recognition model; S6, obtaining the infrasound signal to be identified, converting it into an infrasound mel-spectrogram according to S12, and sending it into the debris flow infrasound identification model to obtain a predicted output.

[0043] In S11, the infrasound wave signal is collected by an infrasound collector, and the other events include water flow, insect chirping, bird singing, airplanes, explosions, flowing water, lightning, cars and / or wind.

[0044] The preprocessing in S12 is to cut the infrasound signal into infrasound segments of preset duration, filter each infrasound segment to retain only the 3-20 Hz signal, and then denoise the filtered infrasound segments using wavelet soft thresholding to obtain a preprocessed infrasound signal.

[0045] The conversion to infrasound mel spectrogram in S12 is as follows: The preprocessed infrasound signal is subjected to analog-to-digital conversion, framing, windowing, and short-time Fourier transform to obtain the frequency spectrum and power spectrum of each frame. The power spectrum of each frame is subjected to a Mel filter bank, logarithmic conversion, and normalization to generate the Mel spectrum of each frame. The Mel spectra of all frames are then stacked along the time axis to obtain a Mel spectrogram.

[0046] The convolution leading layer includes a convolution layer and a batch normalization layer. The convolution layer performs a convolution operation on the sample based on the Conv2dStaticSamePadding function, where the convolution kernel size is 3×3, the step size is 2, the number of input channels is 3, and the number of output channels is 32. The features of are then batch normalized by the batch normalization layer to obtain The first feature F1, where H and W are the height and width of the sample respectively, is the number of channels of F1.

[0047] The lightweight gating unit includes a global average pooling layer, a first two-dimensional convolutional layer, a SiLU function layer, a second two-dimensional convolutional layer, and a Sigmoid function layer, which are arranged in sequence; Input characteristics of lightweight gating unit The global average pooling layer is used for global average pooling, the first two-dimensional convolution layer is used for dimensionality reduction, the SiLU function layer is used for nonlinear activation, the second two-dimensional convolution layer is used for dimensionality increase, and the Sigmoid function layer is used for normalization to [0, 1] to obtain the gate signal β. Then β is compared with Element-wise multiplication is performed to obtain the gated features.

[0048] The control signal α of A1 is used to adjust the main convolution feature map set Y in the Ghost module prim and the cheap feature map set Y cheap The ratio is: The control signal α is virtually expanded to Y by broadcasting prim The inner feature maps have the same size, and the signal α' is obtained, and the output feature of A2 is obtained by splicing according to the following formula output=Concat[α'⊙Y prim , (1-α')⊙Y cheap ], Concat[∙] is concatenation along the channel dimension, and ⊙ is element-wise multiplication.

[0049] The improved MBConv block includes a lightweight gating unit, a main computation branch and a residual branch; The lightweight gating unit is used to improve the input features of the MBConv block according to the input Generate corresponding gating features ; The main computing branch includes an extended convolution layer, a channel-by-channel convolution layer, a channel attention module, and a linear projection convolution layer connected in sequence. The extended convolution layer, the channel-by-channel convolution layer, the channel attention module, and the linear projection convolution layer are sequentially connected. Perform 1×1 dimensionality increase convolution, channel-by-channel convolution, perform channel attention and then 1×1 dimensionality reduction convolution to obtain the output of the improved main computation branch ; The residual branch is used to convert and Identity residual connection as the output of the improved MBConv block , when the step size is 2, As .

[0050] The channel attention module in the MBConv block is the ECA module.

[0051] Example 2: See Figures 1 to 7, based on Example 1, the operation method is given in more detail: The preprocessing described in S12 involves first slicing the data, cutting the infrasound signal into 6-second segments to expand the data set. Then, filtering is performed using a Chebyshev II filter to remove signals with frequencies greater than or equal to 20 Hz and less than or equal to 3 Hz, retaining only signals between 3 and 20 Hz. Zero-phase-shift filtering is used during filtering to avoid phase distortion. Wavelet soft thresholding is then used for denoising, and a multi-scale analysis method is used to retain key features and reduce noise in the debris flow infrasound. Ultimately, the preprocessed infrasound signal is obtained.

[0052] The conversion to infrasound mel spectrogram described in S12 is: set the sampling rate f s The frequency spectrum is 100Hz, the FFT window size is 128 or 256, the frame length is 0.32s, the frame shift is 0.02s, and the number of Mel filter groups is 80. Since the larger the FFT window, the higher the resolution, but the amount of calculation also increases, so this operation selects 128 while ensuring the high resolution of the Mel spectrogram and the relatively small amount of calculation. For each preprocessed infrasound signal, first perform analog-to-digital conversion, then frame division, and divide the infrasound signal into very short segments, each of which can be approximated as a stationary signal. Add a Hamming window to each frame of the audio to ensure a smooth transition of the signal boundary part, and then use short-time Fourier transform to obtain the spectrum and power spectrum of each frame. The power spectrum of each frame is mapped to the Mel scale by the Mel filter group, and then logarithmically transformed and normalized to generate the Mel spectrum of each frame. Then, the Mel spectra of all frames are stacked along the time axis to obtain the infrasound Mel spectrogram. There is a clear difference between the infrasound Mel spectrograms corresponding to debris flow events and noise events, see Figure 6 and Figure 7 . Figure 6 There are obvious and continuous bright areas in the 0-20 Hz range, which represent the continuous energy release of debris flow signals. Figure 7 The corresponding area is intermittent, and there are many gaps in the energy distribution, which means that the noise signal is usually short-term and the energy fluctuates irregularly.

[0053] Example 3: In order to illustrate the effect of the present invention, we designed a comparative experiment: Dataset: A self-made dataset containing 1207 infrasound mel spectrograms, including 607 debris flow infrasound mel spectrograms and 600 other noise infrasound mel spectrograms, divided into training set and test set.

[0054] Experimental environment: The hardware environment is NVIDIA RTX 4060 GPU, and the software environment is PyTorch: 1.12.1+cu116.

[0055] Experimental Grouping: The experimental group used the debris flow infrasound recognition model of the present invention, while the control group used the classic lightweight deep learning recognition model EfficientNet-b0. In the debris flow infrasound recognition model of the present invention, the convolutional layers in the MBConv block used the SiLU activation function, the convolutional layers in the Ghost module used the ReLU activation function, and the pooling layers used average pooling. During training, the cross-entropy loss function and the Adam optimization algorithm were used. The initial learning rate was 0.0001, and the learning rate scheduler was ReduceLROnPlateau, which was dynamically adjusted during training.

[0056] By comparing the model of the present invention and EfficientNet-b0, we obtain Table 1 and Table 2; Table 1. Performance indicators of different methods: Model Name Accuracy Recall F1 score Accuracy Model of the present invention 96.00% 97.56% 96.77% 96.69% EfficientNet-b0 92.97% 96.74% 94.82% 94.63% Table 2 Parameters, computational complexity, and average inference time of different methods Model Name Parameter quantity Computational Amount Average inference time Model of the present invention 2.68M 178.8MMac 1.2240s EfficientNet-b0 5.29M 401.67MMac 1.4553s

[0057] Data Analysis: The proposed model performs well in terms of precision, recall, F1 score, and accuracy, achieving 96.69% and 97.56% accuracy and recall, respectively, significantly outperforming the comparison model EfficientNet-b0. This is due to the effective collaboration between the Ghost module and MBConv. In terms of parameter and computational complexity, the proposed model has 2.68M parameters and 178.8M FLOPs, significantly lower than the original model EfficientNet-b0's 5.29M parameters and 401.67M FLOPs. It also achieves a certain improvement in average inference time, indicating that the proposed model improvement method significantly reduces computational costs while maintaining high performance, enabling effective infrasound image recognition tasks.

[0058] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying debris flow infrasound based on mel-spectrogram and lightweight neural network, characterized in that: The following steps are involved: S1, generates the Mel-spectrogram dataset D, including S11~S13; S11, collects infrasound signals generated by debris flows and other events on site; S12, for each infrasound signal, preprocessing and then converting it into an infrasound mel spectrogram; S13, classifying the infrasound mel spectrograms to obtain samples. All samples constitute a mel spectrogram dataset D, where the categories include debris flow and noise. S2, construct a channel-aligned fusion bridge; It includes the channel alignment convolution layer, base feature extraction layer, control signal generation layer, channel attention layer, and residual mixing fusion layer set in sequence; The channel alignment convolution layer is used to perform a convolution operation on the input features with a step size of 1 or 2, align the channels to a preset number of channels, and then normalize them to obtain aligned features; The base feature extraction layer is used to identify the step size of the channel alignment convolution layer. When the step size is 2, the alignment feature is downsampled to obtain the base feature X_base. When the step size is 1, the alignment feature is used as the base feature X_base. The control signal generation layer is used to perform global average pooling and Sigmoid function operations on X_base to generate a control signal α; The channel attention layer is used to perform channel attention on X_base to generate channel attention features Z; The residual mixing fusion layer is used to generate the output feature Y of the channel-aligned fusion bridge according to the formula Y=X_base+γ(Z-X_base), where γ is the residual fusion factor; S3, construct an improved MBConv block; A lightweight gating unit is set at the front end of the main computing branch of the MBConv block to obtain an improved MBConv block. The lightweight gating unit is used to input the input features of the main computing branch. Generate a gate signal β and The gated features are obtained by element-by-element multiplication and sent to the main calculation branch; S4, construct a debris flow infrasound recognition network, including a convolutional leading layer, a multi-stage lightweight feature extraction module and a classification output layer; The convolution leading layer is used to perform convolution and batch normalization operations on the samples in sequence to obtain the first feature F1; The multi-stage lightweight feature extraction module is used to input F1 and output the second feature F2. It includes a 10-layer structure from front to back, of which the 2nd, 8th, and 9th layers are Ghost modules, the 3rd and 10th layers are improved MBConv blocks, and the remaining layers are MBConv blocks; When the previous layer of the Ghost module is an MBConv block, the MBConv block and the Ghost module are marked as A1 and A2 respectively. The two are connected by a channel alignment fusion bridge. The output features of A1 are used as the input features of A2. The control signal α of A1 is used to adjust the main convolution feature map set Y in the Ghost module. prim and the cheap feature map set Y cheap The remaining adjacent layers are directly connected; The classification output layer is used to input F2 and output the predicted category of the corresponding sample; S5, using the mel-spectrogram dataset D to train the debris flow infrasound recognition network to obtain the debris flow infrasound recognition model; S6, obtaining the infrasound signal to be identified, converting it into an infrasound mel-spectrogram according to S12, and sending it into the debris flow infrasound identification model to obtain a predicted output.

2. The method for identifying debris flow infrasound based on mel-spectrogram and lightweight neural network according to claim 1, characterized in that: In S11, the infrasound wave signal is collected by an infrasound collector, and the other events include water flow, insect chirping, bird singing, airplanes, explosions, flowing water, lightning, cars and / or wind.

3. The method for identifying debris flow infrasound based on Mel-spectrogram and lightweight neural network according to claim 1, characterized in that: The preprocessing in S12 is to cut the infrasound signal into infrasound segments of preset duration, filter each infrasound segment to retain only the 3-20 Hz signal, and then denoise the filtered infrasound segments using wavelet soft thresholding to obtain a preprocessed infrasound signal.

4. The method for identifying debris flow infrasound based on mel-spectrogram and lightweight neural network according to claim 1, characterized in that: The conversion to infrasound mel spectrogram in S12 is as follows: The preprocessed infrasound signal is subjected to analog-to-digital conversion, framing, windowing, and short-time Fourier transform to obtain the frequency spectrum and power spectrum of each frame. The power spectrum of each frame is subjected to a Mel filter bank, logarithmic conversion, and normalization to generate the Mel spectrum of each frame. The Mel spectra of all frames are then stacked along the time axis to obtain a Mel spectrogram.

5. The method for identifying debris flow infrasound based on Mel-spectrogram and lightweight neural network according to claim 1, characterized in that: The convolution leading layer includes a convolution layer and a batch normalization layer. The convolution layer performs a convolution operation on the sample based on the Conv2dStaticSamePadding function, where the convolution kernel size is 3×3, the step size is 2, the number of input channels is 3, and the number of output channels is 32. The features of are then batch normalized by the batch normalization layer to obtain The first feature F1, where H and W are the height and width of the sample respectively, is the number of channels of F1.

6. The method for identifying debris flow infrasound based on mel-spectrogram and lightweight neural network according to claim 1, characterized in that: The lightweight gating unit includes a global average pooling layer, a first two-dimensional convolutional layer, a SiLU function layer, a second two-dimensional convolutional layer, and a Sigmoid function layer, which are arranged in sequence; Input characteristics of lightweight gating unit The global average pooling layer is used for global average pooling, the first two-dimensional convolution layer is used for dimensionality reduction, the SiLU function layer is used for nonlinear activation, the second two-dimensional convolution layer is used for dimensionality increase, and the Sigmoid function layer is used for normalization to [0, 1] to obtain the gate signal β. Then β is compared with Element-wise multiplication is performed to obtain the gated features.

7. The method for identifying debris flow infrasound based on mel-spectrogram and lightweight neural network according to claim 1, characterized in that: The control signal α of A1 is used to adjust the main convolution feature map set Y in the Ghost module prim and the cheap feature map set Y cheap The ratio is: The control signal α is virtually expanded to Y by broadcasting prim The inner feature maps have the same size, and the signal α' is obtained, and the output feature of A2 is obtained by splicing according to the following formula output=Concat[α'⊙Y prim , (1-α')⊙Y cheap ], Concat[∙] is concatenation along the channel dimension, and ⊙ is element-wise multiplication.

8. The method for identifying debris flow infrasound based on mel-spectrogram and lightweight neural network according to claim 1, characterized in that: The improved MBConv block includes a lightweight gating unit, a main computation branch and a residual branch; The lightweight gating unit is used to improve the input features of the MBConv block according to the input Generate corresponding gating features ; The main computing branch includes an extended convolution layer, a channel-by-channel convolution layer, a channel attention module, and a linear projection convolution layer connected in sequence. The extended convolution layer, the channel-by-channel convolution layer, the channel attention module, and the linear projection convolution layer are sequentially connected. Perform 1×1 dimensionality increase convolution, channel-by-channel convolution, perform channel attention and then 1×1 dimensionality reduction convolution to obtain the output of the improved main computation branch ; The residual branch is used to convert and Identity residual connection as the output of the improved MBConv block , when the step size is 2, As .

9. The method for identifying debris flow infrasound based on mel-spectrogram and lightweight neural network according to claim 1, characterized in that: The channel attention module in the MBConv block is the ECA module.

Images