A landslide rockfall detection method and system based on deep learning

Abstract

This invention discloses a landslide and rockfall detection method and system based on deep learning. The detection method includes acquiring images and inputting them into a PCS-YOLO network model, extracting features in the frequency-spatial domain collaboratively, modeling in parallel with three attention branches and fusing them to output multi-scale feature maps, and using a lightweight shared detection head for localization and classification, outputting the coordinates and confidence scores of the rockfall bounding boxes, and triggering an early warning when the confidence score exceeds a threshold. The detection system includes an image acquisition and preprocessing module, a PCS-YOLO model inference module, a frequency-spatial domain collaborative feature extraction module, a parallel attention feature fusion module, a lightweight shared detection localization module, and a result output and early warning module. This invention, through the synergistic effect of frequency domain texture enhancement, parallel attention fusion, and a lightweight shared detection head, significantly improves the detection accuracy and robustness of multi-scale rockfalls in visually camouflaged environments while ensuring that the model is lightweight and suitable for edge deployment.

Description

Technical Field

[0001] This invention belongs to the field of geological disaster detection technology, and in particular relates to a landslide rockfall detection method and system based on deep learning. Background Technology

[0002] Landslides and rockfalls are among the most destructive geological disasters, characterized by their suddenness, wide impact, and high risk of cascading disasters. They seriously threaten the lives and property of mountain transportation arteries, hydropower projects, mining facilities, and surrounding residents. With the increasing frequency of extreme weather events globally and the continuous advancement of infrastructure construction in mountainous areas, the triggering factors for landslides and rockfalls are becoming more complex. Traditional manual inspection methods are no longer sufficient to meet the core requirements of modern disaster prevention and mitigation for all-weather, wide-coverage, real-time, and high-precision operations.

[0003] In existing technologies, visual inspection methods have been designed to replace manual inspections. However, visual inspection methods are sensitive to complex mountain backgrounds and lighting conditions. In visual camouflage environments where the textures of fallen rocks and surrounding rocks and vegetation are highly similar, they suffer from insufficient adaptability and poor robustness.

[0004] In recent years, breakthroughs in computer vision using deep learning technology have provided non-contact solutions for geological disaster detection. Its convolutional neural network-based target detection algorithms, with their end-to-end inference advantages, have gradually become a research hotspot for intelligent landslide and rockfall detection. However, complex mountainous scenes present challenges such as strong background interference, large differences in target scale, and limited computing power at the edge. Existing deep learning-based methods suffer from complex network structures and large parameter counts in general detection models, affecting deployment efficiency on resource-constrained edge devices in the field. Furthermore, most general deep learning-based models struggle to overcome visual camouflage interference in complex backgrounds and lack sufficient feature extraction capabilities for multi-scale rockfalls, resulting in insufficient robustness in detection.

[0005] In the prior art, Chinese invention patent application publication number CN118334313A discloses a lightweight remote sensing image target detection method based on deep learning. This method designs a frequency domain attention module based on discrete wavelet transform and visual Transformer, and constructs a lightweight detection network based on the backbone network of MobileNetv2 and the neck network of YOLOv5s. However, this method mainly addresses the problems of uneven target size and uneven distribution in general remote sensing images. Its frequency domain processing method uses discrete wavelet transform and Transformer, which has high computational complexity and is not specifically optimized for visual camouflage scenes. In complex mountainous scenes where rocks are highly similar to the background, the detection accuracy is insufficient.

[0006] Chinese invention patent application publication number CN120876821A discloses an efficient target detection method in foggy conditions, which designs a defogging image enhancement and fusion module and a dual convolution feature extraction module, mainly to solve the problem of unclear targets caused by fog. However, this method targets image blurring caused by atmospheric scattering, rather than texture similarity problems caused by visual camouflage, and its technical means are difficult to directly use for landslide and rockfall detection tasks.

[0007] The Chinese invention patent application publication number CN121169887A discloses a method for detecting landslides on both banks of a reservoir based on global-local feature perception collaboration, and the invention patent application publication number CN121482550A discloses a landslide detection method that integrates geometric, contextual, and frequency domain multi-source features. Although these methods are designed for landslide detection tasks, they mostly adopt semantic segmentation architectures, have a large number of model parameters, slow inference speed, and are difficult to deploy on field edge detection equipment with limited computing power.

[0008] In summary, existing technologies for landslide and rockfall detection still have the following shortcomings: (1) Visual camouflage makes detection difficult: In real outdoor environments, falling rocks are often highly similar in color and texture to the surrounding rocks, vegetation, soil and other backgrounds, forming visual camouflage, making it difficult for existing models to distinguish between targets and backgrounds, and easily leading to missed detections. (2) Difficulty in taking into account multiple scale targets: The scale of landslides and rocksfalls varies greatly (from small gravel to huge collapses). After multiple layers of convolution downsampling, the existing models are difficult to accurately capture both large-scale and small-scale targets at the same time. In particular, the texture and edge information of small targets are easily lost. (3) Difficulty in deployment at the edge: Existing high-precision detection models have complex network structures, large number of parameters, and high inference latency, making it difficult to achieve second-level real-time early warning on field edge detection devices with limited computing power and power consumption.

[0009] Therefore, it is necessary to provide a landslide and rockfall detection method and system based on deep learning to solve the above-mentioned technical problems. Summary of the Invention

[0010] The main objective of this invention is to provide a deep learning-based method for detecting landslide rocks. Through the collaborative design of frequency domain texture enhancement, parallel attention fusion, and a lightweight shared detection head, the method significantly improves the detection accuracy and robustness of multi-scale rocks in visually camouflaged environments while ensuring that the model is lightweight and suitable for edge deployment.

[0011] This invention achieves the above objective through the following technical solution: a landslide and rockfall detection method based on deep learning, comprising the following steps: Step S1, Image Acquisition and Image Preprocessing: Acquire real-time images and then preprocess them; Step S2, Model Invocation: Input the preprocessed image into the pre-built and trained PCS-YOLO network model. The PCS-YOLO network model is a lightweight convolutional neural network. Step S3, Frequency-Spatial Domain Collaborative Feature Extraction: The PCS-YOLO network model performs forward inference on the input image, extracts image features through the Backbone network, and obtains an initial feature map. The initial feature map is then processed in two parallel steps: the first step is to use the Scharr operator to extract explicit edge information, obtaining spatial domain edge features; the second step is to use Fast Fourier Transform to map the features to the frequency domain to enhance texture details, followed by inverse transform to restore them, obtaining frequency domain texture enhancement features; the spatial domain edge features and the frequency domain texture enhancement features are then fused to output a feature map that enhances edge texture representation. Step S4, Parallel Attention Feature Fusion: The feature map with enhanced edge texture representation output from Step S3 is input into the Neck network, and three parallel attention processes are performed: The first attention process is to perform global average pooling and global max pooling on the feature map to aggregate global information and obtain channel recalibration features; the second attention process is to perform pooling along the height and width directions of the feature map to generate orientation-aware feature maps to capture position information and obtain coordinate-aware features; the third attention process is to perform spatial convolution on the feature map using large kernel convolution to generate a spatial mask and obtain spatial saliency features; the channel recalibration features, coordinate-aware features, and spatial saliency features are adaptively weighted and fused to output a fused multi-scale feature map; Step S5, Lightweight Shared Detection and Localization: Input the multi-scale feature map output from step S4 into the SCSA-Head detection module. The SCSA-Head detection module adopts a cross-scale parameter sharing mechanism, which enables the multi-scale feature maps to share the same set of depthwise separable convolutions for classification and regression prediction. It also introduces learnable scale-aware coefficients to calibrate the output distribution of the multi-scale feature maps and outputs the bounding box coordinates and confidence of the rockfall target. Step S6, Result Output and Warning: When the confidence level of the falling rock target exceeds the preset threshold, the bounding box coordinates of the falling rock target are output and a warning signal is triggered to realize end-to-end real-time detection and warning of landslides and falling rocks in complex mountainous scenes.

[0012] Furthermore, the construction and training method of the PCS-YOLO network model includes the following steps: Step T1, Data Acquisition and Processing: Acquire landslide and rockfall images, and construct a rockfall image dataset containing multiple scene types. The dataset includes a training set, a validation set, and a test set. Step T2, Network Model Construction: An initial PCS-YOLO network model is constructed based on the YOLO framework. The initial PCS-YOLO network model includes a Backbone network, a Neck network, and an SCSA-Head detection module. A CSP-TFSA module is introduced at the connection between the Backbone network and the Neck network to fuse spatial domain edge features and frequency domain texture enhancement features. A PSCA attention module is embedded in the Neck feature fusion layer to model in parallel from three dimensions: channel, coordinate, and space. The SCSA-Head detection module adopts a cross-scale parameter sharing mechanism, enabling multi-scale feature maps to share the same set of depthwise separable convolutions for classification and regression prediction. A learnable scale-aware coefficient is introduced to calibrate the output distribution of features at different scales. Step T3, Model Training: Train the initial PCS-YOLO network model using the training set, and optimize it using the SGD optimizer to obtain the initially trained PCS-YOLO network model and its initial model weights. Step T4, Model Validation and Testing: Validate and fine-tune the pre-trained PCS-YOLO network model using the validation set to obtain the optimized PCS-YOLO network model. Evaluate the performance of the optimized PCS-YOLO network model using the test set to verify its detection accuracy and robustness, and save the final version of the PCS-YOLO network model.

[0013] Furthermore, in step S3, the CSP-TFSA module includes two parallel spatial domain branches and a frequency domain branch. The detailed steps for feature extraction by the CSP-TFSA module are as follows: Step S31: Input Features: Input the feature map X output by the Backbone network; Step S32, Spatial Domain Branching: The feature map X is convolved with the Scharr kernel K respectively. x K y Perform convolution to obtain the horizontal gradient response G. x With the gradient response G in the vertical direction y Among them, G x =X×K x G y =X×K y Calculate the overall gradient magnitude G, G = |G x | + |G y | The spatial domain edge features F are obtained through 3×3 convolution and residual connection. spatial ; Step S33, Frequency Domain Branching: Perform a 2D Fast Fourier Transform on the feature map X to obtain the frequency domain features. After learnable frequency domain filtering and inverse transform, obtain the frequency domain texture enhancement features F. freq ; Step S34, Feature Fusion: The spatial domain edge features F output by the spatial domain branching process are fused together. spatial Frequency domain texture enhancement feature F output from frequency domain branching processing freq The first fusion feature F is obtained by adding elements one by one. fusion ,Right now First fusion feature F fusion The feature map is enhanced by channel rearrangement and 1×1 convolution to improve the representation of edge texture.

[0014] Furthermore, in step S4, the PSCA attention module includes a first attention branch, a second attention branch, and a third attention path in parallel. The first attention branch is a channel attention branch. It aggregates global information using global average pooling and global max pooling, and obtains channel attention weights after activation by a shared multilayer perceptron and a sigmoid function. The output channel recalibration feature F is then generated. channel ; The second attention branch is a coordinate attention branch. It performs one-dimensional average pooling along the height and width directions, and obtains coordinate attention weights after concatenation, convolution, splitting, and sigmoid activation. The output is the coordinate-aware feature F. coordl ; The third attention branch is a spatial attention branch. It performs global average pooling and global max pooling along the channel dimension, then concatenates the results. A 7×7 large kernel convolution is used to generate a spatial mask, which is then activated by a sigmoid function to obtain spatial attention weights. The spatial saliency feature F is then output. spatial ; The outputs of the three attention branches are weighted and summed using learnable fusion weights to obtain the second fusion feature F. out , Where α, β, and γ are adaptive learning weights, and satisfy α + β + γ = 1, the final multi-scale feature F is obtained through dynamic residual gating. final , , where δ is the learnable gating coefficient.

[0015] Furthermore, in step S5, the SCSA-Head detection module adopts a cross-scale parameter sharing mechanism, enabling the feature maps of the three scales of P3, P4, and P5 layers to share the same set of depthwise separable convolutions for classification and regression prediction; the depthwise separable convolution includes two steps: channel-wise convolution and pointwise convolution.

[0016] Furthermore, in step S5, a learnable scale-aware coefficient is introduced to calibrate the output of the regression branch. The calibration formula is as follows: , i=3,4,5; where, F i For the feature map of the i-th layer, Convshared μ represents a shared depthwise separable convolution operation. i B represents the learnable scale-aware coefficient corresponding to the i-th layer. i This is the bounding box regression output corresponding to the feature map of the i-th layer.

[0017] Furthermore, the SCSA-Head detection module uses group normalization to divide the channel dimension into several groups on average. Within each group, the mean and variance are calculated independently for normalization. The calculation formula is as follows: ,in, As input features, The mean of the group. The within-group standard deviation To prevent small constants from being divided by zero, A learnable scaling factor. A learnable translation factor. This is the output feature after normalization.

[0018] Furthermore, the SCSA-Head detection module includes two parallel output branches: a regression branch and a classification branch. The regression branch uses distributed focus loss to model the bounding box coordinates as a discrete probability distribution, and then maps them to continuous coordinates through integral mapping to model the positioning uncertainty of irregular rockfall boundaries. The classification branch outputs the confidence probability of the rockfall target through the Sigmoid activation function.

[0019] Furthermore, in the frequency domain branch of the CSP-TFSA module, the complex spectrum is decomposed into real and imaginary parts and concatenated along the channel dimension to obtain real-valued frequency domain features. These features are then subjected to learnable frequency domain filtering via 3×3 convolution, and the receptive field of the convolution operation covers the entire feature map. The network automatically learns to enhance the frequency domain components corresponding to the rockfall contour and suppress the frequency domain components corresponding to the background noise through backpropagation.

[0020] Another object of the present invention is to provide a landslide and rockfall detection system based on deep learning, comprising: The image acquisition and preprocessing module is configured to acquire surveillance images and perform standardized preprocessing. The PCS-YOLO model inference module is configured to load the pre-trained PCS-YOLO network model weights, perform forward inference on the pre-processed image, and sequentially call subsequent modules to complete rockfall detection. The frequency-spatial domain collaborative feature extraction module is configured to process in parallel through spatial domain branches and frequency domain branches, and then fuse the outputs of the two branches to obtain a feature map that enhances the representation of edge texture. The parallel attention feature fusion module is configured to model the attention mechanism in parallel from three dimensions: channel, coordinate, and space. The parallel attention feature fusion module includes three parallel branches: the channel attention branch aggregates global information to obtain channel recalibration features, the coordinate attention branch captures position information to obtain coordinate-aware features, and the spatial attention branch generates a spatial mask to obtain spatial saliency features. The three outputs are adaptively weighted and fused to output a multi-scale feature map. The lightweight shared detection and localization module is configured to use a cross-scale parameter sharing mechanism to enable feature maps of multiple scales to share the same set of depthwise separable convolutions, introduce learnable scale-aware coefficients to calibrate the output distribution, and use group normalization to adapt to mini-batch inference, and finally output bounding box coordinates and their confidence scores. The results output and early warning module is configured to post-process the detection results and trigger early warnings.

[0021] Compared with existing technologies, the beneficial effects of the landslide rockfall detection method and system based on deep learning of this invention are as follows: 1. Solving the visual camouflage problem through deep synergy between frequency domain texture enhancement and spatial edge extraction: The designed CSP-TFSA module achieves complementary representation of spatial structure and frequency domain texture through parallel processing and fusion of spatial domain branch (Scharr operator to extract explicit edge information) and frequency domain branch (fast Fourier transform to enhance high-frequency texture details). The high-recognition feature map output by the CSP-TFSA module provides a high-quality feature foundation for the subsequent PSCA attention module, enabling the attention mechanism to perform weighted filtering on clearer edge texture features. The positive loop of the two enables the model to effectively distinguish between the target and the background in visual camouflage scenarios where rocks and backgrounds are highly similar, significantly reducing the false negative rate. 2. Accurate full-scale capture, cascading amplification effect of feature enhancement and multi-path attention fusion: To address the challenge of detecting multi-scale targets (from small pebbles to massive landslides), the CSP-TFSA module first enhances the edge texture representation of small targets, solving the problem of losing details in deep networks. Subsequently, the PSCA three-path parallel attention module (channel attention to filter key features, coordinate attention to explicitly encode position information, and spatial attention to generate spatial masks) adaptively weights and fuses these features. The two work together to form a cascading amplification effect of feature enhancement → attention focus → stronger features, enabling the model to accurately locate large-scale landslides as well as capture small pebbles with extremely low pixel counts, achieving robust detection of targets across all scales. 3. Achieving both accuracy and efficiency, the lightweight detection head frees up computing power for the front-end enhancement module: The CSP-TFSA module and PSCA attention module improve model accuracy through frequency domain processing and multi-path parallel attention mechanism. The designed SCSA-Head detection module enables feature maps of three scales to share the same set of depthwise separable convolutions through cross-scale parameter sharing, which greatly reduces the number of parameters and computation at the detection end. This collaborative strategy of enhancing front-end accuracy and optimizing back-end efficiency allows the model to maintain excellent performance indicators while keeping it lightweight, perfectly balancing detection accuracy and the real-time requirements of edge device deployment. 4. Robust Enhancement in Complex Environments: A Three-Tier Progressive Anti-Interference Link: From image input to detection output, three modules work together to form a complete anti-interference link: The CSP-TFSA module enhances texture details through frequency domain processing and is naturally insensitive to changes in illumination; the PSCA attention module's spatial attention mechanism generates a spatial mask, effectively suppressing complex background interference such as rain, fog, and tree occlusion; the SCSA-Head detection module's group normalization mechanism adapts to small-batch inference, ensuring inference stability. The above three modules progress step by step and cooperate with each other, enabling the model to maintain high detection robustness in complex and difficult scenarios such as severe weather, uneven illumination, and cluttered backgrounds. 5. End-to-end real-time early warning and fully collaborative automated monitoring closed loop: The three modules form a complete problem-solving technology chain from feature extraction, feature enhancement and fusion to efficient detection: When the input end encounters a disguised rockfall → the CSP-TFSA module enhances the texture edge → the PSCA attention module focuses on key areas and fuses multi-scale features → the SCSA-Head detection module efficiently outputs the detection results. After being deployed on edge devices, the system can process the monitoring video stream in real time. When the confidence level of the rockfall exceeds the threshold, it immediately triggers on-site audible and visual alarms, monitoring screen annotations, remote message pushes, and local log recording. Compared with traditional manual inspection, this invention achieves all-weather, wide-coverage automated monitoring and second-level emergency response. Attached Figure Description

[0022] Figure 1 This is a landslide rockfall detection method based on deep learning, which is an embodiment of the present invention. Figure 2 This is an embodiment of the present invention: a method for constructing and training a PCS-YOLO network model. Figure 3 This is a block diagram of the landslide and rockfall detection logic based on deep learning in Embodiment 1 of the present invention; Figure 4 This is a logical block diagram of the PCS-YOLO network model construction in Embodiment 1 of the present invention; Figure 5 This is a logic block diagram for locating the SCSA-Head detection module in Embodiment 1 of the present invention. Detailed Implementation

[0023] Example 1: Please refer to Figures 1-5 This embodiment is a landslide rockfall detection method based on deep learning, which includes the following steps S1 to S5. Steps S1 to S5 are described in detail below.

[0024] Step S1, Image Acquisition and Preprocessing: Acquire images captured in real time by surveillance cameras deployed in complex mountainous scenarios, and then preprocess the images. Specifically, the image acquisition method is as follows: Install a camera on a steep slope along a mountain highway to capture images in real time. For example, the slope is approximately 80 meters high and has a gradient of approximately 65°. The camera has 4 megapixels and is mounted on a fixed bracket opposite the slope, approximately 150 meters away, with a downward angle of approximately 15°. In other embodiments, the specific installation parameters and location of the camera can be adjusted according to the actual situation and are not limited here. The camera is connected to an edge computing device via a fiber optic transceiver to capture a video stream in real time at a preset frame rate (e.g., 25fps). The resolution of the video stream is 1920×1080 pixels or higher, such as 2K or 4K pixels. The specific image preprocessing method is as follows: use a video stream reading module (such as Video Capture in OpenCV) to read the acquired video stream, perform image scaling processing on each frame of the image, such as scaling the resolution from 1920×1080 to 640×640, converting the BGR color space to the RGB color space, then normalizing the pixel values ​​to the [0,1] range, and finally converting the image data into the tensor format required by the deep learning model, such as converting it into the PyTorch tensor format, with a tensor dimension of 3×640×640, where 3 represents the number of channels and 640 represents the height and width.

[0025] Step S2, Model Invocation: The preprocessed image is input into the pre-built and trained PCS-YOLO network model. The PCS-YOLO network model is a lightweight convolutional neural network, and its network weight file is pre-stored in the storage unit of the edge computing device. Specifically, the image preprocessed in step S1 (e.g., tensor format 3×640×640) is input into the trained PCS-YOLO network model to prepare for subsequent feature extraction and object detection.

[0026] Step S3, Frequency-Spatial Domain Collaborative Feature Extraction: The PCS-YOLO network model performs forward inference on the input image and extracts image features through the Backbone network to obtain an initial feature map. The initial feature map is then processed in two parallel steps: the first step is to use the Scharr operator to extract explicit edge information, resulting in spatial domain edge features; the second step is to map the features to the frequency domain using a Fast Fourier Transform to enhance texture details, followed by an inverse transform to restore the features, resulting in frequency domain texture enhancement features. The spatial domain edge features and the frequency domain texture enhancement features are then fused to output a feature map that enhances edge texture representation. Specifically, image features are extracted using a backbone network to obtain an initial feature map, which is then input into the CSP-TFSA module. The CSP-TFSA module comprises two parallel branches: a spatial domain branch and a frequency domain branch. The spatial domain branch uses the Scharr operator to extract explicit edge features, resulting in spatial domain edge features. The frequency domain branch maps these features to the frequency domain using Fast Fourier Transform (FFT) to enhance texture details, followed by Inverse Fast Fourier Transform (IFFT) to restore the features and obtain frequency domain texture enhancement features. The spatial domain edge features and frequency domain texture enhancement features are then fused to output a feature map that enhances edge texture representation. By enhancing high-frequency texture details through the frequency domain branch, the CSP-TFSA module can capture subtle differences in texture between the surface of a fallen rock and the background rocks, effectively distinguishing the target from the background.

[0027] The following example illustrates the specific workflow of the CSP-TFSA module during the testing phase: Step S31: Input features: Input the feature map X output by the Backbone network. Taking the medium-scale feature as an example, the size of the feature map X is 40×40×256. Step S32, Spatial Domain Branching: The feature map X is convolved with the Scharr kernel K respectively. x K y Perform convolution to obtain the horizontal gradient response G. x With the gradient response G in the vertical direction y Among them, G x =X×K x G y =X×K y Calculate the overall gradient magnitude G = |G x | + |G y | The spatial domain edge features F are obtained through 3×3 convolution and residual connection. spatial ; Step S33, Frequency Domain Branching: Perform a 2D Fast Fourier Transform on the feature map X to obtain the frequency domain features. After learnable frequency domain filtering and inverse transform, obtain the frequency domain texture enhancement features F. freq ; Step S34, Feature Fusion: The spatial domain edge features F output by the spatial domain branching process are fused together. spatial Frequency domain texture enhancement feature F output from frequency domain branching processing freq The first fusion feature F is obtained by adding elements one by one. fusion ,Right now First fusion feature F fusion The feature map is enhanced by channel rearrangement and 1×1 convolution to improve the representation of edge texture.

[0028] Step S4, Parallel Attention Feature Fusion: The feature map with enhanced edge texture representation output from Step S3 is input into the Neck network, and three parallel attention processes are performed: The first attention process is to perform global average pooling and global max pooling on the feature map to aggregate global information and obtain channel recalibration features; the second attention process is to perform pooling along the height and width directions of the feature map to generate orientation-aware feature maps to capture position information and obtain coordinate-aware features; the third attention process is to use large kernel convolution to perform spatial convolution on the feature map to generate a spatial mask and obtain spatial saliency features, which highlight the target region and suppress background interference; the channel recalibration features, coordinate-aware features, and spatial saliency features are adaptively weighted and fused to output the fused multi-scale feature map. Specifically, the enhanced edge texture representation feature map output by the CSP-TFSA module in step 3 is input into the Neck network. A PSCA attention module is embedded in the Neck feature fusion layer to perform multi-scale feature fusion. The PSCA attention module contains three parallel channel attention branches, coordinate attention branches, and spatial attention branches. The channel attention branches aggregate global information using global average pooling (GAP) and global max pooling (GMP) to obtain channel recalibration features. The coordinate attention branches generate orientation-aware feature maps along the height and width directions. Location information is captured to obtain coordinate-aware features. The spatial attention branch uses 7×7 large kernel convolution to generate a spatial mask and obtain spatial saliency features. The channel recalibration features, coordinate-aware features and spatial saliency features are weighted and fused to obtain a fused multi-scale feature map. For example, the fused multi-scale feature map consists of three scales: a shallow high-resolution feature map (P3 layer, 80×80×128), a medium-resolution feature map (P4 layer, 40×40×256), and a deep low-resolution feature map (P5 layer, 20×20×512).

[0029] Step S5, Lightweight Shared Detection and Localization: Input the multi-scale feature map output from step S4 into the SCSA-Head detection module. The SCSA-Head detection module adopts a cross-scale parameter sharing mechanism, which enables the multi-scale feature maps to share the same set of depthwise separable convolutions (DSC) for classification and regression prediction. It also introduces learnable scale-aware coefficients to calibrate the output distribution of the multi-scale feature maps and outputs the bounding box coordinates and confidence of the falling rock target. Specifically, the three scale feature maps (P3 layer 80×80×128, P4 layer 40×40×256, P5 layer 20×20×512) output from step S4 are input into the SCSA-Head detection module. In order to reduce the number of parameters to adapt to edge devices, the SCSA-Head detection module adopts a cross-scale parameter sharing mechanism, that is, the feature maps of the three scales share the same set of depthwise separable convolutions for classification and regression prediction, which can further reduce the number of parameters. At the same time, depthwise separable convolution (DSC) is used to further reduce the amount of computation. The standard convolution is decomposed into two steps: channel-wise convolution and pointwise convolution, which further reduces the number of parameters. To prevent instability caused by the traditional batch normalization due to the typically 1-sized batch during inference on edge devices in the field, the SCSA-Head detection module uses group normalization instead of batch normalization. Taking the P3 layer as an example, the 128 channels are divided into 8 groups of 16 channels each. The mean and standard deviation are calculated independently for each of the 102,400 pixels (80×80×16) within each group for normalization. Then, a learnable scaling factor γ and a translation factor β are used to recover the feature representation, ensuring stability even with a batch size of 1. Since the same regressor is used across the three scales, but the feature maps at different scales correspond to different target sizes, each feature point in the P3 layer corresponds to an 8×8 pixel region in the original image, suitable for small targets (small offset), while each feature point in the P5 layer corresponds to a 32×32 pixel region in the original image, suitable for large targets (large offset). Therefore, directly using the same output would lead to bias. Thus, a learnable scale-aware coefficient μ is introduced. i Calibration is performed, as shown in the example below. Assuming the shared convolution output bounding box offset is (0.10, 0.10, 0.20, 0.20), the P3 layer multiplied by μ3=0.85 yields (0.085, 0.085, 0.17, 0.17), and the P5 layer multiplied by μ5=1.28 yields (0.128, 0.128, 0.256, 0.256), thus adapting to output distributions at different scales. Finally, the SCSA-Head detection module outputs the normalized bounding box coordinates of each detection box, such as center point x=0.35, center point y=0.42, width=0.12, height=0.09, and the rockfall confidence score, such as 0.92, indicating a 92% probability of a rockfall.

[0030] Step S6, Result Output and Warning: When the confidence level of the falling rock target exceeds the preset threshold, the bounding box coordinates of the falling rock target are output and a warning signal is triggered to realize end-to-end real-time detection and warning of landslides and falling rocks in complex mountainous scenes. Specifically, the detection results output in step S5 are post-processed. First, it is determined whether the confidence level of each detection box exceeds a preset threshold. In this embodiment, the preset threshold is 0.5. Detections below the preset threshold are considered false detections and discarded, while those above the preset threshold are considered valid rockfalls and retained. Then, non-maximum suppression is applied to the retained detection boxes to remove redundant boxes with high overlap. Next, the normalized bounding box coordinates are converted to absolute coordinates in the original image coordinate system. For example, the normalized coordinates (0.35, 0.42, 0.12, 0.09) correspond to the actual bounding box (556.8, 405.0, 787.2, 502.2) in a 1920×1080 image. Finally, the detection results are output and an early warning signal is triggered, including on-site audible and visual alarms, monitoring screen annotations, remote message pushes, and local log recording, thereby achieving end-to-end real-time detection and early warning of landslides and rockfalls in complex mountainous scenes.

[0031] The construction and training method of the PCS-YOLO network model includes the following steps T1 to T4, which are explained in detail below.

[0032] Step T1, Data Acquisition and Processing: Acquire landslide and rockfall images in complex mountainous scenes, and construct a rockfall image dataset containing various scene types. The dataset includes a training set, a validation set, and a test set. Step T1 is explained in further detail below.

[0033] (1) Data sources for landslides and rockfalls: Images from surveillance cameras in real and complex mountainous areas were collected. The image sources included: surveillance cameras along the slopes of a highway in a mountainous area (several locations), monitoring stations on steep slopes in a hydropower reservoir area (several locations), and publicly available rockfall disaster image datasets (including publicly available resources on the Roboflow platform and publicly available datasets in literature). The datasets covered the four seasons of spring, summer, autumn, and winter, and included various weather conditions such as sunny, rainy, foggy, and snowy days, as well as various lighting conditions such as daytime, dusk, and nighttime. A total of several original images were collected, for example, 3,500 images. After all images were manually cleaned and images with severe motion blur, abnormal exposure, or incorrect labeling were removed, 2,039 high-resolution images were retained, with a resolution range of 1280×720 to 1920×1080.

[0034] (2) Labeling the collected data: Manually label the retained images using tools, such as the LabelImg tool, labeling the target category as "Rockfall" in PASCAL VOC format (XML file), including the coordinates of the top left corner of the bounding box (x). miny min ) and the coordinates of the lower right corner (x max y max Each image's annotation is completed independently by multiple professional annotators. For images with inconsistent annotations, senior experts review and correct them to ensure annotation quality.

[0035] Specifically, the following three types of difficult samples should be given special attention during annotation: (a) Visual camouflage type: The color of the falling rocks is extremely close to that of the surrounding rocks and soil, and the texture is highly similar, making it difficult to distinguish with the naked eye. This type is one of the core challenges of the present invention. The detection models in the prior art have a very high false negative rate in this type of scenario. (b) Small target class: These targets have a very low pixel ratio in wide field of view images and an area smaller than 32×32 pixels. These samples are prone to losing detailed information after multi-level sampling. (c) Complex background: Due to the influence of trees, rainy or foggy weather or uneven lighting, the background interference of this type of sample is strong and the distinction between the target and the background is low.

[0036] (3) Dataset partitioning: The dataset is divided into three categories according to the scenario type: Class I dataset: Contains four types of images: simple background rocks, camouflaged rocks, small rocks, and normal background, totaling 1200 images. This dataset is used for basic training of the model, enabling the model to learn the basic visual features of rocks. Type II Hard Sample Dataset: Contains 600 images of extreme environments such as rainy and foggy weather, strong light and shadow. This type of dataset is used for training the model on hard samples to improve the model's robustness in harsh environments. Class III Validation Dataset: Used for final performance validation of the model, consisting of 239 images. This dataset is independent of the training process and is used only for final evaluation to ensure the objectivity of the evaluation results. All images from categories I, II, and III were randomly divided into a training set (1631 images), a validation set (204 images), and a test set (204 images) in a ratio of 8:1:1.

[0037] (4) Data augmentation: To improve the generalization ability of the model, online data augmentation strategies are used for the training set images, including: (a) Geometric transformations: random horizontal flip (e.g., probability 0.5), random rotation (e.g., ±15°), random scaling (e.g., 0.8 to 1.2 times); (b) Stitching enhancement: Image stitching, for example, using Mosaic stitching, randomly cropping 4 images and stitching them into 1 image to enhance the model's adaptability to changes in target scale; (c) Color jitter: Brightness, contrast and saturation are randomly adjusted, for example, by ±20%, to enhance the model's robustness to changes in lighting. (d) Noise injection: Gaussian noise (e.g., standard deviation 0.01) and salt-and-pepper noise (flux density 0.01) are added to enhance the model’s robustness to image noise.

[0038] Step T2, Network Model Construction: An initial PCS-YOLO network model is constructed based on the YOLO framework. The initial PCS-YOLO network model includes a Backbone network, a Neck network, and an SCSA-Head detection module. A CSP-TFSA module is introduced at the connection between the Backbone network and the Neck network to fuse spatial domain edge features and frequency domain texture enhancement features. A PSCA attention module is embedded in the Neck feature fusion layer to model in parallel from three dimensions: channel, coordinate, and space. The SCSA-Head detection module adopts a cross-scale parameter sharing mechanism, enabling multi-scale feature maps to share the same set of depthwise separable convolutions for classification and regression prediction. A learnable scale-aware coefficient is introduced to calibrate the output distribution of features at different scales. Specifically, the initial PCS-YOLO network model is built based on YOLOv11n, the lightest version of the YOLO series. The initial PCS-YOLO network model consists of three parts: the Backbone network, the Neck network, and the SCSA-Head detection module. The Backbone network is the backbone network, which adopts the CSP-Darknet structure and contains 5 CSP modules. Each CSP module contains a convolutional layer, a normalization layer, and a SiLU activation function. The Backbone network is responsible for multi-level downsampling of the input image to extract features at different semantic levels. The Neck network is the neck network, which adopts the PANet (Path Aggregation Network) structure and contains bidirectional feature fusion paths from top to bottom and bottom to top. It is responsible for combining deep semantic information with shallow detail information. The SCSA-Head detection module is a lightweight shared convolutional detection module, which is responsible for target localization and classification on the fused multi-scale feature map. A CSP-TFSA module is introduced at the connection between the Backbone and Neck networks. This module combines frequency domain enhancement of texture details with spatial domain extraction of edge features, addressing the issues of lost details in small-scale rockfalls and difficulty in distinguishing targets in visually camouflaged environments, thus strengthening the model's ability to represent edge textures. A PSCA attention module is embedded in the Neck network's feature fusion layer. Through a three-way parallel attention mechanism (channel, coordinate, and spatial), it overcomes the information decay defects of traditional serial attention, enhancing the model's perception of positional information and its resistance to interference from complex backgrounds. An SCSA-Head detection module is built at the detection end. This module significantly reduces the number of parameters and computational cost through cross-scale parameter sharing and depthwise separable convolutions. It calibrates the output distribution at different scales using learnable scale-aware coefficients and adapts to small-batch inference scenarios on edge devices through group normalization, achieving efficient deployment of the detection head. The organic combination of these three modules enables the model to maintain high detection accuracy while being lightweight, allowing deployment on computationally limited edge devices in the field, achieving end-to-end real-time detection and early warning of landslides and rocks in complex mountainous scenes.The CSP-TFSA module, PSCA attention module, and SCSA-Head detection module are explained in detail below.

[0039] The CSP-TFSA (Cross-Stage Partial Time-Frequency Spatial Attention) module is located at the connection between the Backbone and Neck networks. This module includes spatial and frequency domain branches, employing a dual-branch parallel structure to enhance the edge texture representation of small-scale rockfall targets. In traditional convolutional neural networks, feature extraction primarily operates in the spatial domain. After multiple convolutions and downsampling, the texture and edge information of small targets are easily lost, especially when the rocks and background are highly similar in color, i.e., when visual camouflage exists, spatial domain features struggle to distinguish the target from the background. This invention introduces frequency domain analysis into landslide rockfall detection. Its core principle is that in the frequency domain representation of an image, high-frequency components correspond to detailed information such as edges and textures, while low-frequency components correspond to coarse information such as overall structure and color. By mapping features to the frequency domain through Fast Fourier Transform, the high-frequency components are enhanced, highlighting the edge texture features of the rocks. Then, through inverse transform, the features are restored to the spatial domain and fused with spatial domain edge features, achieving a complementary representation of spatial structure and frequency domain texture.

[0040] Spatial Domain Branch (Boundary Awareness Enhancement): The spatial domain branch uses the Scharr operator to extract explicit edge features. The Scharr operator is an improved version of the Sobel operator, possessing better rotational symmetry and isotropy, and exhibiting superior gradient response to irregular rockfall boundaries. The Scharr convolution kernel is defined as K... x K y ,For example, , where K x The convolutional kernel is used for horizontal gradient detection and is sensitive to vertical edges; K y This is a vertical gradient detection convolution kernel, sensitive to horizontal edges. The input feature map X (size H×W×C, where H is height, W is width, and C is the number of channels) is compared with K... x and K y Perform a convolution operation to obtain the horizontal gradient response G. x and vertical gradient response G y Then calculate the overall gradient magnitude G. This formula uses the L1 norm to approximate the gradient magnitude, compared to the L2 norm ( It requires less computation and is suitable for edge deployment. The edge enhancement feature F is obtained by performing a non-linear mapping on the comprehensive gradient magnitude G through a 3×3 convolution. edge , Then, it is added to the input feature map X through residual connections to recover semantic information, resulting in the intermediate feature map F. edge-temp , The residual connection serves to prevent gradient vanishing while preserving the original semantic information of the input features. Finally, a 3×3 convolution completes the encoding to obtain the spatial domain edge features F. spatial , .

[0041] Frequency Domain Branch (Global Spectral Filtering): The frequency domain branch maps features to the frequency domain using a Fast Fourier Transform (FFT), enhancing texture details with high-frequency components. It performs a two-dimensional real-valued Fast Fourier Transform (2D rFFT) on the input feature map X (e.g., size H×W×C). The transformation formula is as follows: ,in, This represents the Fast Fourier Transform operation, whose output is a complex tensor containing the real part. and the virtual part The complex spectrum is decomposed into real and imaginary parts, and then concatenated along the channel dimension to obtain the real-valued frequency domain features. (Dimensions are H×W×2C) Learnable frequency domain filtering is performed via 3×3 convolution. This convolutional operation is equivalent to learning an adaptive global filter in the frequency domain. Compared to spatial domain convolution, whose receptive field is limited by the kernel size, the frequency domain filter covers the entire feature map, capturing global contextual information. Through backpropagation, the network can automatically learn which frequency domain components correspond to rockfall features (which should be enhanced) and which correspond to background noise (which should be suppressed). The spatial domain is then restored using the inverse real-valued fast Fourier transform (iRFFT). , ,in, This represents the inverse fast Fourier transform, which restores the frequency domain signal to the spatial domain signal. Further thinning using a 3×3 convolution yields the frequency domain texture enhancement feature F. freq , .

[0042] Cross-domain feature fusion: fusing spatial domain edge features F from the spatial domain branch output. spatial Frequency domain texture enhancement feature F with frequency domain branch output freq The first fusion feature F is obtained by adding elements one by one. fusion , Element-wise addition is the simplest fusion method, which not only has low computational cost but also maintains the size of the feature maps. Then, channel shuffle is used to promote cross-group information exchange. The specific operation of channel shuffle is: grouping channels, and rearranging features within each group, allowing features from different groups to be mixed and enhancing information flow. Finally, the first fused feature F... fusionChannel-dimensional fusion is performed using 1×1 convolutions to obtain the feature map that enhances edge texture representation from the output of the CSP-TFSA module. The purpose of the 1×1 convolution is to adjust the number of channels back to the original input number, facilitating subsequent network processing.

[0043] Traditional spatial domain convolution mainly aggregates local neighborhood information, with a limited effective receptive field, and high-frequency details are easily smoothed after multiple downsampling. This invention introduces the CSP-TFSA module. In visual camouflage scenarios, traditional models have difficulty distinguishing between targets and backgrounds. The CSP-TFSA module improves detection capabilities through the following mechanisms: (1) Frequency domain enhancement: High-frequency components highlight the subtle differences in texture between rocks and backgrounds, such as the crack texture of rocks and the angular texture of rocks; (2) Edge enhancement: Gradient features extracted by the Scharr operator highlight the contour information of rocks; (3) Dual-branch fusion: Spatial and frequency domain information complement each other to achieve joint representation of structure and texture. The global characteristics of Fourier transform are used to capture global long-range dependencies, while the high-frequency components in the frequency domain correspond to the edge, texture and other details of the image. Through learnable frequency domain filtering, the network can adaptively enhance the frequency domain components related to the contour of rocks and suppress background noise. The parallel processing of spatial and frequency domain dual branches and their subsequent fusion achieves complementary representation of spatial structure and frequency domain texture, which is particularly suitable for small-scale, low-salience, motion-blurred rock target detection.

[0044] The PSCA (Parallel Spatial-Coordinate Attention) attention module is embedded in the Neck feature fusion layer. The PSCA attention module comprises three parallel branches and employs adaptive fusion and dynamic residual gating mechanisms to overcome background camouflage by modeling in parallel across three dimensions: channel, coordinate, and spatial. Traditional attention mechanisms (such as CBAM and SE) typically use a channel-to-space sequential structure. This sequential structure has two problems: first, channel attention and spatial attention are processed sequentially, and the attention from the later processing can overwrite the results of the earlier processing, leading to information attenuation; second, the sequential structure cannot simultaneously capture feature dependencies of different dimensions. The PSCA attention module proposed in this invention employs three parallel attention branches, simultaneously modeling channel semantics, orientation-aware position, and spatial saliency, and resolves the response conflict problem between different attention branches through adaptive fusion weights.

[0045] The first attention branch is channel attention: the channel attention branch aggregates global information using global average pooling (GAP) and global max pooling (GMP). For example, for input features... Global average pooling and global max pooling are performed separately to obtain the global average pooling result. and global max pooling results , and All are C-dimensional vectors. Global average pooling averages the value of all pixels in each channel, reflecting the overall response level of the channel; global max pooling maximizes the value of all pixels in each channel, reflecting the strongest response of the channel. Global average pooling preserves the overall response of the features, while global max pooling preserves the most significant response; the two are complementary. and Both are input into a shared multilayer perceptron, for example, into an MLP (Multilayer Luminaire), which contains a hidden layer and a compression factor. The structure of an MLP is: linear layer (C → C / r) → ReLU activation → linear layer (C / r → C). This converts the two outputs of the MLP... The result after MLP processing and The result after MLP processing The elements are added together, and then the channel attention weights are obtained by passing them through the Sigmoid activation function. , Sigmoid function The output values ​​are compressed to the (0,1) interval and then output to represent the importance weight of each channel. The output of the channel attention branch is the channel recalibration feature F. channel , ,in, This indicates element-wise multiplication. The role of the channel attention branch is to model the correlation between channels, filter key feature maps, preserve texture details, and suppress irrelevant background noise.

[0046] The second attention branch is coordinate attention: This branch generates orientation-aware feature maps along the height and width directions to capture positional information. To compensate for the weakening of positional information caused by global pooling, one-dimensional average pooling is performed separately along the height and width directions to obtain height-oriented features. and width direction features Then z h and z w By concatenating the features in the spatial dimension, a feature map of size is obtained. After dimensionality reduction via 1×1 convolution, the data is further split into two attention tensors, one horizontal and one vertical, using a non-linear activation function (ReLU). These tensors are then subjected to 1×1 convolution to recover the number of channels C, and finally, coordinate attention weights are obtained through Sigmoid activation. and Finally, the coordinate attention branch outputs coordinate-aware features F. coordl , The multiplication operation is implemented through a broadcast mechanism. The coordinate attention branch explicitly incorporates spatial location information, improving the ability to locate small-scale, dense rockfalls.

[0047] The third attention branch is spatial attention: This branch generates a spatial mask using a 7×7 large kernel convolution. Global average pooling and global max pooling are then performed along the channel dimension to obtain the average-pooled spatial descriptor. and max pooling space descriptor ,Will and The two are spliced ​​together to obtain The feature map is used to generate a spatial mask using 7×7 large kernel convolution. , Spatial attention weights are obtained after Sigmoid activation. , The output of the spatial attention branch is the spatial saliency feature F. spatial , The spatial attention branch is responsible for locating salient regions in the image, highlighting the target area, and suppressing interference from complex backgrounds. Compared to 3×3 convolution, 7×7 large kernel convolution has a larger receptive field, providing a larger receptive field while maintaining a relatively small increase in computation, effectively capturing the global spatial context information of the falling rock target, thereby generating a more accurate spatial attention mask.

[0048] Adaptive weighted fusion and dynamic residual gating: The outputs of the three attention branches are weighted and summed through learnable fusion weights to obtain the second fusion feature F. out , Here, α, β, and γ are adaptively learned weights, automatically learned through a lightweight fusion network. The structure of the fusion network is: global average pooling → fully connected layer (3D output) → softmax function, where the softmax function satisfies α + β + γ = 1. This mechanism allows the network to adaptively adjust the contribution of the three attention pathways according to different input features. For example, for scenes with rich textures, the weight α of channel attention may be larger; for scenes with dense targets, the weight β of coordinate attention may be larger; and for scenes with cluttered backgrounds, the weight γ of spatial attention may be larger. Simultaneously, a dynamic residual gating mechanism is introduced for the second fusion feature F. out After adjustment, the multi-scale feature F is obtained. final , Where δ is a learnable gating coefficient, ranging from 0 to 1, used to adjust the residual injection strength. Setting a minimum residual threshold of δ, such as 0.1, can prevent gradient vanishing. X is the input feature. The role of dynamic residual gating is: when the output of the attention module is the second fused feature F... outWhen the quality is high, the network can learn a larger amount of data. Value, enhancing the influence of residual connections; when the second fusion feature F out When the quality is poor, the network can learn a smaller amount. This reduces the impact of residual joins. This mechanism is better than fixing residual joins ( More flexible.

[0049] After processing by the PSCA attention module, the Neck network outputs feature maps at three scales: P3 layer (e.g., 80×80×128, shallow, high resolution, suitable for small targets), P4 layer (e.g., 40×40×256, medium resolution), and P5 layer (e.g., 20×20×512, deep, low resolution, suitable for large targets), for use by the subsequent SCSA-Head detection module. In complex background scenes, such as dense vegetation and messy rock textures, the PSCA attention module improves detection capabilities through the following mechanisms: (1) Channel attention: filtering feature channels related to falling rocks, such as edge and texture channels, which can suppress background-related channels; (2) Coordinate attention: explicitly encoding location information, enabling the model to know which area of ​​the image the falling rocks appear in, thus improving positioning accuracy; (3) Spatial attention: generating a spatial mask to highlight the falling rock area and suppress the background area; (4) Parallel fusion: processing three types of information simultaneously to avoid information decay in serial structures. The PSCA attention module proposed in this invention adopts a three-way parallel structure, simultaneously modeling channel semantics, orientation-aware position, and spatial saliency, and solving the response conflict problem of different attention branches through adaptive fusion weights. The coordinate attention branch is particularly innovative, explicitly encoding location information, which significantly improves the positioning accuracy of small-scale dense falling rocks.

[0050] The SCSA-Head (Spatial-Coordinate Shared Attention Head, Lightweight Shared Convolution) detection module is one of the core innovative modules of this invention, used for target localization and recognition of feature maps at three scales: P3, P4, and P5. Traditional decoupled head detection heads typically configure independent convolutional towers and prediction branches for each layer of the feature pyramid. While this facilitates independent modeling across scales, it easily leads to parameter redundancy. This invention employs a cross-scale parameter sharing mechanism, allowing feature maps at the P3, P4, and P5 layers to share the same set of depthwise separable convolutions for classification and regression prediction. The depthwise separable convolutions decompose standard convolutions into two steps: Step A, Depthwise Convolution: Perform convolution independently on each input channel. For a feature map X (size H×W×C, where H is height, W is width, and C is the number of channels), the number of input channels is C, and the kernel size is K.x ×K y The number of output channels is also C, and the number of parameters is C×K. x ×K y The computational complexity is C×K x ×K y ×H×W; Step B, Pointwise Convolution: 1×1 convolution is used for channel fusion, with C×C' parameters (C' is the number of output channels). C×C'×K x ×K y .

[0051] The number of parameters is C×C'×K compared to standard convolution. x ×K y The computational complexity is C×C'×K x ×K y ×H×W, the number of parameters for depthwise separable convolution is reduced by approximately 1 / C'+1 / (K). x ×K y ) times. In this embodiment, a 3×3 depthwise separable convolution is used. K x =3,K y =3, parameter count reduced by approximately .for The number of parameters is reduced by approximately 98.5%. The specific implementation of shared convolution involves defining a shared depthwise separable convolutional module. The same convolution operation is applied sequentially to the feature maps of layers P3, P4, and P5. Since the feature maps of the three layers have different sizes (80×80, 40×40, and 20×20), the convolution operation is performed independently in each layer, but the same convolution kernel weights are used.

[0052] Although the cross-scale parameter sharing mechanism significantly reduces the number of parameters in the SCSA-Head detection module, the feature maps at the P3, P4, and P5 scales have inherent differences. To address this issue, this invention introduces a learnable scale-aware coefficient for each scale to calibrate the output of the regression branch. The specific formula is as follows: , i=3,4,5, where F i For the feature map of the i-th layer, Conv shared μ represents a shared depthwise separable convolution operation. i Let B be the learnable scale perception coefficient corresponding to the i-th layer (initial value set to 1.0). i μ is the bounding box regression output corresponding to the feature map of the i-th layer. i As learnable parameters of the network, they are automatically optimized during training through backpropagation. This mechanism allows for the sharing of Conv convolutional layers. sharedIt is responsible for extracting a unified feature representation, while the scaling coefficient μ i The two are responsible for adapting the output distribution to different scales. Their division of labor and cooperation not only ensures the efficiency of parameter sharing, but also solves the scale sensitivity problem.

[0053] Field edge monitoring devices typically employ a small-batch inference mode. Limited by the computing power and memory of these edge devices, the batch size is usually set to 1 (single-frame inference). Traditional batch normalization (BN) suffers from serious problems in this scenario. BN calculates the mean and variance on a single batch of data. When the batch size is 1, the statistics for the mean and variance are highly unstable. Furthermore, BN uses different statistics for training and inference (using the statistics of the current batch during training and the global statistics during inference), potentially leading to performance discrepancies. To address this issue, this invention employs group normalization (GN) instead of traditional batch normalization (BN). Group normalization (GN) is independent of the batch dimension, performing group normalization on the channel dimension of the same sample, regardless of the batch size. The formula for group normalization is: ,in: As input features, The mean of the group. The within-group standard deviation To prevent division by zero, a small constant (usually taken as...) ), This is a learnable scaling factor (initially 1). It is a learnable translation factor (initially 0). This represents the normalized output features. In this embodiment, the number of groups is set to 8, meaning the channel dimension is divided into 8 groups on average, and the mean and variance are calculated independently within each group.

[0054] The SCSA-Head detection module comprises two parallel output branches: a regression branch and a classification branch. The regression branch predicts bounding box coordinates, while the classification branch predicts the confidence level of the rockfall target. The regression branch uses the integral form of Distribution Focal Loss (DFL) to map the discrete probability distribution to continuous bounding box coordinates. Traditional regression methods directly predict bounding box coordinate values, which has limited ability to model irregular rockfall boundaries. Distribution Focal Loss (DFL) models the bounding box coordinates as a probability distribution, obtaining continuous values ​​through integration. , ,in, For the first The probability of a discrete location. These are the corresponding coordinate values. In this embodiment, the bounding box coordinates are discretized into 16 positions from 0 to 15. A probability distribution is obtained through the Softmax activation function, and then integrated to obtain continuous coordinates. Distributed Focal Loss (DFL) can better model the positioning uncertainty of irregular rockfall boundaries. The classification branch outputs the class probability p through the Sigmoid activation function. Output value , indicating the probability that the detection box contains falling rocks.

[0055] The SCSA-Head detection module improves detection efficiency through the following mechanisms: (1) Parameter sharing: The three-layer SCSA-Head detection module shares the same set of convolution kernels, which significantly reduces the number of parameters; (2) Depth-separable convolution: further reduces the number of parameters and computational cost; (3) Scale-aware coefficient: solves the scale sensitivity problem caused by shared weights; (4) Group normalization: adapts to small batch inference and improves training stability.

[0056] Step T3, Model Training: Train the initial PCS-YOLO network model using the training set, and optimize it using the SGD optimizer to obtain the pre-trained PCS-YOLO network model and its initial model weights. Specifically, train the PCS-YOLO network model using the training set. Hardware environment: NVIDIA RTX 4090 GPU (24GB VRAM), Intel i9-13900K CPU, 64GB RAM. Software environment: Ubuntu 20.04 operating system, Python 3.10, PyTorch 2.3, CUDA 12.1. Training hyperparameter settings: image size 640×640 pixels; batch size 16; optimizer SGD (Stochastic Gradient Descent), momentum 0.937, weight decay 0.0005; initial learning rate 0.01, using Cosine Annealing Scheduler, final learning rate decayed to 0.0001; training epochs 300; warmup epochs 3, learning rate linearly increased from 0.001 to 0.01 during the warmup phase; classification loss using Binary Cross-Entropy Loss (BCE Loss), regression loss using CIoU Loss. Complete IoU Loss comprehensively considers overlap area, center distance, and aspect ratio, making it more accurate than traditional IoU Loss.

[0057] Training process: In each epoch, the 1631 images of the training set are input into the model in batches for forward inference to calculate the loss, and then the gradient is calculated by backpropagation. The network weights are updated using the SGD optimizer. After each epoch, mAP@0.5 and mAP@0.5:0.95 are calculated on the validation set (204 images), and the optimal model weights are saved. The training is carried out for a total of 300 rounds, which takes about 24 hours.

[0058] Convergence analysis: In the first 50 training epochs, the loss function decreased rapidly, and the learning rate entered the cosine annealing stage after warming up from 0.01; from epochs 50 to 150, the loss decreased slowly and tended to stabilize, and the model began to learn difficult samples; from epochs 150 to 300, the loss was basically stable, with no signs of overfitting, and the training loss and validation loss curves were highly consistent, with a difference of <0.05. Ultimately, the best mAP@0.5 on the validation set reached 87.1%, and the mAP@0.5:0.95 ratio reached 71.2%.

[0059] Step T4, Model Validation and Testing: The pre-trained PCS-YOLO network model is validated and its parameters are tuned using the validation set to obtain an optimized PCS-YOLO network model. The optimized PCS-YOLO network model is then evaluated using the test set to verify its detection accuracy and robustness. The final version of the PCS-YOLO network model with good performance is saved. Specifically, the optimized model is evaluated using the test set (204 images), and the evaluation metrics include: (1) Precision: Where TP is the number of correctly predicted positive examples, FP is the number of incorrectly predicted positive examples, and precision measures how many of the model's predicted positive examples are correct. (2) Recall: FN represents the number of missed positive examples, while recall measures how many true positive examples the model finds.

[0060] (3) Average Precision (AP): , which is the area under the PR curve, and mAP is the mean of AP for all categories (in this embodiment, there is only one category, so mAP = AP).

[0061] (4) mAP@0.5: The average accuracy when the IoU threshold is 0.5. IoU (Intersection over Union) is the intersection-union ratio between the predicted bounding box and the ground truth bounding box. An IoU ≥ 0.5 is considered a correct test result.

[0062] (5) mAP@0.5:0.95: The average accuracy over 10 thresholds with an IoU threshold of 0.5 to 0.95 and a step size of 0.05. This metric is more stringent, requiring the predicted box to be highly consistent with the ground truth box.

[0063] (6) F1 score: The harmonic mean of precision and recall comprehensively reflects the detection performance.

[0064] (7) Parameters: The total number of trainable parameters of the model, which measures the size of the model.

[0065] (8) GFLOPs: Billions of floating-point operations per second, a measure of computational complexity. 1 GFLOP = The floating-point operations, which quantify the space complexity (model size) and time complexity (computation amount per inference), are key indicators for verifying the feasibility of real-time monitoring in embedded terminals.

[0066] In this embodiment, some evaluation metrics after testing are as follows: accuracy P is 92.5%, mAP@0.5 is 87.1%, mAP@0.5:0.95 is 71.2%, F1 score is 81.5%, number of parameters is 3.2M, and GFLOPs are relatively low or even slightly low. This indicates that the CSP-TFSA module, PSCA attention module, and SCSA-Head detection module used in the constructed PCS-YOLO network model bring significant accuracy gains.

[0067] Example 2: This example is a landslide and rockfall detection system based on deep learning, which is used to execute a landslide and rockfall detection method based on deep learning as described in Example 1, and includes: The image acquisition and preprocessing module is responsible for acquiring monitoring images and performing standardized preprocessing. The image acquisition and preprocessing module acquires video streams in real time through monitoring cameras deployed on steep slopes, and performs scaling, color space conversion, pixel normalization and tensor format conversion on each frame of the image in sequence to make it meet the input requirements of the network model. The PCS-YOLO model inference module, as the core inference engine of the system, is responsible for loading the pre-trained PCS-YOLO network model weights, performing forward inference on the pre-processed image, and sequentially calling subsequent modules to complete the rockfall detection. The frequency-spatial domain collaborative feature extraction module, corresponding to the CSP-TFSA module, aims to enhance the edge texture representation of small-scale falling rocks and visually camouflaged falling rocks. The frequency-spatial domain collaborative feature extraction module processes the spatial domain branch (using the Scharr operator to extract explicit edge information) and the frequency domain branch (using Fourier transform to enhance texture details) in parallel, and then fuses the outputs of the two branches to obtain a feature map that enhances the edge texture representation. The parallel attention feature fusion module, corresponding to the PSCA attention module, is used to model the attention mechanism in parallel from three dimensions: channel, coordinate, and space. The parallel attention feature fusion module contains three parallel branches: the channel attention branch aggregates global information to obtain channel recalibration features, the coordinate attention branch captures position information to obtain coordinate-aware features, and the spatial attention branch generates a spatial mask to obtain spatial saliency features. The three outputs are adaptively weighted and fused to output a multi-scale feature map. The lightweight shared detection and localization module, corresponding to the SCSA-Head detection module, is used for target localization and classification of multi-scale feature maps. The lightweight shared detection and localization module adopts a cross-scale parameter sharing mechanism so that feature maps of three scales share the same set of depthwise separable convolutions. It introduces learnable scale-aware coefficients to calibrate the output distribution and uses group normalization to adapt to mini-batch inference. Finally, it outputs bounding box coordinates and rockfall confidence.

[0068] The results output and early warning module is responsible for post-processing the detection results and triggering early warnings. This module performs confidence filtering and non-maximum suppression on the detection boxes, converts the normalized coordinates into the original image coordinates, and triggers on-site audible and visual alarms, monitoring screen annotations, remote message pushes, and local log recordings when the confidence exceeds the threshold.

[0069] Through the collaborative work of the above modules, after being deployed on an edge computing device, the system can achieve end-to-end real-time detection and early warning of landslides and rockfalls in complex mountainous scenarios.

[0070] The above are merely some embodiments of the present invention. For those skilled in the art, various modifications and improvements can be made without departing from the inventive concept of the present invention, and all such modifications and improvements fall within the scope of protection of the present invention.

Claims

1. A landslide rockfall detection method based on deep learning, characterized in that, It includes the following steps: Step S1, Image Acquisition and Image Preprocessing: Acquire real-time images and then preprocess them; Step S2, Model Invocation: Input the preprocessed image into the pre-built and trained PCS-YOLO network model. The PCS-YOLO network model is a lightweight convolutional neural network. Step S3, Frequency-Spatial Domain Collaborative Feature Extraction: The PCS-YOLO network model performs forward inference on the input image and extracts image features through the Backbone network to obtain an initial feature map. The initial feature map is then processed in two parallel steps: the first step is to use the Scharr operator to extract explicit edge information and obtain spatial domain edge features. The second process involves mapping the features to the frequency domain using a fast Fourier transform to enhance texture details, followed by an inverse transform to restore the original texture, resulting in frequency domain texture enhancement features. The spatial domain edge features are fused with the frequency domain texture enhancement features to output a feature map that enhances the edge texture representation. Step S4, Parallel Attention Feature Fusion: The feature map with enhanced edge texture representation output from Step S3 is input into the Neck network, and three parallel attention processes are performed: The first attention process is to perform global average pooling and global max pooling on the feature map to aggregate global information and obtain channel recalibration features; the second attention process is to perform pooling along the height and width directions of the feature map to generate orientation-aware feature maps to capture position information and obtain coordinate-aware features; the third attention process is to perform spatial convolution on the feature map using large kernel convolution to generate a spatial mask and obtain spatial saliency features; the channel recalibration features, coordinate-aware features, and spatial saliency features are adaptively weighted and fused to output a fused multi-scale feature map; Step S5, Lightweight Shared Detection and Localization: Input the multi-scale feature map output from step S4 into the SCSA-Head detection module. The SCSA-Head detection module adopts a cross-scale parameter sharing mechanism, which enables the multi-scale feature maps to share the same set of depthwise separable convolutions for classification and regression prediction. It also introduces learnable scale-aware coefficients to calibrate the output distribution of the multi-scale feature maps and outputs the bounding box coordinates and confidence of the rockfall target. Step S6, Result Output and Warning: When the confidence level of the falling rock target exceeds the preset threshold, the bounding box coordinates of the falling rock target are output and a warning signal is triggered to realize end-to-end real-time detection and warning of landslides and falling rocks in complex mountainous scenes.

2. The landslide and rockfall detection method based on deep learning as described in claim 1, characterized in that: The construction and training method of the PCS-YOLO network model includes the following steps: Step T1, Data Acquisition and Processing: Acquire landslide and rockfall images, and construct a rockfall image dataset containing multiple scene types. The dataset includes a training set, a validation set, and a test set. Step T2, Network Model Construction: An initial PCS-YOLO network model is constructed based on the YOLO framework. The initial PCS-YOLO network model includes a Backbone network, a Neck network, and an SCSA-Head detection module. A CSP-TFSA module is introduced at the connection between the Backbone network and the Neck network to fuse spatial domain edge features and frequency domain texture enhancement features. A PSCA attention module is embedded in the Neck feature fusion layer to model in parallel from three dimensions: channel, coordinate, and space. The SCSA-Head detection module adopts a cross-scale parameter sharing mechanism, enabling multi-scale feature maps to share the same set of depthwise separable convolutions for classification and regression prediction. A learnable scale-aware coefficient is introduced to calibrate the output distribution of features at different scales. Step T3, Model Training: Train the initial PCS-YOLO network model using the training set, and optimize it using the SGD optimizer to obtain the initially trained PCS-YOLO network model and its initial model weights. Step T4, Model Validation and Testing: Validate and fine-tune the pre-trained PCS-YOLO network model using the validation set to obtain the optimized PCS-YOLO network model. Evaluate the performance of the optimized PCS-YOLO network model using the test set to verify its detection accuracy and robustness, and save the final version of the PCS-YOLO network model.

3. The landslide and rockfall detection method based on deep learning as described in claim 1, characterized in that: In step S3, the CSP-TFSA module contains two parallel spatial domain branches and a frequency domain branch. The detailed steps for feature extraction by the CSP-TFSA module are as follows: Step S31: Input Features: Input the feature map X output by the Backbone network; Step S32, Spatial Domain Branching: The feature map X is convolved with the Scharr kernel K respectively. x K y Perform convolution to obtain the horizontal gradient response G. x With the gradient response G in the vertical direction y Among them, G x =X×K x G y =X×K y Calculate the overall gradient magnitude G, G = |G x | + |G y | The spatial domain edge features F are obtained through 3×3 convolution and residual connection. spatial ; Step S33, Frequency Domain Branching: Perform a 2D Fast Fourier Transform on the feature map X to obtain the frequency domain features. After learnable frequency domain filtering and inverse transform, obtain the frequency domain texture enhancement features F. freq ; Step S34, Feature Fusion: The spatial domain edge features F output by the spatial domain branching process are fused together. spatial Frequency domain texture enhancement feature F output from frequency domain branching processing freq The first fusion feature F is obtained by adding elements one by one. fusion ,Right now First fusion feature F fusion The feature map is enhanced by channel rearrangement and 1×1 convolution to improve the representation of edge texture.

4. The landslide and rockfall detection method based on deep learning as described in claim 1, characterized in that: In step S4, the PSCA attention module includes a first attention branch, a second attention branch, and a third attention path in parallel. The first attention branch is a channel attention branch. It aggregates global information using global average pooling and global max pooling, and obtains channel attention weights after activation by a shared multilayer perceptron and a sigmoid function. The output channel recalibration feature F is then generated. channel ; The second attention branch is a coordinate attention branch. It performs one-dimensional average pooling along the height and width directions, and obtains coordinate attention weights after concatenation, convolution, splitting, and sigmoid activation. The output is the coordinate-aware feature F. coordl ; The third attention branch is a spatial attention branch. It performs global average pooling and global max pooling along the channel dimension, then concatenates the results. A 7×7 large kernel convolution is used to generate a spatial mask, which is then activated by a sigmoid function to obtain spatial attention weights. The spatial saliency feature F is then output. spatial ; The outputs of the three attention branches are weighted and summed using learnable fusion weights to obtain the second fusion feature F. out , Where α, β, and γ are adaptive learning weights, and satisfy α + β + γ = 1, the final multi-scale feature F is obtained through dynamic residual gating. final , , where δ is the learnable gating coefficient.

5. The landslide and rockfall detection method based on deep learning as described in claim 1, characterized in that: In step S5, the SCSA-Head detection module adopts a cross-scale parameter sharing mechanism, enabling the feature maps of the three scales of P3, P4 and P5 layers to share the same set of depthwise separable convolutions for classification and regression prediction; the depthwise separable convolution includes two steps: channel-wise convolution and pointwise convolution.

6. The landslide and rockfall detection method based on deep learning as described in claim 5, characterized in that: In step S5, a learnable scale-aware coefficient is introduced to calibrate the output of the regression branch. The calibration formula is as follows: , i=3,4,5; where, F i For the feature map of the i-th layer, Conv shared μ represents a shared depthwise separable convolution operation. i B represents the learnable scale-aware coefficient corresponding to the i-th layer. i This is the bounding box regression output corresponding to the feature map of the i-th layer.

7. The landslide and rockfall detection method based on deep learning as described in claim 1, characterized in that: The SCSA-Head detection module uses group normalization to divide the channel dimension into several groups on average. Within each group, the mean and variance are calculated independently and then normalized. The calculation formula is as follows: ,in, For input features, The mean of the group. The within-group standard deviation To prevent small constants from being divided by zero, A learnable scaling factor. A learnable translation factor. This is the output feature after normalization.

8. The landslide and rockfall detection method based on deep learning as described in claim 1, characterized in that: The SCSA-Head detection module includes two parallel output branches: a regression branch and a classification branch. The regression branch uses distributed focus loss to model the bounding box coordinates as a discrete probability distribution, and then maps them to continuous coordinates through integral mapping to model the positioning uncertainty of irregular rockfall boundaries; the classification branch outputs the confidence probability of the rockfall target through the Sigmoid activation function.

9. The landslide and rockfall detection method based on deep learning as described in claim 3, characterized in that: In the frequency domain branch of the CSP-TFSA module, the complex spectrum is decomposed into real and imaginary parts and concatenated along the channel dimension to obtain real-valued frequency domain features. Learnable frequency domain filtering is performed by 3×3 convolution, and the receptive field of the convolution operation covers the entire feature map. The network automatically learns to enhance the frequency domain components corresponding to the rockfall contour and suppress the frequency domain components corresponding to the background noise through backpropagation.

10. A landslide and rockfall detection system based on deep learning, characterized in that, It is used to perform a deep learning-based landslide and rockfall detection method as described in any one of claims 1 to 9, comprising: The image acquisition and preprocessing module is configured to acquire surveillance images and perform standardized preprocessing. The PCS-YOLO model inference module is configured to load the pre-trained PCS-YOLO network model weights, perform forward inference on the pre-processed image, and sequentially call subsequent modules to complete rockfall detection. The frequency-spatial domain collaborative feature extraction module is configured to process in parallel through spatial domain branches and frequency domain branches, and then fuse the outputs of the two branches to obtain a feature map that enhances the representation of edge texture. The parallel attention feature fusion module is configured to model the attention mechanism in parallel from three dimensions: channel, coordinate, and space. The parallel attention feature fusion module includes three parallel branches: the channel attention branch aggregates global information to obtain channel recalibration features, the coordinate attention branch captures position information to obtain coordinate-aware features, and the spatial attention branch generates a spatial mask to obtain spatial saliency features. The three outputs are adaptively weighted and fused to output a multi-scale feature map. The lightweight shared detection and localization module is configured to use a cross-scale parameter sharing mechanism to enable feature maps of multiple scales to share the same set of depthwise separable convolutions, introduce learnable scale-aware coefficients to calibrate the output distribution, and use group normalization to adapt to mini-batch inference, and finally output bounding box coordinates and their confidence scores. The results output and early warning module is configured to post-process the detection results and trigger early warnings.

Images