Landslide intelligent detection system and method based on multi-source heterogeneous data and instance segmentation

Abstract

The present application belongs to the field of geological disaster monitoring and artificial intelligence technology, and specifically relates to a landslide intelligent detection system and method based on multi-source heterogeneous data and instance segmentation; the system comprises a data acquisition and preprocessing module, a data set construction module, a network construction and training module, a detection evaluation and visualization module; the method comprises: step one, data acquisition and preprocessing; step two, multi-channel feature encoding strategy and construction of multi-channel data set; step three, network model construction and improvement; step four, model training and landslide detection; step five, detection result post-processing. The present application converts time series and point physical monitoring data into a spatial heat map, directly inputs the visual network through a channel stacking method, avoids a complex double-branch structure, significantly reduces model complexity and inference delay, realizes real early fusion of multi-source heterogeneous data, can capture internal instability precursors that cannot be reflected by optical images, and effectively improves the recognition rate of concealed landslides.

Description

Technical Field

[0001] This invention belongs to the fields of geological disaster monitoring and artificial intelligence technology, specifically relating to an intelligent landslide detection system and method based on multi-source heterogeneous data and instance segmentation. Background Technology

[0002] Landslides, as a type of geological hazard, are characterized by their suddenness and destructive power. Accurate identification and delineation of the landslide's extent are crucial for disaster prevention and mitigation. Current landslide detection technologies have the following limitations.

[0003] Traditional methods rely heavily on optical remote sensing imagery, using only surface texture and spectral characteristics as criteria for judgment. For "hidden landslides" with high vegetation cover, minimal surface deformation, but internal creep, optical imagery often fails to capture them, leading to missed detections.

[0004] Secondly, although InSAR can acquire surface deformation and ground sensors such as rain gauges, pore water pressure gauges and seismometers can acquire data on inducing factors, most of these data are point-like time-series data or low-resolution raster data. They are highly heterogeneous with high-resolution optical images in terms of spatial scale and data format, making it difficult to directly integrate them into a detection model.

[0005] Existing multimodal fusion schemes typically employ dual-branch or multi-branch neural network architectures. This makes the model architecture complex and training difficult. Furthermore, simple feature concatenation often ignores the decisive weight of physical factors in landslide detection, resulting in sluggish model response under extreme conditions.

[0006] Traditional semantic segmentation methods can only distinguish between pixel regions of "landslide" and "non-landslide", but cannot distinguish individual landslides. It is difficult to count the number of landslides in a region and the geometry of individual landslides, which is not conducive to post-disaster emergency assessment.

[0007] Therefore, there is a need for a landslide detection method that can uniformly transform multi-source heterogeneous data into spatial features and utilize lightweight instance segmentation networks to achieve high accuracy and interpretability. Summary of the Invention

[0008] This invention discloses a landslide intelligent detection system and method based on multi-source heterogeneous data and instance segmentation, which solves the problems of difficulty in multi-source data fusion, low recognition rate of hidden landslides and low model inference efficiency in the prior art.

[0009] To achieve the above objectives, the technical solution of the present invention is as follows: The landslide intelligent detection system based on multi-source heterogeneous data and instance segmentation is deployed on an edge computing workstation or cloud server. It includes a data acquisition and preprocessing module for acquiring and preprocessing multi-source data of the target area; a dataset construction module for multi-channel feature encoding and multi-channel dataset construction; a network construction and training module for building and training an improved detection network based on the YOLOv11-Seg architecture; and a detection evaluation and visualization module for post-processing the detection results.

[0010] The detection method of the landslide intelligent detection system based on multi-source heterogeneous data and instance segmentation includes the following steps: Step 1: Data Acquisition and Preprocessing; Step 2: Multi-channel feature encoding strategy and construction of multi-channel dataset; Step 3: Network Model Construction and Improvement; Step 4: Model Training and Landslide Detection; Step 5: Post-processing of test results.

[0011] Preferably, step one includes the following specific steps: collecting multi-source data of the target area, including: high-resolution orthophotos of the UAV, InSAR surface deformation and coherence data, time-series data of rainfall, pore water pressure monitoring data and ground motion parameters; establishing a spatial reference frame based on the UAV orthophotos, and unifying all non-image data into the same geographic coordinate system and spatial resolution grid as the UAV images through coordinate projection and resampling.

[0012] Preferably, step two includes the following specific steps: (21) Convert the non-image data after unifying the multi-source data into a single-channel or multi-channel feature heatmap of the same size as the image; (22) Perform range normalization or Z-Score standardization on each of the physical quantity channels generated above to eliminate dimensional differences; (23) Stack the RGB three channels of UAV imagery with the physical quantity feature channels generated from non-image data in the channel dimension to construct an N-channel multimodal input tensor; where N=3+M, M is the number of physical quantity feature channels; at the same time, create corresponding landslide instance segmentation labels, the labels include landslide category information and polygon vertex coordinate sequence, and then obtain a multi-channel dataset.

[0013] Preferably, the specific step (21) includes: (211) InSAR feature encoding: generating the cumulative surface deformation rate channel and coherence coefficient channel; (212) Hydrodynamic field reconstruction: Based on rainfall station data, the "instantaneous rainfall intensity distribution map" and "accumulated distribution map of previous effective rainfall (API)" were generated using the Kriging interpolation method; based on pore water pressure monitoring points, the "current pore pressure value distribution map" and "pore pressure change rate distribution map" were generated; (213) Earthquake ground field reconstruction: Based on the earthquake magnitude and epicentral distance, the "peak ground acceleration (PGA) distribution map" is calculated and generated using the earthquake ground motion attenuation model.

[0014] Preferably, step three includes the following specific steps: A detection network is constructed based on the YOLOv11-Seg architecture, and the following improvements are made: the number of input channels in the first convolutional layer of the network is modified to adapt to N-channel input; during the model initialization stage, the convolutional kernel weights of the first three channels (RGB channels) of the original pre-trained model are retained, and the convolutional kernel weights of the newly added M physical quantity channels are initialized using a Gaussian distribution random initialization or a mean-shifted initialization based on the RGB weights to accelerate convergence and retain transfer learning features; multi-scale spatial features are extracted using the backbone network, and the adaptive fusion of shallow texture features and deep physical semantic features is achieved through the neck network PANet and FPN, resulting in an improved detection network based on the YOLOv11-Seg architecture.

[0015] Preferably, step four includes the following specific steps: training the improved detection network based on the YOLOv11-Seg architecture using the constructed multi-channel dataset, and introducing a joint optimization function of bounding box loss, classification loss and mask segmentation loss; after training, inputting the multi-channel tensor of the test area, and outputting the bounding box, confidence score and pixel-level segmentation mask of the landslide target.

[0016] Preferably, step five includes the following specific steps: performing morphological optimization on the output segmentation mask to remove noise patches smaller than a set threshold; superimposing the detected landslide instances onto the original UAV image and physical quantity heat map to generate a comprehensive assessment report including landslide location, area, aspect ratio, and intensity of associated disaster-causing factors.

[0017] The beneficial effects of the landslide intelligent detection system and method based on multi-source heterogeneous data and instance segmentation in this invention are as follows: This invention transforms temporal and point-based physical monitoring data into spatial heat maps, which are then directly input into a visual network via channel stacking. This avoids complex dual-branch architectures, significantly reducing model complexity and inference latency, and achieving true early fusion of multi-source heterogeneous data. Furthermore, by introducing physical channels that "see through" the Earth's surface, such as pore pressure change rate and InSAR deformation, the model can capture internal instability precursors that optical images cannot reflect, effectively solving the problem of missed landslide detection in vegetated areas and significantly improving the identification rate of hidden landslides. This invention also uses ablation experiments and Gradient Weighted Class Activation Mapping (Grad-CAM) technology to quantify the contribution of different factors such as rainfall and earthquakes to the detection results, providing data support for disaster cause analysis and giving the model physical interpretability. Moreover, based on the improved YOLOv11-Seg architecture, it can not only locate landslides but also accurately extract the geometric contours of each independent landslide body, facilitating rapid volume calculation and disaster scale assessment. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0019] Figure 2 This is a schematic diagram of the principle of raster reconstruction and multi-channel encoding of multi-source heterogeneous data.

[0020] Figure 3 A schematic diagram of the improved multi-channel instance segmentation network architecture and weight initialization strategy.

[0021] Figure 4 This is an example of a Grad-CAM-based model attention heatmap visualization. Detailed Implementation

[0022] The following description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0023] Example 1: like Figure 1 As shown, the implementation of this invention consists of five core steps: data acquisition and preprocessing, multi-channel feature encoding strategy and construction of multi-channel dataset, network model construction and improvement, model training and landslide detection, and post-processing of detection results.

[0024] Step 1: Data Acquisition and Preprocessing Drone imagery: Using a multi-rotor drone equipped with a visible light camera, orthophotos (DOM) with a resolution of 0.05m are obtained.

[0025] InSAR data: Sentinel-1 satellite data was acquired and processed by SBAS-InSAR to generate an annual average deformation rate map and a coherence map with a resolution of 0.2m.

[0026] Ground monitoring data: Data collected from 5 automatic rain gauges, 3 deep displacement gauges with pore pressure sensors, and nearby seismic networks over the past 7 days.

[0027] Spatial registration: Using the UAV DOM as a reference, InSAR data is resampled to 0.05m resolution using affine transformation; for point sensor data, ordinary kriging is used, and a spherical model is selected as the variogram model to generate a 0.05m resolution raster map covering the entire map.

[0028] Step 2: Multi-channel feature encoding strategy and construction of multi-channel dataset: like Figure 2 Construct a 9-channel input tensor, specifically defined as follows: Channels 1-3: R, G, and B components of UAV imagery, with values ​​normalized to [0, 1].

[0029] Channel 4: InSAR deformation rate, taken as absolute value and logarithmically transformed, normalized to [0, 1].

[0030] Channel 5: InSAR coherence coefficients, directly normalized.

[0031] Channel 6: Daily rainfall distribution map, reflecting the immediate triggering effect.

[0032] Channel 7: Effective Amount of Rainfall in the Early Stage (API), calculated by the formula Papi=i=17ki·Rt-1 where k=0.85, reflecting the soil water saturation.

[0033] Channel 8: Rate of change of pore water pressure Highlight the area where the pore pressure rises sharply.

[0034] Channel 9: PGA distribution map, calculated based on the earthquake attenuation formula ln(PGA)=a+bM-c·ln(R+d).

[0035] Step 3: Network Model Construction and Improvement like Figure 3 YOLOv11n-Seg was chosen as the basic architecture, and the following customized modifications were made: Input layer modification: Modify the first convolutional layer Conv(3, 64, kernel=3) of the original model to Conv(9,64, kernel=3).

[0036] Weight initialization strategy: Load YOLOv11 weights pre-trained on the COCO dataset.

[0037] Extract the weights of the original first-layer convolutional kernel, Worig∈R64×3×3×3.

[0038] Construct a new weight Wnew=∈R64×9×3×3.

[0039] Let Wnew:,:3,:,:=Worig, which preserves the pre-trained feature extraction capability of the RGB channels.

[0040] Let Wnew[:,3:,:,:]∼N(0,0.01), that is, the weights of the 6 newly added physical channels follow a Gaussian distribution with a mean of 0 and a variance of 0.01, to avoid the gradient vanishing due to excessively large initial activation values.

[0041] Loss function: Ltotal = λ1Lbox + λ2Lcls + λ3Lseg, where λ3 (segmentation loss weight) is set to 1.5 to enhance the fitting ability to irregular landslide boundaries.

[0042] A dynamic channel attention mechanism was added to the model. An SE (Squeeze-and-Excitation) module was embedded after the C2f module in the network backbone. This module receives the 9 channel features after global average pooling and learns the weight coefficients of each channel through a fully connected layer. Its function is to automatically reduce the weights of channels 6 and 7 (rainfall-related) and increase the weights of channels 1-3 (texture) on rainless, sunny days; and automatically increase the weights of channels 6-9 after heavy rain or earthquakes. This improvement further enhances the model's robustness under different meteorological conditions, enabling it to adaptively focus on the most critical disaster-causing factors.

[0043] Step 4: Model Training and Landslide Detection Mosaic data enhancement, random rotation (±15°), and HSV color dithering were employed. Note that the same geometric transformation was applied to all nine channels to maintain spatial alignment.

[0044] The input image size is 640×640, the batch size is 16, the initial learning rate is 0.001, the optimizer is AdamW, and the training is performed for 100 epochs. Mosaic enhancement is disabled in the last 10 epochs, and a Dropout layer is introduced to prevent overfitting.

[0045] After the model is trained, the test area is divided into 640×640 multi-channel image patches using a sliding window method and input into the trained model. The model outputs the category, confidence score, bounding box, and binary mask for each landslide instance.

[0046] Step 5: Post-processing of test results: The raw output of the model includes bounding box coordinates, class confidence scores, and a high-resolution mask matrix. To obtain the final usable landslide spatial information, this raw data requires post-processing.

[0047] Non-maximum suppression (NMS) was performed with a confidence threshold of 0.45 and an IoU threshold of 0.6. Noise was removed by opening the segmentation mask. Based on the optimized mask, the geometric properties of each landslide instance were calculated. The processed landslide vector contours were then overlaid onto the original UAV orthophoto and physical quantity heatmap. The system automatically generated a comprehensive assessment report including landslide number, location coordinates, area size, and the intensity of associated hazard factors. Landslide areas of different risk levels were highlighted in different colors on the visualization interface, such as... Figure 4 As shown.

[0048] Example 2: The landslide intelligent detection system based on multi-source heterogeneous data and instance segmentation, which implements the method of this invention, is deployed on an edge computing workstation or cloud server. It includes a data acquisition and preprocessing module for acquiring and preprocessing multi-source data of the target area; a dataset construction module for performing multi-channel feature encoding and constructing a multi-channel dataset; a network construction and training module for building and training an improved detection network based on the YOLOv11-Seg architecture; and a detection evaluation and visualization module for post-processing the detection results. The system front-end displays a drone-based real-world map, while the back-end receives real-time sensor data streams. It automatically updates the physical quantity heatmap channels at regular intervals (e.g., every hour) and triggers an inference detection, achieving a near real-time monitoring mode of "data update equals detection."

[0049] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be included in the scope of the present invention.

Claims

1. A detection method for a landslide intelligent detection system based on multi-source heterogeneous data and instance segmentation, characterized by: The intelligent detection system is deployed on an edge computing workstation or cloud server, and includes a data acquisition and preprocessing module for acquiring multi-source data of the target area and preprocessing the multi-source data; A dataset building module for multi-channel feature encoding and multi-channel dataset construction; a network building and training module for building an improved detection network based on the YOLOv11-Seg architecture and training the network. A detection evaluation and visualization module used for post-processing of detection results; The detection method includes the following steps: Step 1: Data Acquisition and Preprocessing; Step 2: Multi-channel feature encoding strategy and construction of multi-channel dataset; Step 3: Network Model Construction and Improvement; Step 4: Model Training and Landslide Detection; Step 5: Post-processing of test results; Step one includes the following specific steps: collecting multi-source data of the target area, including: high-resolution orthophotos of the UAV, InSAR surface deformation and coherence data, time-series data of rainfall, pore water pressure monitoring data and ground motion parameters; establishing a spatial reference frame based on the UAV orthophotos, and unifying all non-image data into the same geographic coordinate system and spatial resolution grid as the UAV images through coordinate projection and resampling. Step two includes the following specific steps: (21) Convert the non-image data after unifying the multi-source data into a single-channel or multi-channel feature heatmap of the same size as the image; (22) Perform range normalization or Z-Score standardization on each of the physical quantity channels generated above to eliminate dimensional differences; (23) Stack the RGB three channels of UAV imagery with the physical quantity feature channels generated from non-image data in the channel dimension to construct an N-channel multimodal input tensor; where N=3+M, M is the number of physical quantity feature channels; at the same time, make corresponding landslide instance segmentation labels, the labels include landslide category information and polygon vertex coordinate sequence, and then obtain a multi-channel dataset; The specific steps (21) mentioned above include: (211) InSAR feature encoding: generating the cumulative surface deformation rate channel and coherence coefficient channel; (212) Hydrodynamic field reconstruction: Based on rainfall station data, the "instantaneous rainfall intensity distribution map" and "cumulative distribution map of previous effective rainfall" were generated using the Kriging interpolation method; based on pore water pressure monitoring points, the "current pore pressure value distribution map" and "pore pressure change rate distribution map" were generated. (213) Reconstruction of ground motion field: Based on the earthquake magnitude and epicentral distance, the "peak ground acceleration distribution map" is calculated and generated using the ground motion attenuation model; Step three includes the following specific steps: A detection network is constructed based on the YOLOv11-Seg architecture, and the following improvements are made: the number of input channels in the first convolutional layer of the network is modified to adapt to N-channel input; during model initialization, the convolutional kernel weights of the first three channels (RGB channels) of the original pre-trained model are retained, and the convolutional kernel weights of the newly added M physical quantity channels are initialized using Gaussian distribution random initialization or mean-shifted initialization based on RGB weights to accelerate convergence and retain transfer learning features; multi-scale spatial features are extracted using the backbone network, and the adaptive fusion of shallow texture features and deep physical semantic features is achieved through the neck network PANet and FPN, resulting in an improved detection network based on the YOLOv11-Seg architecture.

2. The detection method of the landslide intelligent detection system based on multi-source heterogeneous data and instance segmentation as described in claim 1, characterized in that, Step four includes the following specific steps: training the improved detection network based on the YOLOv11-Seg architecture using the constructed multi-channel dataset, and introducing a joint optimization function of bounding box loss, classification loss and mask segmentation loss; After training, input the multi-channel tensor of the region to be tested, and output the bounding box, confidence score, and pixel-level segmentation mask of the landslide target.

3. The detection method of the landslide intelligent detection system based on multi-source heterogeneous data and instance segmentation as described in claim 2, characterized in that, Step five includes the following specific steps: performing morphological optimization on the output segmentation mask to remove noise patches smaller than a set threshold; overlaying the detected landslide instances onto the original UAV imagery and physical quantity heat map to generate a comprehensive assessment report including landslide location, area, aspect ratio, and intensity of associated disaster-causing factors.

Images