Landslide identification method, system, equipment and medium

Abstract

The present invention discloses a landslide identification method, system, device, and medium, relating to the field of disaster detection technology. The method comprises the following steps: obtaining remote sensing image data and digital elevation data of a target slope segment as a landslide dataset; segmenting the image data in the landslide dataset into n×n pixel tokens; inputting the tokens into a neural network model based on an encoder and decoder structure; processing the tokens by the encoder and decoder to obtain reconstructed image tokens and reconstruction loss between the tokens and the original image tokens; obtaining an optimal neural network model with the goal of minimizing the reconstruction loss; inputting the real-time remote sensing image data and digital elevation data of the target slope segment into the optimal neural network model for image interpretation, and obtaining a landslide map of the target slope segment. The present invention optimizes the neural network model for image interpretation, and the regional image interpretation results show high accuracy, further verifying its application potential in the field of disaster prevention and mitigation.

Description

Technical Field

[0001] The present invention relates to the field of disaster detection technology, and in particular to a landslide identification method, system, equipment and medium. Background Art

[0002] Landslide events are often associated with specific triggering factors, such as earthquakes, severe storms, rapid snow melt, or volcanic eruptions. Such events can involve anything from a single landslide to thousands of them, and, similar to earthquakes and volcanic activity, they can trigger severe geological hazards.

[0003] After a disaster occurs, quickly and accurately assessing potential losses and obtaining precise information about the landslide's location and affected area are crucial for earthquake emergency rescue and resettlement planning. Existing deep learning network models for automatically identifying rapid landslides require extensive processing, including model and data training. Limited by the training data, their accuracy is significantly affected by the quality of the sample data. This process consumes significant resources and time, hindering rapid and accurate early warning. Summary of the Invention

[0004] The purpose of the present invention is to provide a landslide identification method, system, equipment and medium to solve the problems in the prior art.

[0005] The present invention specifically provides the following technical solutions:

[0006] A landslide identification method comprises the following steps:

[0007] Obtain remote sensing image data and digital elevation data of the target slope section and construct a landslide dataset;

[0008] The image data in the landslide dataset is segmented into n×n pixel tokens, which are then fed into a neural network model based on an encoder and decoder structure. The encoder embeds the tokens not covered by the mask into features, and the self-attention mechanism captures the relationship between all tokens to generate the final output features. The decoder's multi-objective reconstruction strategy is then used to reconstruct the final output features. The reconstructed image tokens and the reconstruction loss between them and the original image tokens are obtained, and the optimal neural network model is constructed with the goal of minimizing the reconstruction loss.

[0009] The real-time remote sensing image data and digital elevation data of the target slope section are input into the optimal neural network model for image interpretation to obtain the landslide map of the target slope section.

[0010] Preferably, the step of acquiring remote sensing image data and digital elevation data of the target slope section and constructing a landslide dataset includes:

[0011] Define the time range before and after the earthquake for the target slope section, filter remote sensing image data based on the time range, and load digital elevation data;

[0012] Pre-earthquake images were filtered according to time, region, and corner points to obtain an image set that completely covered the study area. Edge masks were applied to remove edge noise. The image with the least cloud was selected from the filtered images and cropped to the target slope segment.

[0013] The remote sensing images of the target slope section were subjected to single-band image separation and NVID processing to obtain the RGB images before and after the earthquake and the NDVI before and after the earthquake. DEM-related factors were calculated on the digital elevation data to obtain the hillside shadow and slope. The RGB images before and after the earthquake, the NDVI before and after the earthquake, the hillside shadow and slope were stacked into a multi-band image, and the multi-band image was used as the landslide dataset.

[0014] Preferably, before dividing the image data in the landslide dataset into n×n pixel tokens, the method further includes:

[0015] Obtain NDVI images before and after the earthquake, and add them as additional channels of the RGB images before and after the earthquake as image data to be segmented. The specific expression of the added NDVI image is:

[0016]

[0017] Among them, NIR and RED are the reflectances of the near-infrared and red bands, respectively.

[0018] Preferably, the relationship between all tokens is captured by the self-attention mechanism, and the specific expression is:

[0019] Q i =x i W Q ,K i =x i W K ,V i =x i W V ;

[0020]

[0021] z i =Attention(Q i ,K i ,V i )=S i V i ;

[0022] Where d represents the dimension of embedding, Qi For query, K i is the key, V i is the value, x i is the input, S i is the attention score, z i is the output feature, W Q Q i The corresponding weight matrix is used to convert the input hidden state into the query vector Q i , W K K i The corresponding weight matrix is used to convert the input hidden state into the key vector K i , W V V i The corresponding weight matrix is used to convert the input hidden state into the key vector V i , K i The transpose of .

[0023] Preferably, the multi-objective reconstruction strategy of the decoder is used to reconstruct the final output features to obtain the token of the reconstructed image and the reconstruction loss between the token of the original image, including:

[0024] The final output features are reconstructed using the decoder's multi-objective reconstruction strategy to obtain the token of the reconstructed image; the specific expression is:

[0025]

[0026] in, is the token for reconstructing the image, g φ is the decoder, M is the mask matrix, specifically a binary matrix used to indicate which tokens are masked and which tokens should be retained, x is the token of the original image, and ⊙ is the element-wise multiplication or Hadamard product;

[0027] Get the reconstruction loss between the token of the reconstructed image and the token of the original image. The specific expression is:

[0028]

[0029] Among them, L is the overall reconstruction loss, L token is the token-to-token reconstruction loss, L spectral is the spectral to spectral reconstruction loss, and λ represents the loss used to adjust L token and L spectral The combined hyperparameters, m is the masked token (H p ×W p), x is the token of the original image, are the tokens of the reconstructed image, i, j, (r, c, 1) and Both represent indices, n is the number of spectral tokens, and vis is the index of the token not covered by the mask, for example, x vis ={x i |i∈vis} represents the token not covered by the mask, and j corresponds to the summation symbol A summation variable, representing the iteration index in a certain range. Specifically, j starts from 1 and ends at n, indicating that each item in this range is summed. Each spectral token is composed of (D is the channel dimension of the input image, k is the channel dimension of the token) standard tokens, and (r,c) represents the token in the rth row and cth column in the spectral data.

[0030] The present invention provides a landslide identification system, comprising:

[0031] The acquisition module is used to obtain remote sensing image data and digital elevation data of the target slope section and construct a landslide dataset;

[0032] The model training module is used to segment the image data in the landslide dataset into n×n pixel tokens and input the tokens into a neural network model based on an encoder and decoder structure. The encoder is used to embed the tokens not covered by the mask. The self-attention mechanism is used to capture the relationship between all tokens to generate the final output features. The final output features are reconstructed using the multi-objective reconstruction strategy of the decoder. The tokens of the reconstructed image and the reconstruction loss between them and the original image tokens are obtained. The optimal neural network model is constructed with the goal of minimizing the reconstruction loss.

[0033] The output module is used to input the real-time remote sensing image data and digital elevation data of the target slope section into the optimal neural network model for image interpretation to obtain the landslide map in the target slope section.

[0034] The present invention provides a computer device, comprising a memory and a processor. The memory stores a program, and when the program is executed by the processor, the processor executes the steps of the above-mentioned landslide identification method.

[0035] The present invention provides a storage medium storing a computer program, wherein the computer program implements the steps of the above-mentioned landslide identification method when executed by a processor.

[0036] Compared with the prior art, the present invention has the following significant advantages:

[0037] The present invention divides the image data in the landslide dataset into n×n pixel tokens, inputs the tokens not covered by the mask into the neural network model, uses the encoder and decoder of the neural network model to process the remote sensing image data and digital elevation data of the target slope section, and performs image reconstruction optimization on the neural network model based on the reconstruction loss optimization of the reconstructed image and the original image. The encoding, decoding and reconstruction loss optimization achieves the shortening of the entire process from data labeling to model training to image interpretation, and provides pixel-level recognition capability by processing the n×n pixel tokens, providing a fast and accurate basis for the subsequent acquisition of landslide distribution. By inputting real-time data into the trained model for interpretation, the landslide map of the target slope section can be obtained, thereby improving the efficiency of geological disaster prevention and emergency response. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is the overall process of a landslide identification method of the present invention;

[0039] Figure 2 Visualize the results for the data: Figure 2 (a) is the image before and after the earthquake (RGB), Figure 2 (b) is the image before and after the earthquake (NDVI), Figure 2 (c) is the hillside shadow, Figure 2 (d) is the slope;

[0040] Figure 3 is the accuracy index curve on the validation set during the SpectralGPT model training process; Figure 3 (a) is the curve under recall conditions, Figure 3 (b) is the curve under Precision conditions, Figure 3 (c) is the curve under F1 Score conditions, Figure 3 (d) is the curve under IoU conditions, Figure 3 (e) is the curve under the Mean IoU condition;

[0041] Figure 4 Models deployed on the PIE Engine AI platform (via upload or online model training);

[0042] Figure 5 Schematic diagram of the workflow of the SpectralGPT model and its migration to downstream tasks;

[0043] Figure 6The network architecture for semantic segmentation (top) and change detection (bottom) for the downstream tasks of the SpectralGPT model;

[0044] Figure 7 Image (left) and DEM (right) data of the study area (Bijie City); red dots are identified landslide locations;

[0045] Figure 8 These are different types of landslide examples in the study area, each with a different sample size. A 40-meter extension is retained as the background for each example. Figure 8 (a1)- Figure 8 (f6) are examples of different types of landslides;

[0046] Figure 9 Images of some areas before and after the earthquake: Figure 9 (a) is Hokkaido, Figure 9 (b) is Jiuzhaigou, China;

[0047] Figure 10 YOLOv5 model structure;

[0048] Figure 11 This is the interface saved in the model library after YOLOv5 deployment, showing information such as accuracy and network structure;

[0049] Figure 12 To use the PIE-Engine AI image intelligent processing platform for intelligent image interpretation;

[0050] Figure 13 The landslide prediction results in the Hokkaido study area (yellow represents the actual location of the landslide, and red represents the predicted result);

[0051] Figure 14 The landslide prediction results for the Hokkaido study area (partial);

[0052] Figure 15 The landslide prediction results in the Jiuzhaigou study area (yellow represents the actual location of the landslide, and red represents the predicted result);

[0053] Figure 16 This is the landslide prediction result (partial) in the Jiuzhaigou study area. DETAILED DESCRIPTION

[0054] The following is a clear and complete description of the technical solutions of the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0055] Combining the PIE Engine cloud platform with deep learning model-based landslide identification technology aims to overcome these shortcomings and provide a more efficient, cost-effective, and timely landslide identification solution. This improves the existing landslide identification workflow and provides users with a better experience and results. This paper addresses the issue of timely landslide mapping by proposing an online intelligent mapping solution.

[0056] like Figure 1 As shown, the present invention provides a landslide identification method to be described below, which specifically includes the following steps:

[0057] Step S1: Acquire remote sensing image data and digital elevation data of the target slope section and construct a landslide dataset.

[0058] First, we use the Google Earth Engine (GEE) platform to acquire and preprocess Sentinel-2 image data, i.e., remote sensing image data. The specific data processing flow during the GEE code execution is as follows:

[0059] (1) Define the area and initialize the parameters:

[0060] Define a study area: Define a rectangular region based on the bounding rectangle of the study area (such as Jiuzhaigou or Hokkaido) to frame the study area and clip the imagery. Defining the rectangular region using geographic coordinates ensures that imagery within the study area can be accurately clipped and analyzed.

[0061] Set time parameters: Define the time range before and after the earthquake for the target slope section. Filter remote sensing image data based on this time range and set a specific time period to ensure that the selected image collection accurately reflects environmental changes before and after the earthquake. Load digital elevation data.

[0062] (2) Load the dataset:

[0063] Load SRTM DEM data: Load SRTM elevation data.

[0064] Load Sentinel-2 image data: Load the Sentinel-2 image dataset s2Sr and cloud probability dataset s2Clouds.

[0065] Define edge mask function: Define edge mask function to remove edge noise.

[0066] (3) Filtering and preprocessing image data:

[0067] Filter pre- / post-earthquake images: Filter pre-earthquake images based on time, region, and corner points to obtain an image set that completely covers the study area, and apply edge masks to remove edge noise; Select the image with the least cloud: Select the image with the least cloud from the filtered images and crop it to the target slope section (area).

[0068] (4) Image de-clouding processing:

[0069] Associate cloud probability data: Associate cloud probability data with Sentinel-2 image data and add cloud masks.

[0070] Apply Cloud Mask: During image processing, use the cloud removal function to remove clouds from the image. This step is optional, and its implementation depends on whether the cloud cover of the least-cloudy image within the selected time range affects the image's usability. If the cloud cover of the least-cloudy image within the selected time range does not affect the image's usability, the least-cloudy image is selected directly. If the least-cloudy image within the selected time range does not meet the requirements, the cloud removal function is called to create a composite image of multiple images after cloud removal.

[0071] (5) Calculate and display NDVI:

[0072] The remote sensing imagery of the target slope segment was subjected to single-band image separation and NVID processing to obtain the RGB imagery and NDVI values before and after the earthquake (using the GEE normalizedDifference function to calculate the NVID before and after the earthquake). NDVI visualization parameters were then set and displayed on a map. This step not only calculated the NDVI value but also ensured the accuracy of the obtained NDVI data through visual inspection.

[0073] (6) Image normalization and resampling:

[0074] Define the target resolution and projection: Set the target resolution to 10 meters and the target projection to UTM Zone 48N / UTM Zone 54N (Jiuzhaigou / Hokkaido).

[0075] Check and resample bands: Define resampling and reprojection functions, check the image resolution, and resample and reproject. If the image resolution is inconsistent with the target resolution, use resample('bicubic') to resample it to a uniform resolution of 10m*10m for subsequent processing.

[0076] Calculate the actual maximum and minimum values of NDVI: Calculate the actual maximum and minimum values of NDVI for normalization and visualization of NDVI in GEE.

[0077] Normalize the imagery: Normalize the imagery based on a calculated value range (such as NDVI) or a known data value range. This step ensures that data from different bands are compared and analyzed on the same scale, thereby improving the accuracy and consistency of data processing. Training models on normalized datasets can accelerate model convergence and reduce training time. Normalizing datasets scales the input data to the same value range, making the model training process more stable and efficient.

[0078] (7) Stack bands and export images:

[0079] The DEM-related factors of the digital elevation data were calculated to obtain the hillside shadow and slope, and the bands were stacked: the normalized bands were arranged in sequence, with a total of 10 channels, in the following order: the hillside shadow, post-earthquake image RGB, post-earthquake NDVI, pre-earthquake image RGB and slope were stacked into a multi-band image, and the multi-band image was used as the landslide dataset.

[0080] Rename Band: Set a new band name for the stacked image to facilitate subsequent use.

[0081] Export images: Export the stacked images as GeoTIFF files and set parameters such as resolution and projection coordinate system to ensure that the exported data meets the requirements.

[0082] Step S2: The image data in the landslide dataset is divided into tokens of n×n pixels, and the tokens are input into a neural network model based on an encoder and decoder structure (the SpectralGPT model in this embodiment). The linear projection matrix of the encoder is used to embed the tokens not covered by the mask, and the relationship between all tokens is captured through the self-attention mechanism to generate the final output features of the encoder. The final output features are reconstructed using the multi-objective reconstruction strategy of the decoder to obtain the tokens of the reconstructed image and the reconstruction loss between the tokens and the original image tokens, and the optimal neural network model is obtained with the goal of minimizing the reconstruction loss.

[0083] (1) Dataset construction and model selection:

[0084] In order to fine-tune a semantic segmentation model specifically for identifying landslides, the present invention refers to the SegMunich dataset structure used for fine-tuning semantic segmentation downstream tasks and constructs a Jiuzhaigou landslide dataset with the same structure. Figure 2 Data visualization results are shown: (a) pre- and post-earthquake images (RGB), (b) pre- and post-earthquake images (NDVI), (c) hillside shadows, and (d) slope. Tokens are vector representations of small image blocks, used to represent local image regions.

[0085] In the selection of pre-training models, the present invention adopts the SpectralGPT+ model that performs progressive pre-training on multiple datasets. This model shows better performance on multiple datasets.

[0086] (2) Model fine-tuning and parameter setting

[0087] This paper mainly uses the semantic segmentation function in the downstream task of SpectralGPT to realize the pixel-level recognition of landslides, thereby completing landslide mapping. The SpectralGPT model workflow is as follows Figure 5 As shown, in the pre-training phase, SpectralGPT first trains the model from scratch on a dataset (e.g., fMoW-S2, containing 712,874 images), using random weight initialization based on (3D) tensors. Subsequently, the model is progressively trained on more datasets (e.g., BigEarthNet-S2, containing 354,196 images) with different image sizes, time series information, and geographic regions. SpectralGPT is built following the MAE architecture and incorporates a spectral tensor (3D) mask, where 90% of the labels are masked. For downstream tasks such as classification, segmentation, and change detection, the pre-trained SpectralGPT is connected to the task-specific head network, trained, and fine-tuned. The process of fine-tuning SpectralGPT on the Jiuzhaigou landslide dataset is as follows:

[0088] 1) Data preparation: The present invention collects optical data from the Sentinel-2 satellite and the digital elevation model (DEM) of the Shuttle Radar Topography Mission (SRTM) to create the Jiuzhaigou landslide dataset for fine-tuning a semantic segmentation model for landslide identification. Based on the minimum cloud cover principle, two pairs of Level-1C images (one pair each for Jiuzhaigou and Hokkaido) were selected for the dual-phase data corresponding to each study area. The Normalized Difference Vegetation Index (NDVI), as a key indicator reflecting the growth status of vegetation, is easily affected by the loss of vegetation that may be caused by landslides. Therefore, the NDVI images before and after the event are added as additional channels to overcome the shortcomings of using only RGB spectral data to detect landslides.

[0089] Before segmenting the image data in the landslide dataset into n×n pixel tokens, the following steps are also included:

[0090] Obtain NDVI images before and after the earthquake, and add them as additional channels of the RGB images before and after the earthquake as image data to be segmented. The specific expressions of the added NDVI images before and after the earthquake are:

[0091]

[0092] Among them, NIR and RED are the reflectances of the near-infrared and red bands, respectively.

[0093] Slope is a key topographic factor that directly influences landslide movement. Hillshade is another topographic factor widely used in landslide mapping. Hillshade and slope are extracted from the DEM, and these topographic factors are added as auxiliary data to enhance model performance. Finally, each study area has ten input bands of the same resolution (10 meters), which are stacked in the following order: hillshade, post-earthquake BGR band, post-earthquake NDVI, pre-earthquake BGR band, pre-earthquake NDVI, and slope.

[0094] 2) Data Segmentation and Augmentation: The image data is segmented into 128×128 pixel tokens. The dataset is split into training and validation sets in a 7:3 ratio. Data augmentation techniques are also performed, including random flipping and rotation.

[0095] 3) Fine-tuning parameters: During fine-tuning, a batch size of 96 is used and the base learning rate is set to 5×10 -4 ,Warmup-epochs is set to the first 30 epochs to reduce the oscillation and instability that may occur in the early stage of training and reduce the risk of overfitting. Weight-decay is set to 5×10 -3 , to enhance the generalization ability of the model and prevent the model from overfitting the training data.

[0096] 4) Fine-tuning the model: During fine-tuning, a specific head network needs to be connected to SpectralGPT, such as Figure 6 As shown. In this architecture, based on the SpectralGPT pre-trained model, the present invention further trains the UperNet head network to support the downstream tasks of semantic segmentation (top) and change detection (bottom). Among them, MLP stands for multi-layer perceptron. During the fine-tuning process, the present invention continuously monitors the accuracy indicators on the validation set, such as Figure 5 to evaluate model performance and make necessary adjustments.

[0097] The specific model architecture of SpectralGPT mentioned in the "SpectralGPT model fine-tuning" section is as follows:

[0098] 1) Encoder ViT:

[0099] The encoder of SpectralGPT is defined as follows:

[0100] Input processing: All visible tokens (i.e. tokens not covered by the mask) are first passed through a shared linear projection matrix E s Perform feature embedding conversion. These feature embeddings are combined with position encoding Epos Combined to form the input to the encoder f θ The final expression of .

[0101] Encoder structure: Encoder f θ It consists of multiple stacked self-attention (SA) Transformer blocks. The input embedding z i Generate query Q through linear transformation i 、Key(Key)K i Sum value (Value) V i Embed.

[0102] Self-attention mechanism: query Q i and key K i The attention score S between embeddings i The dot product is calculated and scaled by dimension d, and then normalized by the softmax function. The normalized attention score is used for weighted embedding to generate the final output embedding z i .

[0103] The relationship between all tokens is captured through the self-attention mechanism. The specific expression is:

[0104] Q i =x i W Q ,K i =x i W K ,V i =x i W V ,;

[0105]

[0106] z i =Attention(Q i ,K i ,V i )=S i V i ,;

[0107] Where d represents the dimension of embedding, Q i For query, K i is the key, V i is the value, x i is the input, S i is the attention score, z i is the output feature, W Q Q i The corresponding weight matrix is used to convert the input hidden state into the query vector Q i , W KK i The corresponding weight matrix is used to convert the input hidden state into the key vector K i , W V V i The corresponding weight matrix is used to convert the input hidden state into the key vector V i , K i The transpose of .

[0108] Output features: final output features z i With input x i have the same dimensions and can be further processed by subsequent encoders.

[0109] The encoder's primary function is to convert input visible tokens into feature embeddings and capture the relationships between these tokens through a self-attention mechanism. The encoder consists of multiple stacked self-attention Transformer blocks. Each block processes the input embeddings through a linear transformation and a self-attention mechanism to generate final output features. These output features can be further used for subsequent tasks such as classification and segmentation.

[0110] 2) Decoder MAE and loss function:

[0111] The decoder of SpectralGPT is defined as follows:

[0112] Input and target: Given the encoder output feature z, a lightweight decoder g is trained at the same time φ , using a multi-objective reconstruction strategy to restore the token of the original image. The final output features are reconstructed using the decoder's multi-objective reconstruction strategy to obtain the token of the reconstructed image and the reconstruction loss between it and the original image token, including:

[0113] The final output features are reconstructed using the decoder's multi-objective reconstruction strategy to obtain the token of the reconstructed image; the specific expression is:

[0114]

[0115] in, is the token for reconstructing the image, g φ is the decoder, M is the mask matrix, specifically a binary matrix used to indicate which tokens are masked and which tokens should be retained, x is the token of the original image, and ⊙ is the element-wise multiplication or Hadamard product.

[0116] Decoder structure: Decoder g φIt is usually narrower and shallower than the encoder, and usually consists of several Transformer blocks and a linear reconstruction layer.

[0117] Training method: The proposed SpectralGPT model trains the encoder f in an end-to-end manner θ and decoder g φ , to minimize the reconstructed image token The reconstruction loss between the token x of the original image can be obtained. The reconstruction loss between the token of the reconstructed image and the token of the original image is obtained. Reconstruction loss: The reconstruction loss consists of two parts: token-to-token and spectral-to-spectral. This multi-objective reconstruction allows the learned representation to effectively capture the spatial-spectral coupling characteristics and spectral sequence information. The overall loss LL is quantified in the pixel space using the mean square error (MSE), and the specific expression is:

[0118]

[0119]

[0120] Among them, L is the overall reconstruction loss, L token is the token-to-token reconstruction loss, L spectral is the spectral to spectral reconstruction loss, and λ represents the loss used to adjust L token and L spectral The combined hyperparameters, m is the masked token (H p ×W p ), x is the token of the original image, is the token of the reconstructed image, i,j,(r,c,1), etc. represent indices, n is the number of spectral tokens, and vis is the index of the token not covered by the mask, for example, x vis ={x i |i∈vis} represents the token not covered by the mask, and j corresponds to the summation symbol A summation variable that represents the iteration index in a certain range. Specifically, j starts from 1 and ends at n, indicating that each item in this range is summed. Each spectral token consists of (D is the channel dimension of the input image, k is the channel dimension of the token) standard tokens (standardtokens), and (r,c) represents the token in the rth row and cth column in the spectral data.

[0121] (3) Model deployment:

[0122] After fine-tuning is completed, a Docker image is built based on the model and used as the operating environment. Subsequently, the present invention binds the Docker image with the trained model and uploads it to the PIE EngineAI platform to complete the deployment of the model. Figure 4 shown.

[0123] This deployment model can be used for online prediction and intelligent image interpretation, providing efficient and accurate solutions for application scenarios such as landslide identification.

[0124] Step S3: inputting the real-time remote sensing image data and digital elevation data of the target slope section into the optimal neural network model for image interpretation to obtain a landslide map in the target slope section.

[0125] In another embodiment of the present invention, YOLOv5 model training deployment is adopted, specifically:

[0126] (1) Construction of cloud platform dataset:

[0127] Because YOLOv5 requires direct training on the PIE Engine AI platform, an online training dataset that meets the platform's requirements must be created. To this end, this paper leverages PIE Engine AI's collaborative sample annotation platform to create a cloud-based Bijie landslide dataset that fully meets the platform's training requirements, providing support for online training models.

[0128] The Bijie landslide dataset contains TripleSat satellite imagery acquired between May and August 2018, along with shape files and digital elevation models (DEMs) of landslide boundaries. The dataset includes 770 landslide samples, including rockfalls, rock slides, and a few debris landslides, as well as 2,003 negative samples with different backgrounds. Figure 7 The red dots in the figure are the actual locations of landslides. The landslide samples vary in size, and a 40-meter buffer zone is added to the bounding box of each sample as a background. This method helps the deep learning network better learn the characteristics of landslides. The RGB image resolution in the dataset is 0.8 meters, and the elevation accuracy of the DEM is 2 meters. The landslide shape vectors were manually drawn using ArcGIS, and the non-landslide samples included mountains, villages, roads, rivers, forests, and farmland. These samples were used to comprehensively evaluate the effectiveness of the landslide detection method. Some landslide instances in the area are shown in Figure 8 middle.

[0129] Furthermore, the Bijie landslide dataset's label format (binary images in .png format used as masks) couldn't be used directly for model training. Therefore, we wrote Python code to convert the labels to the required format for training. Finally, we divided the labels and images into training, validation, and test sets, reorganizing them into the required directory format to create a dataset suitable for direct model training.

[0130] (2) Online model training and deployment:

[0131] In the case of landslide mapping without the need for high-precision prediction results, if only the landslide needs to be quickly identified, selecting a target detection model with a small number of parameters and the ability to quickly predict can meet the needs. For this application scenario, the present invention adopts the target detection model YOLOv5 built into the PIE Engine AI platform. Thanks to the built-in model structure and training environment of the PIE cloud platform, the present invention only needs to set parameters in the platform model training interface and select the online data set produced by the present invention to quickly train the required model and achieve one-click deployment, which greatly improves efficiency. Utilizing the fast prediction characteristics of YOLOv5, the present invention can quickly identify the location of the landslide, thereby efficiently completing the task.

[0132] Specifically, YOLOv5 (You One Look Once version 5) mentioned in the section "(2) Online Model Training and Deployment" is a single-stage target detection algorithm that can complete the target detection task in one neural network forward propagation. The YOLOv5 network structure is the standard CSPDarknet+PAFPN+non-decoupled Head. The overall structure of the model is as follows: Figure 10 As shown in the figure, the upper part is the model overview, the middle part is the specific composition of each submodule, and the lower part is the specific network structure. The network structure mainly consists of three parts: backbone, neck and detection head:

[0133] (1) Model Backbone:

[0134] The overall structure of CSPDarknet is similar to ResNet. The P5 model structure consists of 5 layers: 1 StemLayer and 4 Stage Layers.

[0135] The Stem layer uses a ConvModule with a 6x6 kernel, which is more efficient than the Focus module in versions prior to v6.1. Except for the final Stage Layer, all other Stage Layers consist of a ConvModule and a CSPLayer. The specific structure is shown in the Details section of the figure above. The ConvModule is composed of Conv2d, BatchNorm, and SiLU activation functions. The CSPLayer, also known as the C3 module in the official YOLO v5 repository, consists of three ConvModules and n Darknet Bottlenecks with residual connections.

[0136] The final Stage Layer adds the SPPF module. The SPPF module works by passing the input serially through multiple 5x5 MaxPool2d layers. It achieves the same effect as the SPP module while running faster. After Stage Layers 2-4, the model outputs a feature map that enters the Neck structure. Taking a 640x640 input image as an example, the output features are (B, 256, 80, 80), (B, 512, 40, 40), and (B, 1024, 20, 20), corresponding to strides of 8, 16, and 32, respectively, where B represents the batch size.

[0137] (2) Model neck:

[0138] YOLOv5 uses the PAFPN architecture, combined with the CSP2 architecture designed by CSPNet and PANET as the neck for feature aggregation. The neck is mainly used to generate feature pyramids, enhancing the model's ability to detect objects of different scales and enabling recognition of objects of different sizes and scales.

[0139] (3) Model detection head:

[0140] The YOLOv5 head structure is identical to that of YOLOv3, employing a non-decoupled head design. The head module consists of only three convolutional layers with unshared weights, which transform the input feature maps. The PAFP N (Path Aggregation Feature Pyramid Network) still outputs three feature maps of different scales: (B, 256, 80, 80), (B, 512, 40, 40), and (B, 1024, 20, 20).

[0141] Because YOLOv5 uses a non-decoupled output approach, tasks such as classification and bounding box detection are performed in different channels of the same convolutional layer. Taking the COCO 80-category dataset as an example, with an input resolution of 640×640, the feature map shapes output by the Head module are: (B, 3×(4+1+80), 80, 80), (B, 3×(4+1+80), 40, 40), and (B, 3×(4+1+80), 20, 20). Here, 3 represents three anchors, 4 represents the bounding box (bbox) prediction branch, 1 represents the object presence (obj) prediction branch, and 80 represents the category prediction branch of the COCO dataset.

[0142] (4) Design of loss function:

[0143] YOLOv5 contains a total of 3 Loss, namely:

[0144] Classes loss: BCE loss is used.

[0145] Objectness loss: BCE loss is used.

[0146] Location loss: CIoU loss is used.

[0147] The three losses are summarized according to a certain ratio:

[0148] Loss=λ1L cls +λ2L obj +λ3L loc .

[0149] (5) The model processes the data as it passes through it:

[0150] During the data preprocessing phase of YOLOv5, various data augmentation techniques, such as Mosaic, RandomAffine, and MixUp, are used. These techniques randomly transform the training data to increase data diversity and improve the model's generalization capabilities. Additionally, image pixel values are normalized to the range [0, 1] to facilitate neural network processing.

[0151] The method further comprises:

[0152] The images from the landslide dataset were fed into the YOLOv5 model. In terms of model structure, YOLOv5 uses CSPDarknet as the feature extraction network (Backbone) to extract multi-scale features of the images. Features at different scales are further fused (Neck) using PAFPN (Path Aggregation Network).

[0153] The detection head uses a non-decoupled output method to perform image data classification and bounding box detection in different channels of the same convolution, and outputs the bounding box coordinates, category probability and confidence score of each target.

[0154] The image is parsed using the bounding box coordinates, class probability, and confidence score of each target to obtain a landslide map in the target slope segment.

[0155] During training, YOLOv5 uses a composite loss function consisting of a bounding box regression loss, a classification loss, and an object confidence loss. The Adam or SGD optimizer is typically used to update model parameters. During inference, the input image undergoes preprocessing before being fed into the model. The backbone network extracts image features, the neck network fuses these features, and the head network generates predictions, including bounding boxes and class probabilities. Finally, non-maximum suppression (NMS) is used to remove overlapping prediction boxes, retaining the optimal detection result. Through these steps, YOLOv5 can quickly and efficiently detect target objects in the input image and output the bounding box coordinates, class probability, and confidence score for each object.

[0156] Finally, the dataset is used to complete the training and deployment of the YOLOv5 model on the platform. The deployed YOLOv5 model can be used for online prediction to achieve intelligent image interpretation. During the training process, 660 rounds of training are performed according to the parameter settings of offline model training. After the training is completed, the trained model is saved to the model library of the cloud platform. The results of model training and deployment on the cloud platform are shown in the figure. Figure 11 shown.

[0157] Intelligent image interpretation:

[0158] Users can select the deployed model and upload the image to be processed through the PIE-Engine AI image intelligent processing platform. After completing the intelligent interpretation of the image using the powerful computing power of the cloud platform, users can view the interpretation results online. Figure 12 In addition, users can also interpret the results (such as Figures 13 to 16 ) from the platform to a local computer and then imported into software such as ArcGIS for further analysis or other tasks. To evaluate the generalization capability of the model, the present invention downloaded the prediction results to a local computer and, combined with the landslide cataloging in the test set area, calculated the accuracy of the model in an area where the model had never been seen (Hokkaido), as shown in Table 1.

[0159] Technical Results: YOLOv5 deployed on the PIE-Engine AI platform interpreted a 5888×6313 image in just 2 minutes and 24 seconds, while SpectralGPT interpreted a 3350×2867 image in just 4 minutes and 19 seconds, fully demonstrating the efficiency of deep learning models deployed on the PIE cloud platform. Furthermore, the system demonstrated high accuracy in interpreting imagery in completely unseen areas, further validating its potential for application in disaster prevention and mitigation.

[0160] Table 1 Precision indicators of the prediction results of the trained SpectralGPT model in the Hokkaido area

[0161] category IoU Precision Recall background 0.9462 0.9744 0.9703 landslide 0.4008 0.5543 0.5914 mean 0.6735 0.7643 0.7809

[0162] Based on the above method, the present invention provides a landslide identification system, which includes: an acquisition module, a model training module and an output module.

[0163] Among them, the acquisition module is used to obtain the remote sensing image data and digital elevation data of the target slope section and construct a landslide dataset; the model training module is used to segment the image data in the landslide dataset into n×n pixel tokens, and input the tokens into a neural network model based on the encoder and decoder structure. The encoder is used to embed the tokens not covered by the mask, and the relationship between all tokens is captured through the self-attention mechanism to generate the final output features. The final output features are reconstructed using the multi-objective reconstruction strategy of the decoder to obtain the tokens of the reconstructed image and the reconstruction loss between the tokens and the original image. The optimal neural network model is constructed with the goal of minimizing the reconstruction loss; the output module is used to input the real-time remote sensing image data and digital elevation data of the target slope section into the optimal neural network model for image interpretation to obtain the landslide map in the target slope section.

[0164] The present invention also provides a computer device, comprising a memory and a processor. The memory stores a program, and when the program is executed by the processor, the processor executes the steps of a landslide identification method.

[0165] According to the disclosed embodiments, a computing device may communicate with one or more external devices (e.g., a keyboard, a pointing device, Bluetooth communications, etc.), or with any device that enables a computing device to communicate with one or more other computing devices (e.g., a router, a modem, etc.).

[0166] The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a landslide identification method are implemented.

[0167] According to the disclosed embodiments, the storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0168] The above content further explains the present invention in detail in combination with specific preferred embodiments. For those skilled in the art of the technical field to which the present invention belongs, several simple deductions or replacements can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A landslide identification method, characterized in that: include: Obtain remote sensing image data and digital elevation data of the target slope section and construct a landslide dataset; The image data in the landslide dataset is segmented into n×n pixel tokens, which are then fed into a neural network model based on an encoder and decoder structure. The encoder embeds the tokens not covered by the mask into features, and the self-attention mechanism captures the relationship between all tokens to generate the final output features. The decoder's multi-objective reconstruction strategy is then used to reconstruct the final output features. The reconstructed image tokens and the reconstruction loss between them and the original image tokens are obtained, and the optimal neural network model is constructed with the goal of minimizing the reconstruction loss. The real-time remote sensing image data and digital elevation data of the target slope section are input into the optimal neural network model for image interpretation to obtain the landslide map of the target slope section.

2. A landslide identification method according to claim 1, characterized in that: The step of acquiring remote sensing image data and digital elevation data of the target slope section and constructing a landslide dataset includes: Define the time range before and after the earthquake for the target slope section, filter remote sensing image data based on the time range, and load digital elevation data; Pre-earthquake images were filtered according to time, region, and corner points to obtain an image set that completely covered the study area. Edge masks were applied to remove edge noise. The image with the least cloud was selected from the filtered images and cropped to the target slope segment. The remote sensing images of the target slope section were subjected to single-band image separation and NVID processing to obtain the RGB images before and after the earthquake and the NDVI before and after the earthquake. DEM-related factors were calculated on the digital elevation data to obtain the hillside shadow and slope. The RGB images before and after the earthquake, the NDVI before and after the earthquake, the hillside shadow and slope were stacked into a multi-band image, and the multi-band image was used as the landslide dataset.

3. A landslide identification method according to claim 2, characterized in that: Before dividing the image data in the landslide dataset into n×n pixel tokens, the method further includes: Obtain NDVI images before and after the earthquake, and add them as additional channels of the RGB images before and after the earthquake as image data to be segmented. The specific expressions of the added NDVI images before and after the earthquake are: Among them, NIR and RED are the reflectances of the near-infrared and red bands, respectively.

4. A landslide identification method according to claim 1, characterized in that: The self-attention mechanism is used to capture the relationship between all tokens. The specific expression is: Q i =x i W Q ,K i =x i W K ,V i =x i W V ; z i =Attention(Q i ,K i ,V i )=S i V i ; Where d represents the dimension of embedding, Q i For query, K i is the key, V i is the value, x i is the input, S i is the attention score, z i is the output feature, W Q Q i The corresponding weight matrix is used to convert the input hidden state into the query vector Q i , W K K i The corresponding weight matrix is used to convert the input hidden state into the key vector K i , W V V i The corresponding weight matrix is used to convert the input hidden state into the key vector V i , K i The transpose of .

5. A landslide identification method according to claim 1, characterized in that: The multi-objective reconstruction strategy of the decoder is used to reconstruct the final output features, obtain the token of the reconstructed image and the reconstruction loss between the token of the original image, including: The final output features are reconstructed using the decoder's multi-objective reconstruction strategy to obtain the token of the reconstructed image; the specific expression is: in, is the token for reconstructing the image, g φ is the decoder, M is the mask matrix, specifically a binary matrix used to indicate which tokens are masked and which tokens should be retained, x is the token of the original image, and ⊙ is the element-wise multiplication or Hadamard product; Get the reconstruction loss between the token of the reconstructed image and the token of the original image. The specific expression is: L=L token +λL spectral Among them, L is the overall reconstruction loss, L token is the token-to-token reconstruction loss, L spectral is the spectral to spectral reconstruction loss, and λ represents the loss used to adjust L token and L spectral The combined hyperparameters, m is the masked token (H p ×W p ), x is the token of the original image, are the tokens of the reconstructed image, i, j, (r, c, 1) and Both represent indices, n is the number of spectral tokens, vis is the index of tokens not covered by the mask, and x vis =[x i |i∈vis} represents the token not covered by the mask, and j corresponds to the summation symbol A summation variable, representing the iteration index in a certain range. Specifically, j starts from 1 and ends at n, indicating that each item in this range is summed. Each spectral token is composed of (D is the channel dimension of the input image, k is the channel dimension of the token) standard tokens, and (r,c) represents the token in the rth row and cth column in the spectral data.

6. A landslide identification system, characterized in that: include: The acquisition module is used to obtain remote sensing image data and digital elevation data of the target slope section and construct a landslide dataset; The model training module is used to segment the image data in the landslide dataset into n×n pixel tokens and input the tokens into a neural network model based on an encoder and decoder structure. The encoder is used to embed the tokens not covered by the mask. The self-attention mechanism is used to capture the relationship between all tokens to generate the final output features. The final output features are reconstructed using the multi-objective reconstruction strategy of the decoder. The tokens of the reconstructed image and the reconstruction loss between them and the original image tokens are obtained. The optimal neural network model is constructed with the goal of minimizing the reconstruction loss. The output module is used to input the real-time remote sensing image data and digital elevation data of the target slope section into the optimal neural network model for image interpretation to obtain the landslide map in the target slope section.

7. A computer device, characterized in that: The method comprises a memory and a processor, wherein a program is stored in the memory, and when the program is executed by the processor, the processor executes the steps of the landslide identification method according to any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a landslide identification method according to any one of claims 1 to 5 are implemented.

Images