Mask2Former landslide identification method and device based on terrain characteristics

Abstract

This invention discloses a terrain-feature-based Mask2Former landslide identification method and device. It proposes a deep learning landslide identification method based on an improved Mask2Former that takes terrain features into account. This method can calculate the uncertainty of landslide identification and address the impact of objects such as rivers and waterways in the study area on the accuracy of landslide identification. The method includes: collecting and transforming relevant terrain feature data; fusing terrain features with remote sensing imagery; performing cropping, rotation, and data enhancement operations on the processed remote sensing imagery; annotating landslide samples and converting the labels into JSON; inputting the relevant data into an improved Mask2Former model for training; selecting the training model with the highest API index and applying it to landslide identification; automatically extracting the identification mask to obtain the identification result, and automatically calculating the uncertainty of the identification area and the identification evaluation index.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and more particularly to a Mask2Former landslide recognition method and device based on terrain features. Background Art

[0002] Landslides cause extensive damage to infrastructure and loss of life. Accurate and rapid landslide identification is crucial for disaster relief efforts. There is a growing demand for accurate, real-time landslide identification capabilities that can rapidly calculate and automatically process terrain features. Traditional methods, such as visual interpretation and field surveys, are labor-intensive, inefficient, and unable to meet the demands of real-time identification. Consequently, there is a high demand for automated and semi-automated landslide identification technologies.

[0003] Landslide identification methods are primarily categorized as pixel-based and object-oriented. Pixel-based methods analyze individual pixel values ​​but ignore geometric and contextual features. In contrast, object-oriented methods partition the region into non-overlapping areas and consider spatial, textural, contextual, geometric, and spectral features. Traditional machine learning methods are unable to determine the optimal scale for different regions, which affects the accuracy of identification results.

[0004] Deep learning has emerged as a promising solution to these challenges. Deep learning models such as Mask R-CNN, DenseNet, and YOLO5 have demonstrated higher accuracy in landslide identification, but are still affected by objects such as river channels, leaving room for improvement. Furthermore, current deep learning methods consume a large amount of memory and experience slow processing speed when considering terrain features, hindering their application to real-time landslide identification. Summary of the Invention

[0005] This paper aims to address the issues of insufficient accuracy and excessive computational power consumption in landslide identification by existing deep learning models. It proposes a terrain-based landslide identification method called Mask2Former. This model utilizes the ResNet50 model to enhance its feature extraction capabilities. The training phase is modified based on the Swin-Transformer module, which constrains cross-attention to extract local features within the predicted coverage area, reducing the computational effort to less than one-third of the original workload. Furthermore, the method can process multi-channel remote sensing data and incorporate terrain feature data into the model training to enhance landslide identification.

[0006] To achieve the above objectives, the present invention provides a Mask2Former landslide identification method based on terrain features, comprising:

[0007] S1: Collect remote sensing data of landslide areas and elevation data of corresponding pixels; pre-process the remote sensing data of landslide areas, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the pre-processed remote sensing image to obtain a multi-channel image;

[0008] S2: cutting, rotating and enhancing the obtained multi-channel image;

[0009] S3: Label the enhanced multi-channel images, construct image recognition training and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into a json file;

[0010] S4: Build an improved landslide recognition model. The improved landslide recognition model adds a channel for extracting terrain feature data in the backbone data extraction. Change the number of transformer layers for processing multi-channel data and modify the model's loss function so that the model considers the pixels of the terrain feature image during training. Modify the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images.

[0011] S5: Input the images of the image recognition training set and the json files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators;

[0012] S6: The model with the highest overall landslide identification accuracy index is selected and applied to landslide identification in the test set.

[0013] In one embodiment, step S1 preprocesses the remote sensing data of the landslide area, including image correction and pixel registration. Step S1 also includes evenly distributing the terrain feature data to each color of the single-channel image to visualize the terrain features; setting the terrain features as a transparency layer, and the terrain feature data includes surface roughness and slope.

[0014] In one embodiment, the multi-channel images in S2 are set to have a 60% overlap and are cropped, rotated, and enhanced for different sizes.

[0015] In one embodiment, the backbone network of the improved landslide recognition model in S4 adopts an improved Mask2Former model, including a multi-channel pixel decoder, a multi-channel Swin-Transformer decoder and a feature extraction part. The feature extraction part is a ResNet50-based model, wherein the multi-channel pixel decoder adopts a feature pyramid FPN architecture, takes a single-scale image of any size as input, and outputs feature maps of corresponding scale sizes at multiple levels in a fully convolutional manner. The multi-channel Swin-Transformer decoder uses masked attention to improve the accuracy of landslide recognition, limiting the decoder's attention to the foreground area of ​​each object. The ResNet50-based model can process multi-channel data.

[0016] In one embodiment, the improved landslide identification model in S4 uses binary cross entropy loss and Dice loss as mask loss, and uses the mask loss and classification loss as the final loss.

[0017] In one embodiment, the image recognition accuracy related indicators in S5 include AP, AP50, AP75, AP S 、AP m 、AP l , where AP represents the average precision, which is equal to the area under the precision-recall curve. AP50 and AP75 represent the AP values ​​corresponding to the thresholds of 0.5 and 0.75, respectively. S 、AP m 、AP l Represents the predicted values ​​considering only small objects, medium objects, and large objects, respectively.

[0018] In one embodiment, the testing process of step S6 includes:

[0019] The softmax function is used to convert the Logits output from the model training part into probabilities, automatically extract the mask and calculate the evaluation index of the landslide recognition model.

[0020] Based on the same inventive concept, the second aspect of the present invention provides a Mask2Former landslide identification device based on terrain features, comprising:

[0021] The data collection and preprocessing module is used to collect remote sensing data of areas with landslides and elevation data of corresponding pixels; preprocess the remote sensing data of areas with landslides, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the preprocessed remote sensing image to obtain a multi-channel image;

[0022] Image enhancement module, used to perform cutting, rotation and enhancement processing on the obtained multi-channel images;

[0023] The labeling module is used to label the enhanced multi-channel images, build image recognition training sets and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into a json file;

[0024] The model building module is used to build an improved landslide recognition model. The improved landslide recognition model adds a channel for extracting terrain feature data in the backbone data extraction; changes the number of transformer layers for processing multi-channel data and modifies the model's loss function so that the model considers the pixels of the terrain feature image during training; and modifies the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images.

[0025] The model training module is used to input the images of the image recognition training set and the JSON files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators;

[0026] The model testing module is used to select the model with the highest overall landslide identification accuracy index and apply it to landslide identification in the test set.

[0027] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the Mask2Former landslide identification method based on terrain features described in the first aspect.

[0028] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the Mask2Former landslide identification method based on terrain features described in the first aspect is implemented.

[0029] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0030] The present invention provides a Mask2Former landslide identification method based on terrain features and constructs an improved landslide identification model. The model uses a masked-attention architecture to constrain calculations to the predicted mask area, significantly reducing the memory occupied by model training. While improving the model training speed, the model incorporates terrain feature data into the calculation, reducing the impact of landslide-like objects such as rivers on landslide identification, and improving the accuracy of landslide identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 Flowchart of a landslide remote sensing image recognition method according to an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of terrain feature solution in an embodiment of the present invention;

[0034] Figure 3 This is a network structure diagram of network identification in an embodiment of the present invention;

[0035] Figure 4 This is the structural diagram of the feature extraction module;

[0036] Figure 5 This is the Bottleneck structure diagram;

[0037] Figure 6 A schematic diagram of terrain feature conversion results in an embodiment of the present invention;

[0038] Figure 7 A landslide identification result comparison diagram in an embodiment of the present invention;

[0039] Figure 8 Schematic diagram of the quantification result of landslide identification probability in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0041] Example 1

[0042] This embodiment discloses a Mask2Former landslide identification method based on terrain features, see Figure 1 ,include:

[0043] S1: Collect remote sensing data of landslide areas and elevation data of corresponding pixels; pre-process the remote sensing data of landslide areas, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the pre-processed remote sensing image to obtain a multi-channel image;

[0044] S2: cutting, rotating and enhancing the obtained multi-channel image;

[0045] S3: Label the enhanced multi-channel images, construct image recognition training and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into a json file;

[0046] S4: Build an improved landslide recognition model. The improved landslide recognition model adds a channel for extracting terrain feature data in the backbone data extraction. Change the number of transformer layers for processing multi-channel data and modify the model's loss function so that the model considers the pixels of the terrain feature image during training. Modify the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images.

[0047] S5: Input the images of the image recognition training set and the json files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators;

[0048] S6: Select the model with the highest overall landslide identification accuracy index and apply it to landslide identification in the test set.

[0049] Specifically, S1 collects remote sensing data of areas with landslides and fuses it with terrain feature data to obtain multi-channel image data. When cutting, S2 can cut it into different sizes according to actual needs, and then increase the amount of data that can be used through rotation processing, and perform data enhancement processing on the image to emphasize the main features of the image. S3 annotates the landslide samples, establishes a label library for the image recognition training set and test set, and cuts and rotates the labels in the same way as step S2, and converts the label symbols into a json file. S4 constructs a multi-channel recognition model based on the improved Mask2Former, that is, the improved landslide recognition model, adds a channel for terrain feature data extraction in the backbone data extraction; changes the number of layers of the transformer processing multi-channel data for the terrain feature channel, expanding 10 layers to 17 layers, and modifies the model's loss function so that the model can take the pixels of the terrain feature image into account during training; modifies the recognition part of the model so that the trained model can be applied to the recognition of multi-channel images.

[0050] S5 inputs the images in the image recognition training set and the json files corresponding to the labels processed in step S3 into the improved landslide recognition model, and writes a program to automatically save the current training model every 5000 epochs and automatically calculate image recognition accuracy related indicators, such as APl, AP50, etc.

[0051] S6 selected the model with the highest overall landslide recognition accuracy index and applied it to landslide recognition in the test set, automatically extracted the landslide mask, wrote a Python program to automatically calculate the uncertainty of landslide recognition, and calculated common image recognition indicators.

[0052] In one embodiment, step S1 preprocesses the remote sensing data of the landslide area, including image correction and pixel registration. Step S1 also includes evenly distributing the terrain feature data to each color of the single-channel image to visualize the terrain features; setting the terrain features as a transparency layer, and the terrain feature data includes surface roughness and slope.

[0053] In the specific implementation process, elevation and other terrain feature data are stored in txt format, and the corresponding numbers correspond to the elevation of each pixel in the satellite remote sensing image. The elevation data is automatically converted into other terrain data such as slope and surface roughness by Python and visualized as single-channel data. Figure 6 , which is a schematic diagram of the terrain feature conversion result in an embodiment of the present invention.

[0054] In one embodiment, the multi-channel images in S2 are set to have a 60% overlap and are cropped, rotated, and enhanced for different sizes.

[0055] In addition, the label image of the landslide should be cut in the same way as step S2, and the image size, label image save location and label ID type should be extracted, and the relevant data should be stored in json format.

[0056] In one embodiment, the backbone network of the improved landslide recognition model in S4 adopts an improved Mask2Former model, including a multi-channel pixel decoder, a multi-channel Swin-Transformer decoder and a feature extraction part. The feature extraction part is a ResNet50-based model, wherein the multi-channel pixel decoder adopts a feature pyramid FPN architecture, takes a single-scale image of any size as input, and outputs feature maps of corresponding scale sizes at multiple levels in a fully convolutional manner. The multi-channel Swin-Transformer decoder uses masked attention to improve the accuracy of landslide recognition, limiting the decoder's attention to the foreground area of ​​each object. The ResNet50-based model can process multi-channel data.

[0057] In one embodiment, the improved landslide identification model in S4 uses binary cross entropy loss and Dice loss as mask loss, and uses the mask loss and classification loss as the final loss.

[0058] In one embodiment, the image recognition accuracy related indicators in S5 include AP, AP50, AP75, AP S 、AP , 、AP l , where AP represents the average precision, which is equal to the area under the precision-recall curve. AP50 and AP75 represent the AP values ​​corresponding to the thresholds of 0.5 and 0.75, respectively. S 、AP m 、AP l Represents the predicted values ​​considering only small objects, medium objects, and large objects, respectively.

[0059] During the specific implementation process, the calculation formula for image recognition accuracy related indicators is as follows:

[0060]

[0061] In formula (1), N is the number of thresholds used, R m is the recall rate of the nth threshold, P n is the accuracy of the nth threshold, AP is the average precision equal to the area under the precision-recall curve. AP50 and AP75 refer to the AP values ​​corresponding to the thresholds of 0.5 and 0.75 respectively. S 、AP , 、AP l Represents the predicted values ​​considering only small objects, medium objects, and large objects, respectively.

[0062] In one embodiment, the testing process of step S6 includes:

[0063] The softmax function is used to convert the Logits output from the model training part into probabilities, automatically extract the mask and calculate the evaluation index of the landslide recognition model.

[0064] In the specific implementation process, the conversion is performed through the following formula:

[0065]

[0066] In formula (7), P landslide Refers to the probability of an area being identified as a landslide, L landslide Refers to the Logits value output by the model training part, It refers to the Logit value of the i-th pixel point identified as the landslide area.

[0067] In one embodiment, step S6 further includes: automatically extracting the mask and calculating the evaluation index of the landslide identification model by writing a Python program. The index calculation formula is as follows:

[0068]

[0069] Formula (8) and Formula (9) are the formulas corresponding to the precision and recall values, F1-score is the F1 score, mIOU is the mean intersection-over-union ratio, where TP is the number of true positives when the prediction is correct, FP is the number of false positives when the prediction is wrong, FN is the number of false negatives when the prediction is wrong, and TN is the number of true negatives when the prediction is correct.

[0070] Regarding terrain feature processing: the present invention aligns the pixel points of each channel data of the remote sensing image input, and writes a corresponding Python program based on the elevation data of each pixel point. By calculating the relationship between the elevation data of the surrounding pixel points and the elevation data of the central pixel point, the derived terrain feature data such as slope and surface roughness are obtained, and they are visualized as corresponding single-channel images respectively, and then fused with the remote sensing image to form multi-channel data and saved in tiff format.

[0071] Regarding the quantification of uncertainty in landslide identification: This model uses the softmax function to transform the logits value of each pixel obtained during model training to obtain the confidence that each pixel is identified as a landslide. These pixels exist in the area identified as a landslide. A Python program is written to identify individual landslides based on boundary connectivity to complete instance segmentation. The pixels belonging to a single landslide are identified, and the confidence of all pixels within it is weighted averaged to obtain the confidence that a single landslide area is identified as a landslide.

[0072] The present invention will be further described below by way of examples and in conjunction with the accompanying drawings:

[0073] See also Figure 1 As shown, the present invention provides a Mask2Former landslide identification method based on terrain features, the method comprising the following steps:

[0074] Step 1: Collect remote sensing data: Collect remote sensing images and ground elevation text data of corresponding pixel points. The ground elevation data is stored in txt format.

[0075] Convert terrain data into a single-channel image: Figure 2 As shown in the figure, the slope and surface roughness of the middle pixel are solved using the elevation data of the surrounding pixels, and Python code is written to automatically calculate them. The relevant calculation formula is as follows:

[0076]

[0077] Where e1-e8 are the elevations of the surrounding pixels, we and sn are the slope changes of adjacent pixels in two directions calculated based on the elevations. The indicator slope is the slope of the corresponding pixel, and roughness is the surface roughness of the pixel.

[0078] Then, a Python program was written to visualize the converted slope and surface roughness parameters as a single-channel image. Remote sensing data and terrain features were integrated: Terrain features were set as a transparency layer, and a Python program was written to integrate the data from each channel.

[0079] Step 2: Crop and rotate the images and perform data augmentation: Crop the training set images with 60% overlap, while the test set images are not overlapped. Data augmentation is added to the feature extraction process of the model, and automatic flipping is added to the training process to increase the amount of data.

[0080] Step 3: Label the landslide samples and convert the labels into JSON format: mark landslide objects as 1 and non-landslide data as 0. Write a Python program to record the size of each cropped image and the location of the pixels marked as landslides, and convert the relevant data into a JSON file.

[0081] Step 4: Build the improved Mask2Former model:

[0082] Step 5: Input the data into the improved Mask2Former model for training: The model input data includes the multi-channel fused image data of the segmented training set and test set and the label data in json format. The image data are all stored in tiff format.

[0083] The present invention uses pytorch as the deep learning framework, is developed based on the Detectron2 framework, and uses Mask2ormer as the baseline network model. The runtime environment parameters are shown in Table 1.

[0084] Table 1

[0085]

[0086] The model's feature extraction backbone is a ResNet50 model, with training modified from the swin-transformer module. The model was trained on four GPUs with a batch size of 16 and fp16 training to reduce memory usage. The experimental parameters for model training and testing are shown in Table 2.

[0087] Table 2

[0088]

[0089]

[0090] The model captures the global context in the image and generates high-quality, accurate masks directly from the encoded features. At the same time, the model uses an end-to-end trainable model, eliminating the need for separate architectures for different segmentation tasks. This simplifies the training process and improves the scalability and efficiency of the recognition model when processing datasets.

[0091] Select the trained model with the highest accuracy and apply it to landslide identification: A Python program was written to automatically export the trained model and save it as a PTH file. The model applied the PTH file to landslide identification on the test set and automatically calculated the relative accuracy of the object recognition operation, exporting accuracy metrics such as AP, AP50, AP75, APs, APm, and AP1. The experimenter manually selected the model with the highest accuracy and applied it to the landslide identification task on the test set.

[0092] In addition, the recognition mask is automatically extracted to obtain the recognition result, and the uncertainty and evaluation index of the recognition area are calculated: the recognition result is set as a red mask with 30% transparency, and a related program is written to extract the red mask. This embodiment tests the landslide images recognized by different terrain feature fusion scheme models, such as Figure 7 As shown in Figure 1 ((a) is the Mask2Former model recognition result, (b) is the Mask2Former model recognition result based on elevation data, (c) is the Mask2Former model recognition result based on slope data, and (d) is the Mask2Former model recognition result based on surface roughness data). The model identifies each landslide based on the connectivity of the graph and calculates the uncertainty of each landslide recognition based on the uncertainty of all pixels within a single landslide. The schematic diagram of the calculated landslide recognition probability quantification result is shown in Figure 1. Figure 8 shown.

[0093] The following describes a network structure used by the aforementioned model for feature extraction:

[0094] In this embodiment, the feature extraction portion of the model is modified based on the ResNet50 model. The ResNet50 model primarily addresses the problem of model accuracy degradation caused by vanishing gradients in deep neural networks. The ResNet50 model uses residual connections to skip the convolutional blocks between two Relu activation units, allowing the model to learn the residual function instead of directly learning the underlying mapping. This allows the model to skip the regularization process during feature extraction, and the model performance is not affected by the number of added layers.

[0095] This model consists of 16 residual blocks, each of which consists of a convolutional layer with residual connections. The architecture also includes a pooling layer, a fully connected layer, and a softmax output layer for classification.

[0096] Model architecture such as Figure 4 As shown in the figure, this embodiment increases the number of channels that the RseNet50 model can process. In the input layer, the model uses a 256*256*4 image as input, where 4 is the number of channels in the model input. The input layer of the model consists of a convolutional layer, a batch normalization layer, a ReLU layer, and a max pooling layer.

[0097] The convolutional layer uses the building block of the convolutional neural network (CNN) to extract features from the image and generate a set of feature maps for input to the next layer of the network.

[0098] The feature maps are then batch normalized, which allows the model to converge faster and achieve improved generalization and regularization.

[0099] The activation function of the input layer uses the Relu activation function, which is defined as:

[0100] Relu(x)=max(0,x) (16)

[0101] When the input is positive, the Relu function is equal to the output, otherwise it is equal to 0. Relu is a piecewise linear function that introduces nonlinearity into the neuron output, improves the efficiency of model calculation and differentiation, and can effectively prevent the gradient disappearance problem.

[0102] The input layer finally uses a max pooling layer to reduce the spatial dimensions of the feature map generated by the convolutional layer. The max pooling layer automatically selects the maximum value in each window, retaining the most important features in the input feature map while reducing the impact of noise and small changes.

[0103] The middle part of ResNet50 is composed of BottleNeck modules. Figure 5 As shown in the figure. The BottleNeck module consists of three convolutional layers: 1×1 convolution, 3×3 convolution, and 1×1 convolution. The 1×1 convolution is responsible for reducing and then increasing (restoring) the dimension, making the 3×3 convolution a bottleneck with smaller input / output dimensions. The model places batch normalization and Relu activation modules after each convolutional layer, but the last layer adds a skip connection before the Relu activation. In addition, the first two convolutional layers have the same number of filters, and the last convolutional layer has four times the number of filters of the previous convolutional layer.

[0104] The model first uses a global average pooling module at the output to reduce the spatial size of the output tensor to a vector. This operation calculates the average of each feature map, resulting in a feature vector with the same number of channels as the number of filters in the last convolutional layer. The model then uses a data compaction module to convert the output of the previous layer into a 1D vector, which is then fed into a fully connected layer. Each neuron in a fully connected layer receives input from all neurons in the previous layer, and its output is calculated as the weighted sum of these inputs.

[0105] The framework of the improved Mask2Former model backbone network is as follows Figure 3 The model is based on the DETR architecture and uses a transformer decoder to predict the mask of each object. The following is a detailed description of the model's backbone network structure.

[0106] Multi-channel feature extraction: The model adopts the ResNet50 structure mentioned above. In order to take into account the terrain characteristics, the present invention modifies its model extraction structure so that it can incorporate multi-channel data into the model calculation.

[0107] Multi-channel pixel decoder: It adopts the feature pyramid FPN architecture, takes a single-scale image of any size as input, and outputs feature maps of corresponding scales at multiple levels in a fully convolutional manner. Since landslide recognition is an instance segmentation task with a small number of recognition types, the present invention uses a multi-scale attention converter (MSDeformAttn) as a multi-channel pixel decoder, applies 6 MSDeformAttn layers to feature maps with resolutions of 1 / 8, 1 / 16, 1 / 16, and 1 / 32, and uses a simple upsampling layer connected laterally on the final 1 / 8 feature map to generate feature maps of resolution Figure 1 / 4 as per-pixel embedding.

[0108] Multi-channel Swin-Transformer decoder: This implementation uses masked attention to improve landslide recognition accuracy, limiting the decoder's attention to the foreground area of ​​each object. The model sets the mask value of the masked area to 0 and the unmasked area to 1, indicating that the real pixels here should be included in the calculation. Masked attention helps the decoder focus on the relevant features of each object and prevents the decoder from focusing on background noise. At the same time, to enable the model to better recognize landslides of different sizes, the system directs a single scale of the multi-scale features to a separate Swin-Transformer decoder layer during each decoder iteration. Therefore, each Swin-Transformer decoder layer processes features at scales of 1 / 8, 1 / 16, 1 / 16, and 1 / 32.

[0109] The model samples K random points from the predicted mask for loss calculation, which significantly reduces the GPU memory usage of the model training. The model uses binary cross entropy loss and Dice loss as the mask loss, and the calculation formula is as follows:

[0110] L mask =m ce L ce +m dice L dice (17)

[0111] m ce 、m dice Both are set to 5.0, and the final loss is a combination of the mask loss and the classification loss.

[0112] L total =L mask +m cls L cls (18)

[0113] For predictions that match the ground truth, set L ce = 2.0; for predictions that do not match any ground truth, set L ce =0.1.

[0114] Regarding the quantification of uncertainty in model identification of landslides, since landslide identification has only one category, “landslide”, and the probability that an object belongs to a landslide is 1, the present invention only considers the probability that a pixel is identified as a mask.

[0115] The model first uses the ResNet50 feature extraction network to extract features from the input image and convert the original image X into the high-dimensional feature map required for subsequent analysis. The feature extraction network is denoted as F(X):

[0116] F(X)=Backbone(X) (19)

[0117] These feature maps are processed through a transformation encoder to generate the corresponding feature representation Fe:

[0118] F e =TransformerEncoder(F(X)) (20)

[0119] The model uses the learned query embedding matrix to represent various potential landslide instances, helping to guide the attention mechanism to the relevant areas in the feature map. The attention weight A is calculated as follows:

[0120]

[0121] Where K is the key matrix from the feature map, d kis the dimension of the key vector. The resulting attention output Q focuses on areas where landslides are likely to occur.

[0122] In the prediction head, the model generates logits for category and mask predictions. For category probabilities, the logits Lc are transformed using the sigmoid function:

[0123] L m =W m O+b m (twenty two)

[0124]

[0125] Where W m and b m is the weight matrix and bias for mask prediction, L m represents the logit of the mask prediction, O represents the input feature or the output of the previous layer, usually the output from a convolutional neural network (CNN) or other feature extraction network, and P m represents the predicted probability of the mask.

[0126] Example 2

[0127] Based on the same inventive concept, this embodiment discloses a Mask2Former landslide identification device based on terrain features, comprising:

[0128] The data collection and preprocessing module is used to collect remote sensing data of areas with landslides and elevation data of corresponding pixels; preprocess the remote sensing data of areas with landslides, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the preprocessed remote sensing image to obtain a multi-channel image;

[0129] Image enhancement module, used to perform cutting, rotation and enhancement processing on the obtained multi-channel images;

[0130] The labeling module is used to label the enhanced multi-channel images, build image recognition training sets and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into a json file;

[0131] The model building module is used to build an improved landslide recognition model. The improved landslide recognition model adds a channel for extracting terrain feature data in the backbone data extraction; changes the number of layers of the transformer to process multi-channel data, and modifies the model for calculating the loss function so that the model considers the pixels of the terrain feature image during the training process; and modifies the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images.

[0132] The model training module is used to input the images of the image recognition training set and the JSON files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators;

[0133] The model testing module is used to select the model with the highest overall landslide identification accuracy index and apply it to landslide identification in the test set.

[0134] Since the device described in Example 2 of the present invention is used to implement the Mask2Former landslide identification method based on terrain features described in Example 1 of the present invention, the specific structure and variations of the device are readily understood by those skilled in the art based on the method described in Example 1 of the present invention, and thus will not be described in detail here. All devices used in the method described in Example 1 of the present invention fall within the scope of protection of the present invention.

[0135] Example 3

[0136] Based on the same inventive concept, the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the first embodiment is implemented.

[0137] Since the computer-readable storage medium described in the third embodiment of the present invention is used to implement the Mask2Former landslide identification method based on terrain features described in the first embodiment of the present invention, the specific structure and variations of the computer-readable storage medium are readily understood by those skilled in the art based on the method described in the first embodiment of the present invention, and thus will not be further described here. All computer-readable storage media used in the method of the first embodiment of the present invention fall within the scope of protection of the present invention.

[0138] Example 4

[0139] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first embodiment when executing the program.

[0140] Since the computer device described in Example 4 of the present invention is used to implement the Mask2Former landslide identification method based on terrain features described in Example 1 of the present invention, the specific structure and variations of the computer device are readily understood by those skilled in the art based on the method described in Example 1 of the present invention, and thus will not be described in detail here. All computer devices used in the method of Example 1 of the present invention fall within the scope of protection of the present invention.

[0141] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, the present invention is intended to include such changes and modifications to the embodiments of the present invention if they fall within the scope of the claims and their equivalents.

Claims

1. A Mask2Former landslide identification method based on terrain features, characterized in that: include: S1: Collect remote sensing data of landslide areas and elevation data of corresponding pixels; Preprocess the remote sensing data of the landslide area, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the preprocessed remote sensing image to obtain a multi-channel image; S2: cutting, rotating and enhancing the obtained multi-channel image; S3: Label the enhanced multi-channel images, construct image recognition training and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into a json file; S4: Construct an improved landslide recognition model, wherein the improved landslide recognition model adds a channel for terrain feature data extraction in the backbone data extraction; changes the number of layers of the transformer to process multi-channel data, and modifies the loss function of the model so that the model considers the pixels of the terrain feature image during training; modifies the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images. The backbone network of the improved landslide recognition model adopts an improved Mask2Former model, including a multi-channel pixel decoder, a multi-channel Swin-Transformer decoder and a feature extraction part. The feature extraction part is a model based on ResNet50. The multi-channel pixel decoder adopts a feature pyramid FPN architecture, takes a single-scale image of any size as input, and outputs feature maps of corresponding scales at multiple levels in a fully convolutional manner. The multi-channel Swin-Transformer decoder uses masked attention to improve the accuracy of landslide recognition, limiting the decoder's attention to the foreground area of ​​each object. The model based on ResNet50 can process multi-channel data. S5: Input the images of the image recognition training set and the json files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators; S6: The model with the highest overall landslide identification accuracy index is selected and applied to landslide identification in the test set.

2. The Mask2Former landslide identification method based on terrain features according to claim 1, characterized in that: Step S1 preprocesses the remote sensing data of the landslide area, including image correction and pixel registration. Step S1 also includes evenly distributing the terrain feature data to each color of the single-channel image to visualize the terrain features; setting the terrain features as a transparency layer, and the terrain feature data includes surface roughness and slope.

3. The Mask2Former landslide identification method based on terrain features according to claim 2, characterized in that: The multi-channel images in S2 were set to 60% overlap and cropped, rotated, and enhanced for different sizes.

4. The Mask2Former landslide identification method based on terrain features according to claim 1, wherein: The improved landslide recognition model in S4 uses binary cross entropy loss and Dice loss as mask loss, and the combination of mask loss and classification loss is used as the final loss.

5. The Mask2Former landslide identification method based on terrain features according to claim 1, characterized in that: The image recognition accuracy related indicators in S5 include AP, AP50, AP75, 、 、 , where AP represents the average precision, which is equal to the area under the precision-recall curve, and AP50 and AP75 represent the AP values ​​corresponding to the thresholds of 0.5 and 0.75, respectively. 、 、 Represents the predicted values ​​considering only small objects, medium objects, and large objects, respectively.

6. The Mask2Former landslide identification method based on terrain features according to claim 1, wherein: The testing process of step S6 includes: The softmax function is used to convert the Logits output from the model training part into probabilities, automatically extract the mask and calculate the evaluation index of the landslide recognition model.

7. A Mask2Former landslide identification device based on terrain features, characterized in that: include: Data collection and preprocessing module, used to collect remote sensing data of landslide areas and elevation data of corresponding pixels; Preprocess the remote sensing data of the landslide area, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the preprocessed remote sensing image to obtain a multi-channel image; Image enhancement module, used to perform cutting, rotation and enhancement processing on the obtained multi-channel images; The labeling module is used to label the enhanced multi-channel images, build image recognition training sets and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into a json file; A model building module is used to build an improved landslide recognition model, wherein the improved landslide recognition model adds a channel for terrain feature data extraction in the backbone data extraction; changes the number of layers of the transformer to process multi-channel data, and modifies the loss function of the model so that the model considers the pixels of the terrain feature image during training; modifies the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images. The backbone network of the improved landslide recognition model adopts an improved Mask2Former model, including a multi-channel pixel decoder, a multi-channel Swin-Transformer decoder and a feature extraction part. The feature extraction part is a model based on ResNet50. The multi-channel pixel decoder adopts a feature pyramid FPN architecture, takes a single-scale image of any size as input, and outputs feature maps of corresponding scales at multiple levels in a fully convolutional manner. The multi-channel Swin-Transformer decoder uses masked attention to improve the accuracy of landslide recognition, limiting the decoder's attention to the foreground area of ​​each object. The model based on ResNet50 can process multi-channel data. The model training module is used to input the images of the image recognition training set and the JSON files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators; The model testing module is used to select the model with the highest overall landslide identification accuracy index and apply it to landslide identification in the test set.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the Mask2Former landslide identification method based on terrain features as claimed in any one of claims 1 to 6 is implemented.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the Mask2Former landslide identification method based on terrain features according to any one of claims 1 to 6 is implemented.

Images