Mask2Former landslide identification method and device based on topographic features

Abstract

The invention discloses a Mask2Former landslide identification method and a Mask2Former landslide identification device based on topographic features, and provides a deep learning landslide identification method which is established based on an improved Mask2Former and considers the topographic features, so that the uncertainty of landslide identification can be calculated; and the influence of objects such as rivers and river channels in a research area on the landslide identification accuracy is solved. The method comprises the following steps: collecting and converting related topographic feature data; fusing the topographic features with the remote sensing image; performing cutting, rotation and data enhancement operation on the processed remote sensing image; labeling a landslide sample and converting a label into json; related data are input into an improved Mask2Former model for training; selecting a training model with the highest API index, and applying the training model to landslide identification; and automatically extracting the identification mask to obtain an identification result, and automatically calculating the uncertainty of an identification area and an identification evaluation index.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and more specifically, to a Mask2Former landslide recognition method and device based on terrain features. Background Art

[0002] Landslides have caused extensive damage to infrastructure and casualties, and accurate and rapid landslide identification is critical for disaster relief. There is an increasing demand for accurate, real-time landslide identification capabilities that can quickly calculate and automatically process terrain features. Traditional methods (such as visual interpretation and field surveys) are labor-intensive, inefficient, and unable to meet real-time identification needs. Therefore, there is a high demand for automated and semi-automated landslide identification technologies in the landslide identification field.

[0003] Landslide recognition methods are mainly divided into pixel-based methods and object-oriented methods. Pixel-based methods analyze individual pixel values ​​but ignore geometric and contextual features. In contrast, object-oriented methods divide the region into non-overlapping areas and consider spatial, textural, contextual, geometric and spectral features. Traditional machine learning methods cannot determine the optimal scale of different regions, which affects the accuracy of recognition results.

[0004] Deep learning has become a promising solution to these challenges. Deep learning models such as Mask R-CNN, DenseNet, and YOLO5 have higher accuracy in landslide identification, but are still disturbed by objects such as rivers, and there is room for improvement in accuracy. At the same time, current deep learning methods consume a lot of memory and have slow processing speed when considering terrain features, which hinders the application of terrain features to real-time identification of landslides. Summary of the invention

[0005] The purpose of this invention is to solve the problem of insufficient accuracy and excessive computing power consumption of existing deep learning models in identifying landslides. A Mask2Former landslide identification method based on terrain features is proposed. The model uses the ResNet50 model to improve the feature extraction ability of the model; the training part is modified based on the swin-transformer module, constraining the cross attention to extract local features in the predicted coverage area, reducing the amount of calculation to less than 1 / 3 of the original workload. At the same time, this method can process multi-channel remote sensing data and incorporate terrain feature data into the model training to improve the effect of landslide identification.

[0006] To achieve the above object, the present invention provides a Mask2Former landslide identification method based on terrain features, comprising:

[0007] S1: Collect remote sensing data of the area with landslides and the elevation data of the corresponding pixels; pre-process the remote sensing data of the area with landslides, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the pre-processed remote sensing image to obtain a multi-channel image;

[0008] S2: cutting, rotating and enhancing the obtained multi-channel images;

[0009] S3: Annotate the enhanced multi-channel images, construct image recognition training sets and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into json files;

[0010] S4: Build an improved landslide recognition model, where the improved landslide recognition model adds a channel for terrain feature data extraction in the backbone data extraction; change the number of layers of the transformer to process multi-channel data, and modify the model's loss function so that the model considers the pixels of the terrain feature image during training; modify the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images;

[0011] S5: Input the images of the image recognition training set and the json files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators;

[0012] S6: Select the model with the highest overall landslide identification accuracy index and apply it to landslide identification in the test set.

[0013] In one embodiment, step S1 preprocesses the remote sensing data of the landslide area including image correction and pixel registration. Step S1 also includes evenly distributing the terrain feature data to each color of the single-channel image to visualize the terrain features; setting the terrain features as a transparency layer, and the terrain feature data includes surface roughness and slope.

[0014] In one embodiment, the multi-channel images in S2 are set to have an overlap of 60% and are cropped, rotated, and enhanced for different sizes.

[0015] In one embodiment, the backbone network of the improved landslide recognition model in S4 adopts an improved Mask2Former model, including a multi-channel pixel decoder, a multi-channel Swin-Transformer decoder and a feature extraction part. The feature extraction part is a model based on ResNet50, wherein the multi-channel pixel decoder adopts a feature pyramid FPN architecture, takes a single-scale image of any size as input, and outputs feature maps of corresponding proportional sizes at multiple levels in a fully convolutional manner. The multi-channel Swin-Transformer decoder uses masked attention to improve the accuracy of landslide recognition, and limits the decoder's attention to the foreground area of ​​each object. The model based on ResNet50 can process multi-channel data.

[0016] In one implementation, the improved landslide identification model in S4 uses binary cross entropy loss and Dice loss as mask loss, and uses the mask loss and classification loss as the final loss.

[0017] In one embodiment, the image recognition accuracy related indicators in S5 include AP, AP50, AP75, AP S 、AP m 、AP l , where AP means average precision, which is equal to the area under the precision-recall curve. AP50 and AP75 refer to the AP values ​​corresponding to the thresholds of 0.5 and 0.75, respectively. S ,AP m ,AP l Respectively represent the predicted values ​​considering only small objects, medium objects, and large objects.

[0018] In one embodiment, the testing process of step S6 includes:

[0019] The softmax function is used to convert the Logits output from the model training part into probabilities, automatically extract the mask and calculate the evaluation index of the landslide recognition model.

[0020] Based on the same inventive concept, the second aspect of the present invention provides a Mask2Former landslide identification device based on terrain features, comprising:

[0021] The data collection and preprocessing module is used to collect remote sensing data of the landslide area and the elevation data of the corresponding pixels; preprocess the remote sensing data of the landslide area, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the preprocessed remote sensing image to obtain a multi-channel image;

[0022] An image enhancement module is used to perform cutting, rotation and enhancement processing on the obtained multi-channel images;

[0023] The labeling module is used to label the enhanced multi-channel images, construct image recognition training sets and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into json files;

[0024] The model building module is used to build an improved landslide recognition model, wherein the improved landslide recognition model adds a channel for terrain feature data extraction in the backbone data extraction; changes the number of layers of the transformer for processing multi-channel data, and modifies the loss function of the model so that the model considers the pixels of the terrain feature image during the training process; modifies the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images;

[0025] The model training module is used to input the images of the image recognition training set and the json files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators;

[0026] The model testing module is used to select the model with the highest overall landslide identification accuracy index and apply it to the landslide identification of the test set.

[0027] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the Mask2Former landslide identification method based on terrain features described in the first aspect.

[0028] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the Mask2Former landslide identification method based on terrain features described in the first aspect is implemented.

[0029] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0030] The present invention provides a Mask2Former landslide identification method based on terrain features and constructs an improved landslide identification model. The model uses a masked-attention architecture to constrain calculations to the predicted mask area, greatly reducing the memory occupied by model training. While improving the model training speed, the model incorporates terrain feature data into the calculation, reduces the influence of landslide-like objects such as rivers on landslide identification, and improves the accuracy of landslide identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 is a flow chart of a landslide remote sensing image recognition method in an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of terrain feature solution in an embodiment of the present invention;

[0034] Figure 3 A network structure diagram of network identification in an embodiment of the present invention;

[0035] Figure 4 This is the structural diagram of the feature extraction module;

[0036] Figure 5 This is the Bottleneck structure diagram;

[0037] Figure 6 A schematic diagram of the terrain feature conversion result in an embodiment of the present invention;

[0038] Figure 7 A landslide identification result comparison diagram in an embodiment of the present invention;

[0039] Figure 8 It is a schematic diagram of the quantification result of landslide identification probability in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Embodiment 1

[0042] This embodiment discloses a Mask2Former landslide identification method based on terrain features, see Figure 1 ,include:

[0043] S1: Collect remote sensing data of the area with landslides and the elevation data of the corresponding pixels; pre-process the remote sensing data of the area with landslides, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the pre-processed remote sensing image to obtain a multi-channel image;

[0044] S2: cutting, rotating and enhancing the obtained multi-channel images;

[0045] S3: Annotate the enhanced multi-channel images, construct image recognition training sets and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into json files;

[0046] S4: Build an improved landslide recognition model, where the improved landslide recognition model adds a channel for terrain feature data extraction in the backbone data extraction; change the number of layers of the transformer to process multi-channel data, and modify the model's loss function so that the model considers the pixels of the terrain feature image during training; modify the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images;

[0047] S5: Input the images of the image recognition training set and the json files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators;

[0048] S6: Select the model with the highest overall landslide identification accuracy index and apply it to landslide identification in the test set.

[0049] Specifically, S1 collects remote sensing data of landslide areas and fuses them with terrain feature data to obtain multi-channel image data. When cutting, S2 can cut into different sizes according to actual needs, and then increase the amount of data that can be used through rotation processing, and perform data enhancement processing on the image to emphasize the main features of the image. S3 annotates landslide samples, establishes a label library for image recognition training sets and test sets, and cuts and rotates the labels in the same way as step S2, and converts the label symbols into json files. S4 builds a multi-channel recognition model based on the improved Mask2Former, that is, the improved landslide recognition model, adds channels for terrain feature data extraction in the backbone data extraction; changes the number of layers of transformer processing multi-channel data for the terrain feature channel, expands 10 layers to 17 layers, and modifies the model's loss function so that the model can take the pixels of the terrain feature image into account during the training process; modifies the recognition part of the model so that the trained model can be applied to the recognition of multi-channel images.

[0050] S5 inputs the images in the image recognition training set and the json files corresponding to the labels processed by step S3 into the improved landslide recognition model, and writes a program to automatically save the training model at that time every 5000 epochs, and automatically calculate image recognition accuracy related indicators, such as APl, AP50, etc.

[0051] S6 selected the model with the highest overall landslide recognition accuracy index and applied it to the landslide recognition of the test set, automatically extracted the landslide mask, wrote a python program to automatically calculate the uncertainty of landslide recognition, and calculated the common image recognition indicators.

[0052] In one embodiment, step S1 preprocesses the remote sensing data of the landslide area including image correction and pixel registration. Step S1 also includes evenly distributing the terrain feature data to each color of the single-channel image to visualize the terrain features; setting the terrain features as a transparency layer, and the terrain feature data includes surface roughness and slope.

[0053] In the specific implementation process, the terrain feature data such as elevation is stored in txt format, and the corresponding numbers correspond to the elevation of each pixel in the satellite remote sensing image. The elevation data is automatically converted into other terrain data such as slope and surface roughness by python, and visualized as single-channel data. Figure 6 , which is a schematic diagram of the terrain feature conversion result in an embodiment of the present invention.

[0054] In one embodiment, the multi-channel images in S2 are set to have an overlap of 60% and are cropped, rotated, and enhanced for different sizes.

[0055] In addition, the label image of the landslide should be cut in the same way as step S2, and the image size, label image save location and label ID type should be extracted, and the relevant data should be stored in json format.

[0056] In one embodiment, the backbone network of the improved landslide recognition model in S4 adopts an improved Mask2Former model, including a multi-channel pixel decoder, a multi-channel Swin-Transformer decoder and a feature extraction part. The feature extraction part is a model based on ResNet50, wherein the multi-channel pixel decoder adopts a feature pyramid FPN architecture, takes a single-scale image of any size as input, and outputs feature maps of corresponding proportional sizes at multiple levels in a fully convolutional manner. The multi-channel Swin-Transformer decoder uses masked attention to improve the accuracy of landslide recognition, and limits the decoder's attention to the foreground area of ​​each object. The model based on ResNet50 can process multi-channel data.

[0057] In one implementation, the improved landslide identification model in S4 uses binary cross entropy loss and Dice loss as mask loss, and uses the mask loss and classification loss as the final loss.

[0058] In one embodiment, the image recognition accuracy related indicators in S5 include AP, AP50, AP75, AP S ,AP , ,AP l , where AP means average precision, which is equal to the area under the precision-recall curve. AP50 and AP75 refer to the AP values ​​corresponding to the thresholds of 0.5 and 0.75, respectively. S ,AP m ,AP l Respectively represent the predicted values ​​considering only small objects, medium objects, and large objects.

[0059] In the specific implementation process, the calculation formula of the image recognition accuracy related indicators is as follows:

[0060]

[0061] In formula (1), N is the number of thresholds used, R m is the recall rate of the nth threshold, P n is the accuracy of the nth threshold, AP is the average precision equal to the area under the precision-recall curve. AP50 and AP75 refer to the AP values ​​corresponding to the thresholds of 0.5 and 0.75 respectively. S ,AP , ,AP l Respectively represent the predicted values ​​considering only small objects, medium objects, and large objects.

[0062] In one embodiment, the testing process of step S6 includes:

[0063] The softmax function is used to convert the Logits output from the model training part into probabilities, automatically extract the mask and calculate the evaluation index of the landslide recognition model.

[0064] In the specific implementation process, the conversion is performed through the following formula:

[0065]

[0066] In formula (7), P landslide Refers to the probability of an area being identified as a landslide, L landslide Refers to the Logits value output by the model training part, It refers to the Logit value of the i-th pixel point identified as the landslide area.

[0067] In one embodiment, step S6 further includes: automatically extracting the mask and calculating the evaluation index of the landslide identification model by writing a python program. The index calculation formula is as follows:

[0068]

[0069] Formula (8) and Formula (9) are the formulas corresponding to the precision and recall values, F1-score is the F1 score, mIOU is the mean intersection-over-union ratio, where TP is the number of true positives when the prediction is correct, FP is the number of false positives when the prediction is wrong, FN is the number of false negatives when the prediction is wrong, and TN is the number of true negatives when the prediction is correct.

[0070] Regarding terrain feature processing: the present invention aligns the pixel points of each channel data of the remote sensing image input, writes the corresponding Python program according to the elevation data of each pixel point, calculates the relationship between the elevation data of the surrounding pixel points and the elevation data of the central pixel point, and obtains the derived terrain feature data such as slope and surface roughness, and visualizes them as corresponding single-channel images respectively, which are fused with the remote sensing image to form multi-channel data and saved in tiff format.

[0071] Regarding the quantification of uncertainty in landslide identification: This model uses the softmax function to transform the Logits value of each pixel obtained from model training to obtain the confidence that each pixel is identified as a landslide. These pixels exist in the area identified as a landslide; a python program is written to identify a single landslide based on boundary connectivity and complete instance segmentation; the pixels belonging to a single landslide are identified, and the confidence of all the pixels within it is weighted averaged to obtain the confidence that a single landslide area is identified as a landslide.

[0072] The present invention will be further described below by way of embodiments and in conjunction with the accompanying drawings:

[0073] See also Figure 1 As shown, the present invention provides a Mask2Former landslide identification method based on terrain features, and the method comprises the following steps:

[0074] Step 1: Collect remote sensing data: Collect remote sensing images and ground elevation text data of corresponding pixel points. The ground elevation data is stored in txt format.

[0075] Convert terrain data into a single-channel image: Figure 2 As shown in the figure, the slope and surface roughness of the middle pixel are solved by using the elevation data of the surrounding pixels, and Python code is written to automatically calculate. The relevant calculation formula is as follows:

[0076]

[0077] Among them, e1-e8 is the elevation of the surrounding pixels, we and sn are the slope changes of adjacent pixels in two directions calculated based on the elevation. The indicator slope is the slope of the corresponding pixel, and roughness is the surface roughness of the pixel.

[0078] Then, a python program was written to visualize the converted slope and surface roughness parameters as a single-channel image. Fusion of remote sensing data and terrain features: The terrain features were set as a transparency layer, and a python program was written to fuse the data from each channel.

[0079] Step 2: Crop, rotate the image, and perform data enhancement: Crop the training set images with 60% overlap, and do not set overlap for the test set. Add data enhancement processing to the model feature extraction part, and add an automatic flipping part to the program training part to increase the amount of data.

[0080] Step 3: Label the landslide samples and convert the labels into JSON format: mark the landslide objects as 1 and non-landslide data as 0. Write a Python program to record the size of each cut image and the location of the pixels marked as landslides, and convert the relevant data into a JSON file.

[0081] Step 4: Build the improved Mask2Former model:

[0082] Step 5: Input the data into the improved Mask2Former model for training: The model input data includes the multi-channel fused image data of the training set and the test set after cutting and the label data in json format. The image data are all stored in tiff format.

[0083] The present invention uses pytorch as the deep learning framework, is developed based on the Detectron2 framework, and uses Mask2ormer as the baseline network model. The operating environment parameters are shown in Table 1.

[0084] Table 1

[0085]

[0086] The backbone network for model feature extraction is the ResNet50 model, and the training part is modified based on the swin-transformer module. The model is trained using 4 GPUs, the batch size batch_size is 16, and fp16 training is specified to reduce memory. The experimental parameters for model training and testing are shown in Table 2.

[0087] Table 2

[0088]

[0089]

[0090] The model captures the global context in the image and generates high-quality, accurate masks directly from the encoded features. At the same time, the model uses an end-to-end trainable model, eliminating the process of using separate architectures for different segmentation tasks to simplify the training process and improve the scalability and efficiency of the recognition model when processing data sets.

[0091] Select the training model with the highest accuracy index and apply it to landslide identification: Write a Python program to make the model automatically export its trained model and save it as a pth format file. The model applies the pth file to the landslide identification of the test set, and automatically calculates the relevant accuracy of the object recognition operation, and exports the accuracy indicators such as AP, AP50, AP75, APs, APm, and APl. The experimenter manually selects the model with the highest accuracy and applies it to the execution of the landslide identification task of the test set.

[0092] In addition, the identification mask is automatically extracted to obtain the identification result, and the uncertainty and evaluation index of the identification area are calculated: the identification result is set as a red mask with 30% transparency, and a related program is written to extract the red mask. This embodiment tests the landslide images identified by different terrain feature fusion scheme models, such as Figure 7 As shown in (a) the Mask2Former model recognition result, (b) the Mask2Former model recognition result based on elevation data, (c) the Mask2Former model recognition result based on slope data, and (d) the Mask2Former model recognition result based on surface roughness data). The model identifies each landslide based on the connectivity of the graph, and calculates the uncertainty of each landslide recognition based on the uncertainty of all pixels in a single landslide. The schematic diagram of the quantified result of the calculated landslide recognition probability is shown in Figure 8 shown.

[0093] The following is a network structure for feature extraction using the aforementioned model:

[0094] In this embodiment, the feature extraction part of the model is modified based on the ResNet50 model. The ResNet50 model mainly solves the problem of model accuracy degradation caused by gradient disappearance in deep neural networks. The ResNet50 model uses residual connections to skip the convolution blocks between two Relu activation units, so that the model can learn the residual function instead of directly learning the underlying mapping. This allows the model to skip the regularization process during feature extraction, and the performance of the model will not be affected by the number of added layers.

[0095] This model consists of 16 residual blocks, each of which consists of a convolutional layer with residual connections. The architecture also includes a pooling layer, a fully connected layer, and a softmax output layer for classification.

[0096] The model architecture is as follows Figure 4 As shown in the figure, this embodiment increases the number of channels that the RseNet50 model can process. In the input layer part, the model uses a 256*256*4 image as input, and 4 is the number of channels input to the model. The input part of the model consists of a convolutional layer, a batch normalization part, a Relu part, and a maximum pooling layer.

[0097] The convolutional layer uses the building block of the convolutional neural network (CNN) to extract features from the image and generate a set of feature maps for the input of the next layer of the network.

[0098] The feature maps are then batch normalized, which allows the model to converge faster and achieve improved generalization and regularization.

[0099] The activation function of the input layer uses the Relu activation function, which is defined as:

[0100] Relu(x)=max(0,x) (16)

[0101] When the input is positive, the Relu function is equal to the output, otherwise it is equal to 0. Relu is a piecewise linear function that introduces nonlinearity into the neuron output, improves the efficiency of model calculation and differentiation, and can effectively prevent the gradient disappearance problem.

[0102] The input layer finally uses a max pooling layer to reduce the spatial dimension of the feature map generated by the convolutional layer. The max pooling layer automatically selects the maximum value in each window, retaining the most important features in the input feature map while reducing the impact of noise and small changes.

[0103] The structure of the middle part of ResNet50 is composed of BottleNeck modules. Figure 5 As shown. The BottleNeck module consists of three convolutional layers, namely 1×1 convolution, 3×3 convolution, and 1×1 convolution. The 1×1 convolution is responsible for reducing and then increasing (restoring) the dimension, making the 3×3 convolution a bottleneck with smaller input / output dimensions. The model places batch normalization and Relu activation modules after each convolutional layer, but the last layer first adds a skip connection process before the Relu activation. In addition, the number of filters in the first two convolutional layers is the same, and the number of filters in the last convolutional layer is 4 times the number of filters in the previous convolutional layer.

[0104] The model first uses a global average pooling module in the output part to reduce the spatial size of the output tensor to a vector. This operation calculates the average value of each feature map, resulting in a feature vector with the same number of channels as the number of filters in the last convolutional layer. The model then uses a data compaction module to convert the output of the previous layer into a 1d vector, which is then input into the fully connected layer. Each neuron in the fully connected layer receives input from all neurons in the previous layer, and the output of each neuron is calculated as the weighted sum of these inputs.

[0105] The framework of the improved Mask2Former model backbone network is as follows Figure 3 The model is based on the DETR architecture and uses a transformer decoder to predict the mask of each object. The following is a detailed description of the model's backbone network structure.

[0106] Multi-channel feature extraction: The model adopts the ResNet50 structure mentioned above. In order to consider the terrain characteristics, the present invention modifies its model extraction structure so that it can incorporate multi-channel data into the model calculation.

[0107] Multi-channel pixel decoder: It adopts the feature pyramid FPN architecture, takes a single-scale image of any size as input, and outputs feature maps of corresponding scales at multiple levels in a fully convolutional manner. Since landslide recognition is an instance segmentation task with a small number of recognition types, the present invention uses a multi-scale attention transformer (MSDeformAttn) as a multi-channel pixel decoder, applies 6 MSDeformAttn layers to feature maps with resolutions of 1 / 8, 1 / 16, 1 / 16, and 1 / 32, and uses a simple upsampling layer connected laterally on the final 1 / 8 feature map to generate feature maps of resolution Figure 1 / 4 as per pixel embedding.

[0108] Multi-channel Swin-Transformer decoder: This implementation uses masked attention to improve landslide identification accuracy, limiting the decoder's attention to the foreground area of ​​each object. The model sets the mask value of the masked area to 0 and the unmasked area to 1, indicating that the real pixels should be included in the calculation here. Masked attention helps the decoder focus on the relevant features of each object and prevents the decoder from paying attention to background noise. At the same time, in order to enable the model to better identify landslides of different sizes, the system directs a single scale of the multi-scale features to a separate Swin-Transformer decoder layer during each iteration of the decoder. Therefore, each Swin-Transformer decoder layer processes features at scales of 1 / 8, 1 / 16, 1 / 16, and 1 / 32.

[0109] The model samples K random points from the predicted mask for loss calculation, which greatly reduces the GPU memory usage of model training. The model uses binary cross entropy loss and Dice loss as the Mask loss, and the calculation formula is as follows:

[0110] L mask =m ce L ce +m dice L dice (17)

[0111] m ce 、m dice Both are set to 5.0, and the final loss is a combination of the mask loss and the classification loss.

[0112] L total =L mask +m cls L cls (18)

[0113] For predictions that match the ground truth, set L ce = 2.0; for predictions that do not match any ground truth, set L ce =0.1.

[0114] Regarding the quantification of the uncertainty of the model identifying landslides, since there is only one category of "landslide" in landslide identification and the probability that an object belongs to a landslide is 1, the present invention only considers the probability that a pixel is identified as a mask.

[0115] The model first uses the ResNet50 feature extraction network to extract features from the input image and convert the original image X into a high-dimensional feature map required for subsequent analysis. The feature extraction network is denoted as F(X):

[0116] F(X)=Backbone(X) (19)

[0117] These feature maps are processed through a transformation encoder to generate the corresponding feature representation Fe:

[0118] F e =TransformerEncoder(F(X)) (20)

[0119] The model uses the learned query embedding matrix to represent various potential landslide instances, helping to guide the attention mechanism to the relevant areas in the feature map. The calculation method of attention weight A is as follows:

[0120]

[0121] Where K is the key matrix from the feature map, d kis the dimension of the key vector. The resulting attention output Q focuses on areas where landslides are likely to occur.

[0122] In the prediction head, the model generates logits for class and mask predictions. For class probabilities, the logits Lc are transformed using the sigmoid function:

[0123] L m =W m O+b m (twenty two)

[0124]

[0125] Where W m and b m is the weight matrix and bias of mask prediction, L m represents the logit of the mask prediction, O represents the input feature or the output of the previous layer, usually the output from a convolutional neural network (CNN) or other feature extraction network, and P m represents the predicted probability of the mask.

[0126] Embodiment 2

[0127] Based on the same inventive concept, this embodiment discloses a Mask2Former landslide identification device based on terrain features, comprising:

[0128] The data collection and preprocessing module is used to collect remote sensing data of the landslide area and the elevation data of the corresponding pixels; preprocess the remote sensing data of the landslide area, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the preprocessed remote sensing image to obtain a multi-channel image;

[0129] An image enhancement module is used to perform cutting, rotation and enhancement processing on the obtained multi-channel images;

[0130] The labeling module is used to label the enhanced multi-channel images, construct image recognition training sets and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into json files;

[0131] The model building module is used to build an improved landslide recognition model, wherein the improved landslide recognition model adds a channel for terrain feature data extraction in the backbone data extraction; changes the number of layers of the transformer to process multi-channel data, and modifies the model for calculating the loss function so that the model considers the pixels of the terrain feature image during the training process; modifies the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images;

[0132] The model training module is used to input the images of the image recognition training set and the json files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators;

[0133] The model testing module is used to select the model with the highest overall landslide identification accuracy index and apply it to the landslide identification of the test set.

[0134] Since the device introduced in the second embodiment of the present invention is a device used to implement the Mask2Former landslide identification method based on terrain features in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, the person skilled in the art can understand the specific structure and deformation of the device, so it is not repeated here. All devices used in the method in the first embodiment of the present invention belong to the scope of protection of the present invention.

[0135] Embodiment 3

[0136] Based on the same inventive concept, the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the first embodiment is implemented.

[0137] Since the computer-readable storage medium introduced in the third embodiment of the present invention is a computer-readable storage medium used to implement the Mask2Former landslide identification method based on terrain features in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of the computer-readable storage medium, so it is not repeated here. All computer-readable storage media used in the method of the first embodiment of the present invention belong to the scope of protection of the present invention.

[0138] Embodiment 4

[0139] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in Embodiment 1 when executing the program.

[0140] Since the computer device introduced in the fourth embodiment of the present invention is a computer device used to implement the Mask2Former landslide identification method based on terrain features in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, the person skilled in the art can understand the specific structure and deformation of the computer device, so it is not repeated here. All computer devices used in the method of the first embodiment of the present invention belong to the scope of protection of the present invention.

[0141] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic creative concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A Mask2Former landslide identification method based on terrain features, characterized in that: include: S1: Collect remote sensing data of landslide areas and elevation data of corresponding pixels; Preprocess the remote sensing data of the landslide area, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the preprocessed remote sensing image to obtain a multi-channel image; S2: cutting, rotating and enhancing the obtained multi-channel images; S3: Annotate the enhanced multi-channel images, construct image recognition training sets and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into json files; S4: Build an improved landslide recognition model, where the improved landslide recognition model adds a channel for terrain feature data extraction in the backbone data extraction; change the number of layers of the transformer to process multi-channel data, and modify the model's loss function so that the model considers the pixels of the terrain feature image during training; modify the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images; S5: Input the images of the image recognition training set and the json files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators; S6: Select the model with the highest overall landslide identification accuracy index and apply it to landslide identification in the test set.

2. The Mask2Former landslide identification method based on terrain features as claimed in claim 1, characterized in that: Step S1 preprocesses the remote sensing data of the landslide area including image correction and pixel registration. Step S1 also includes evenly distributing the terrain feature data to each color of the single-channel image to visualize the terrain features; setting the terrain features as a transparency layer, and the terrain feature data includes surface roughness and slope.

3. The Mask2Former landslide identification method based on terrain features as claimed in claim 2, characterized in that: The multi-channel images in S2 were set to 60% overlap and were cropped, rotated, and enhanced for different sizes.

4. The Mask2Former landslide identification method based on terrain features as claimed in claim 1, characterized in that: The backbone network of the improved landslide recognition model in S4 adopts an improved Mask2Former model, including a multi-channel pixel decoder, a multi-channel Swin-Transformer decoder and a feature extraction part. The feature extraction part is a model based on ResNet50. The multi-channel pixel decoder adopts a feature pyramid FPN architecture, takes a single-scale image of any size as input, and outputs feature maps of corresponding scales at multiple levels in a fully convolutional manner. The multi-channel Swin-Transformer decoder uses masked attention to improve the accuracy of landslide recognition, limiting the decoder's attention to the foreground area of ​​each object. The ResNet50-based model can process multi-channel data.

5. The Mask2Former landslide identification method based on terrain features as claimed in claim 1, characterized in that: The improved landslide recognition model in S4 uses binary cross entropy loss and Dice loss as mask loss, and takes mask loss and classification loss as the final loss.

6. The Mask2Former landslide identification method based on terrain features as claimed in claim 1, characterized in that: The image recognition accuracy related indicators in S5 include AP, AP50, AP75, AP S ,AP m ,AP l , where AP means average precision, which is equal to the area under the precision-recall curve. AP50 and AP75 refer to the AP values ​​corresponding to the thresholds of 0.5 and 0.75, respectively. S ,AP m ,AP l Respectively represent the predicted values ​​considering only small objects, medium objects, and large objects.

7. The Mask2Former landslide identification method based on terrain features as claimed in claim 1, characterized in that: The testing process of step S6 includes: The softmax function is used to convert the Logits output from the model training part into probabilities, automatically extract the mask and calculate the evaluation index of the landslide recognition model.

8. A Mask2Former landslide identification device based on terrain features, characterized in that: include: Data collection and preprocessing module, used to collect remote sensing data of landslide areas and elevation data of corresponding pixels; Preprocess the remote sensing data of the landslide area, automatically convert the elevation data into terrain feature data, and fuse the obtained terrain feature data with the preprocessed remote sensing image to obtain a multi-channel image; An image enhancement module is used to perform cutting, rotation and enhancement processing on the obtained multi-channel images; The labeling module is used to label the enhanced multi-channel images, construct image recognition training sets and test sets, cut and rotate the labels in the same way as step S2, and convert the label symbols into json files; The model building module is used to build an improved landslide recognition model, wherein the improved landslide recognition model adds a channel for terrain feature data extraction in the backbone data extraction; changes the number of layers of the transformer for processing multi-channel data, and modifies the loss function of the model so that the model considers the pixels of the terrain feature image during the training process; modifies the feature extraction part of the model so that the trained model can be applied to the recognition of multi-channel images; The model training module is used to input the images of the image recognition training set and the json files corresponding to the processed labels into the improved landslide recognition model for training, and automatically calculate the image recognition accuracy related indicators; The model testing module is used to select the model with the highest overall landslide identification accuracy index and apply it to the landslide identification of the test set.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the Mask2Former landslide identification method based on terrain features as claimed in any one of claims 1 to 7 is implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the Mask2Former landslide identification method based on terrain features as described in any one of claims 1 to 7 is implemented.

Images