Landslide remote sensing cross-domain extraction method based on image mask and form enhancement

Abstract

The invention relates to the technical field of image recognition, and particularly provides a landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement, which comprises the following steps: processing a target domain image by using a random mask through an image mask domain adaptation module, and forcing a student network to only access local information of the image; through the mask consistency loss of a pseudo label generated by a teacher network, students are promoted to learn context information in a local image through the network, and the landslide feature capturing capability of the model is improved; the method makes full use of the context information of the target domain image, helps the model to better understand the spatial relationship between the landslide and the surrounding features, extracts the morphological information of the landslide through the morphological information extraction and optimization module, further optimizes the morphological information to form a morphological pseudo-label, and guides the model to learn the morphological features of the landslide, thereby improving the accuracy of landslide identification. Therefore, the accuracy and integrity of landslide identification are improved.

Description

Technical Field

[0001] The invention relates to the technical field of image recognition, and in particular to a landslide remote sensing cross-domain extraction method based on image mask and morphology enhancement. Background Art

[0002] Landslide is a type of geological disaster that poses a serious threat to people's lives and property. Remote sensing technology provides an effective means for monitoring and identifying landslides. Remote sensing images can capture terrain changes before and after landslides. However, due to differences in natural conditions between different geographical regions, the feature expressions of landslides on remote sensing images are significantly different. This feature difference poses a challenge to the cross-domain extraction of landslides, especially when the model is applied to a geographical area different from the training data, the performance of the model often drops significantly. The existing solutions are as follows:

[0003] Domain adaptation methods: To solve the problem of feature differences in cross-domain landslide extraction, researchers have introduced domain adaptation methods. These methods aim to reduce the feature differences between the source domain and the target domain and improve the performance of the model on the target domain. For example, methods based on adversarial training and methods based on regularization. Methods based on adversarial training: Through adversarial training, the model learns domain-invariant feature representations while distinguishing the features of the source domain and the target domain. Regularization-based methods: By adding regularization terms, the difference between the output of the model on the target domain and the output on the source domain is limited, thereby reducing the difference between domains.

[0004] Object detection and few-shot learning: In the field of object detection, deep feature-based object detection methods have made significant progress due to the powerful feature extraction capabilities of convolutional neural networks. At the same time, few-shot learning methods have also been applied to landslide identification to address the problem of scarce labeled data.

[0005] Knowledge distillation: Knowledge distillation is a method of model compression and transfer learning that extracts knowledge from a complex model to train a simpler model. In landslide identification, knowledge distillation can be used to transfer the performance of a large model to a small model to reduce computational costs and increase inference speed.

[0006] These existing implementations still have some shortcomings, such as focusing mainly on feature-level alignment while ignoring the use of contextual information, and the difficulty in accurately capturing the morphological features of landslides, which leads to incomplete capture of landslide morphological features and limited recognition accuracy.

[0007] In summary, there is an urgent need to provide a cross-domain landslide remote sensing extraction method based on image masking and morphological enhancement to improve the accuracy of landslide identification. Summary of the invention

[0008] The purpose of the present invention is to provide a landslide remote sensing cross-domain extraction method based on image masking and morphological enhancement. The specific scheme is as follows:

[0009] A landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement includes the following steps:

[0010] S1. Dataset preparation: Select source domain data and target domain data to obtain source domain images with landslide labels and target domain images without labels;

[0011] S2, construct and optimize the landslide cross-domain extraction model;

[0012] S3. Output the landslide prediction results based on the optimized landslide cross-domain extraction model.

[0013] Furthermore, in S2, the landslide cross-domain extraction model includes a teacher-student network model, an image mask domain adaptation module, and a morphological information extraction and optimization module.

[0014] Furthermore, in S2, the optimized landslide cross-domain extraction model is specifically:

[0015] S2.1. Train the teacher network; perform mask processing on the target domain image based on the trained teacher network to obtain the masked target domain image; extract the target domain image features through the teacher network, and perform multi-scale feature fusion to obtain the pseudo label of the target domain image;

[0016] S2.2, training the student network; predicting the masked target domain image based on the trained student network to obtain the prediction result of the student network; updating the parameters of the student network;

[0017] Input the target domain image, extract the landslide morphology information, and optimize the landslide morphology information to form a morphology pseudo label;

[0018] S2.3, input the pseudo label of the target domain image, the prediction result of the student network and the morphological pseudo label, and calculate the loss function;

[0019] S2.4. Optimize the student network and the teacher network, and output updated student network parameters and teacher network parameters, as well as the optimized landslide cross-domain extraction model.

[0020] Furthermore, in S2.2, the student network parameters are updated by calculating the consistency loss between the student network prediction results and the pseudo labels generated by the teacher network.

[0021] Furthermore, in S2.2, the extraction and optimization of landslide morphological information is specifically:

[0022] According to the prior information of pixel intensity of landslide in remote sensing image, the threshold segmentation method is used to segment and extract the landslide morphological information.

[0023] The segmentation results are optimized through morphological operations, and morphological rules are established to further filter non-landslide elements.

[0024] Furthermore, in S2.3, the loss function includes a cross-domain loss function, a mask consistency loss function and a morphological consistency loss function. The cross-domain loss is used to ensure that the model learns the correct semantic information of the landslide, the mask consistency loss is used to promote the student network to learn contextual information, and the morphological consistency loss is used to guide the student network to learn the morphological features of the landslide.

[0025] Furthermore, the cross-domain loss includes semantic segmentation loss And pseudo label loss

[0026] is the binary cross entropy loss. For each image, the calculation formula is as follows:

[0027]

[0028] Where N is the number of pixels, y i is the true label of the i-th pixel, p i is the predicted probability of the i-th pixel;

[0029] Pseudo-label loss Based on the weighted binary cross entropy loss calculation, the formula is as follows:

[0030]

[0031] Where N is the number of pixels, for the i-th pixel, is the generated pseudo label, w i is the pseudo-label confidence weight, p i is the predicted probability of the i-th pixel.

[0032] Furthermore, the mask consistency loss function is specifically:

[0033]

[0034] in, represents the data distribution of the target domain, f represents the student network model, represents the masked target domain image, is the pseudo label generated by the teacher network model, CE represents the cross entropy loss function, and w is the pseudo label confidence weight.

[0035] Furthermore, the morphological consistency loss function is specifically:

[0036]

[0037] in, represents the data distribution of the target domain, f(x t ) represents the prediction result of the student network model on the target domain image, is the pseudo label generated by the teacher network model, and CE represents the cross entropy loss function.

[0038] Furthermore, in S2.4, based on the teacher network in S2.1, the updated student network in S2.2 and the loss function in S2.3, the parameters of the student network and the teacher network are optimized by the back propagation algorithm to obtain the optimized landslide cross-domain extraction model.

[0039] The application of the technical solution of the present invention has the following beneficial effects:

[0040] The cross-domain extraction method of landslide remote sensing based on image mask and morphological enhancement provided by the present invention uses a random mask to process the target domain image through an image mask domain adaptation module, forces the student network to only access the local information of the image, and promotes the student network to learn the context information in the local image through the mask consistency loss of the pseudo label generated by the teacher network, thereby improving the model's ability to capture landslide features; the method makes full use of the context information of the target domain image to help the model better understand the spatial relationship between the landslide and the surrounding features, and guides the learning of the student network by constructing a teacher-student network model and using the teacher network to generate pseudo labels of the complete image of the target domain. Through parameter updating, the teacher network can obtain gradually enhanced context information learning ability, generate higher quality pseudo labels, and improve the accuracy of landslide identification; at the same time, through the morphological information extraction and optimization module, the morphological information of the landslide is extracted and further optimized to form a morphological pseudo label, and the morphological pseudo label guides the model to learn the morphological characteristics of the landslide, thereby improving the accuracy and completeness of landslide identification.

[0041] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0043] Figure 1 It is a flow chart of the landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement in the present invention;

[0044] Figure 2 is the cross-domain landslide identification result of the target domain 1 in the present invention;

[0045] Figure 3 It is the cross-domain landslide identification result of the target domain 2 in the present invention. DETAILED DESCRIPTION

[0046] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0047] Example:

[0048] See also Figure 1 This embodiment provides a landslide remote sensing cross-domain extraction (IMMDA) method based on image mask and morphological enhancement, comprising the following steps:

[0049] S1. Dataset preparation: Select source domain data and target domain data to obtain source domain images with landslide labels and target domain images without labels;

[0050] S2, construct and optimize the landslide cross-domain extraction model;

[0051] S3. Output the landslide prediction results based on the optimized landslide cross-domain extraction model.

[0052] In S2, the landslide cross-domain extraction model includes a teacher-student network model, an image mask domain adaptation module and a morphological information extraction and optimization module. The teacher-student network is used to make full use of contextual information, the image mask domain adaptation module is used to mask the image, and the morphological information extraction and optimization module is used to extract and optimize the landslide morphological information.

[0053] In S2, the optimized landslide cross-domain extraction model is specifically:

[0054] S2.1. Input the source domain image and its corresponding landslide label into the model and train the teacher network. Input the target domain image into the trained teacher network, perform mask processing on the target domain image, randomly mask part of the image area, and obtain the masked target domain image. Introduce ResNet101 as the basic network in the teacher network to extract image features of the target domain image, perform multi-scale feature fusion through ASSP (Atrous Spatial Pyramid Pooling) Head, and apply Sigmoid activation function to obtain the pseudo label of the target domain image.

[0055] S2.2. Input the masked target domain image and the pseudo-label of the target domain image generated by the teacher network into the model to train the student network; predict the masked target domain image based on the trained student network to obtain the prediction result of the student network; update the student network parameters by calculating the mask consistency loss between the prediction result of the student network and the pseudo-label generated by the teacher network, so as to promote the student network to learn the contextual information in the local image and improve the performance of the student network in the target domain.

[0056] The target domain image is input into the morphological information extraction and optimization module. According to the prior information of the pixel intensity of the landslide in the remote sensing image (such as the landslide area is usually brighter), the threshold segmentation method is used to segment and extract the landslide morphological information (threshold segmentation is for the grayscale image, the landslide area is the brighter part, and an empirical parameter range is established to extract the landslide pixels); the segmentation result is optimized through morphological operation (morphological operation is for the rough extraction result of the previous step, filling the discontinuous holes in the image and optimizing the edges), and morphological rules are established to further filter non-landslide elements (morphological rules include area and shape. The area is used to filter out smaller noise, and the shape is calculated by calculating the aspect ratio and the area perimeter ratio to remove narrow roads). Finally, the optimized landslide morphological information is output to form a morphological pseudo-label.

[0057] S2.3. Input the pseudo-label of the target domain image, the prediction result of the student network and the morphological pseudo-label, and calculate the loss function; the loss function includes the cross-domain loss function, the mask consistency loss function and the morphological consistency loss function. The cross-domain loss is used to ensure that the model learns the correct semantic information of the landslide, the mask consistency loss is used to promote the student network to learn the contextual information, and the morphological consistency loss is used to guide the student network to learn the morphological characteristics of the landslide.

[0058] Cross-domain loss includes semantic segmentation loss And pseudo label loss It is used to ensure the model effect of source domain image and label training, and ensure that the model can learn the correct landslide semantic information; for landslide identification tasks, Designed as a binary cross entropy loss, for each image, the calculation formula is as follows:

[0059]

[0060] Where N is the number of pixels, y i is the true label of the i-th pixel, p i is the predicted probability of the i-th pixel;

[0061] The pseudo-label loss is used to calculate the error between the pseudo-label generated by the teacher network model and the predicted result of the student network model, thereby improving the performance of the student network model in the target domain; the pseudo-label loss Based on the weighted binary cross entropy loss calculation, the formula is as follows:

[0062]

[0063] Where N is the number of pixels, for the i-th pixel, is the generated pseudo label, w i is the pseudo-label confidence weight, p i is the predicted probability of the i-th pixel.

[0064] The mask consistency loss masks part of the target domain image and forces the student network model to keep the prediction of the masked part consistent with the pseudo label, thereby enhancing the model's understanding of the contextual relationship of the target domain data and improving the effect of domain adaptation. The mask consistency loss function is:

[0065]

[0066] in, represents the data distribution of the target domain, f represents the student network model, represents the masked target domain image, is the pseudo label generated by the teacher network model, CE represents the cross entropy loss function, and w is the pseudo label confidence weight.

[0067] The morphological consistency loss guides the prediction results of the student network model through morphological information, so that the student network model can learn the morphological information of the landslide and improve the completeness of landslide identification. The specific morphological consistency loss function is:

[0068]

[0069] in, represents the data distribution of the target domain, f(x t ) represents the prediction result of the student network model on the target domain image, is the pseudo label generated by the teacher network model, and CE represents the cross entropy loss function.

[0070] S2.4. Based on the teacher network in S2.1, the updated student network in S2.2 and the loss function in S2.3, the parameters of the student network and the teacher network are optimized through the back propagation algorithm, and the updated student network parameters and the teacher network parameters, as well as the optimized landslide cross-domain extraction model are output.

[0071] In order to verify the effectiveness of the method proposed in the present invention, the following experiments were performed in this embodiment:

[0072] The DMLD public data was selected as the source domain, and two different cities, Jiuzhaigou and Chimanimani, in the GVLM data were selected as the target domain (Jiuzhaigou was the target domain 1, and Chimanimani was the target domain 2). The complete landslide image was cropped into a landslide dataset by overlapping sampling, and divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The image pixel size was 512x512, and the cropping step size was 256.

[0073] (1) Method comparison experiment

[0074] The IMMDA method proposed in this paper is compared with four domain-adaptive semantic segmentation methods, namely FADA

[129] , DACS

[100] , HRDA

[130] , and DAFormer

[131] . The method is tested on two different target domains. The effectiveness of the method is analyzed through quantitative accuracy results and qualitative recognition results. The specific experimental results are shown in Table 1 and Figure 2 and Figure 3 .

[0075] Table 1 Quantitative evaluation results of different domain adaptation methods in Jiuzhaigou and Chimanimani (%)

[0076]

[0077] As shown in Table 1, Source only means that only source domain data is used without using domain adaptation strategy; IMMDA represents the method proposed by the present invention. First, by analyzing the evaluation results in Jiuzhaigou and Chimanimani areas, it can be seen that IMMDA has achieved the best F1 score and IoU (intersection over union) compared with other methods, and has good comprehensive performance, indicating the adaptability of the IMMDA method to different regions and different tasks.

[0078] Figure 2 The qualitative recognition results of different domain adaptation methods in Jiuzhaigou area are shown. Among them, IMMDA has the most comprehensive recognition of landslides, and the extracted landslide morphology is closest to the real label, which shows the effectiveness of morphological information enhancement in cross-domain landslide extraction. Among the other methods, DACS can achieve results close to IMMDA, but it is not accurate enough. The recognition results of FADA, HRDA and DAFormer are poor, and there are many missed recognitions.

[0079] Figure 3The results of landslide recognition in the Chimanimani area using different domain adaptation methods are shown. IMMDA can obtain the most complete landslide boundary information and distinguish the details of landslides and non-landslides, which shows that contextual information is fully utilized, and indirectly proves the effectiveness of the image mask idea. For large landslide areas, IMMDA's recognition is particularly effective because the morphological information extraction and optimization module provides powerful landslide morphological information priors.

[0080] (2) Module effectiveness analysis experiment

[0081] In order to verify the effectiveness of the image mask domain adaptation module and the morphological information extraction and optimization module used in the method of the present invention, a quantitative analysis was performed through an ablation experiment. The experimental results are shown in Table 3. The experimental area is selected as Chimanimani and the comparison method is DACS. When only the image mask domain adaptation module is used, the IoU of the model is 46.29%, which is 8.2% higher than DACS; the F1-Score is 63.29%, which is 8.12% higher than DACS. When only the morphological information extraction and optimization module is used, the IoU is 47.33% and the F1-Score is 64.25%, which is also a significant improvement. When the image mask module and the morphological information module are used at the same time, the IoU and F1-Score of the model reach the highest, with an IoU of 47.82% and an F1-Score of 64.70%, indicating that the synergy between the image mask domain adaptation module and the morphological information extraction and optimization module can effectively further improve the accuracy.

[0082] Table 3 Module effectiveness ablation experiment

[0083]

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement, characterized in that: The steps include: S1. Dataset preparation: Select source domain data and target domain data to obtain source domain images with landslide labels and target domain images without labels; S2, construct and optimize the landslide cross-domain extraction model; S3. Output the landslide prediction results based on the optimized landslide cross-domain extraction model.

2. According to claim 1, a landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement is characterized in that: In S2, the landslide cross-domain extraction model includes a teacher-student network model, an image mask domain adaptation module, and a morphological information extraction and optimization module.

3. The landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement according to claim 2 is characterized in that: In S2, the optimized landslide cross-domain extraction model is specifically: S2.

1. Train the teacher network; perform mask processing on the target domain image based on the trained teacher network to obtain the masked target domain image; extract the target domain image features through the teacher network, and perform multi-scale feature fusion to obtain the pseudo label of the target domain image; S2.2, training the student network; predicting the masked target domain image based on the trained student network to obtain the prediction result of the student network; updating the parameters of the student network; Input the target domain image, extract the landslide morphology information, and optimize the landslide morphology information to form a morphology pseudo label; S2.3, input the pseudo label of the target domain image, the prediction result of the student network and the morphological pseudo label, and calculate the loss function; S2.

4. Optimize the student network and the teacher network, and output updated student network parameters and teacher network parameters, as well as the optimized landslide cross-domain extraction model.

4. The landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement according to claim 3 is characterized in that: In S2.2, the student network parameters are updated by calculating the consistency loss between the student network prediction results and the pseudo labels generated by the teacher network.

5. The landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement according to claim 3 is characterized in that: In S2.2, the extraction and optimization of landslide morphology information is as follows: According to the prior information of pixel intensity of landslide in remote sensing image, the threshold segmentation method is used to segment and extract the landslide morphological information. The segmentation results are optimized through morphological operations, and morphological rules are established to further filter non-landslide elements.

6. The landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement according to claim 3 is characterized in that: In S2.3, the loss functions include cross-domain loss function, mask consistency loss function and morphological consistency loss function. The cross-domain loss is used to ensure that the model learns the correct semantic information of the landslide, the mask consistency loss is used to promote the student network to learn contextual information, and the morphological consistency loss is used to guide the student network to learn the morphological characteristics of the landslide.

7. The landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement according to claim 6 is characterized in that: Cross-domain loss includes semantic segmentation loss And pseudo label loss is the binary cross entropy loss. For each image, the calculation formula is as follows: Where N is the number of pixels, y i is the true label of the i-th pixel, p i is the predicted probability of the i-th pixel; Pseudo-label loss Based on the weighted binary cross entropy loss calculation, the formula is as follows: Where N is the number of pixels, for the i-th pixel, is the generated pseudo label, w i is the pseudo-label confidence weight, p i is the predicted probability of the i-th pixel.

8. The landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement according to claim 6 is characterized in that: The mask consistency loss function is specifically: in, represents the data distribution of the target domain, f represents the student network model, represents the masked target domain image, is the pseudo label generated by the teacher network model, CE represents the cross entropy loss function, and w is the pseudo label confidence weight.

9. The landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement according to claim 6 is characterized in that: The morphological consistency loss function is specifically: in, represents the data distribution of the target domain, f(x t ) represents the prediction result of the student network model on the target domain image, is the pseudo label generated by the teacher network model, and CE represents the cross entropy loss function.

10. The landslide remote sensing cross-domain extraction method based on image mask and morphological enhancement according to claim 3 is characterized in that: In S2.4, based on the teacher network in S2.1, the updated student network in S2.2 and the loss function in S2.3, the parameters of the student network and the teacher network are optimized by the back propagation algorithm to obtain the optimized landslide cross-domain extraction model.

Images