A remote sensing image landslide mapping method and related device

Abstract

The present invention discloses a remote sensing image landslide mapping method and related devices. During training, the recognition model adopted by the present invention constructs an input perspective consistency loss function, a feature perspective consistency loss function, and a model perspective consistency loss function, and forcibly aligns the input, feature, and model perturbation landslide mapping results with the generated weak data enhancement perturbation pseudo-label, thereby achieving the unification of multi-perspective perturbations and broadening the perturbation space, thereby solving the problem of limited consistent learning perspectives in landslide mapping and effectively reducing the cost of landslide mapping, providing reliable technical support for meeting the requirements of fast and low-cost emergency rescue and geological disaster investigation tasks.

Description

Technical Field

[0001] The invention relates to a remote sensing image landslide mapping method and related devices, belonging to the technical field of remote sensing identification. Background Art

[0002] Landslides are natural disasters caused by the sliding and destruction of soil, rock, or artificial fill along a slope due to human, natural, or a combination of factors. After an extreme disaster occurs, rapid and accurate landslide mapping of the affected area, precisely delineating the location and extent of the landslide, is crucial for disaster relief, quantitative assessment of the damage, and support for post-disaster reconstruction.

[0003] With the advancement of Earth observation technology, landslide mapping based on remote sensing imagery has garnered widespread attention. In recent years, deep learning technology, with its powerful automatic feature extraction and hierarchical representation learning capabilities, has significantly advanced the development of pixel-by-pixel landslide mapping. Currently, combining deep learning with Earth observation remote sensing imagery for landslide mapping has become a hot topic in both academic research and engineering applications.

[0004] While deep learning techniques have flourished in landslide mapping, their success relies on massive amounts of landslide remote sensing imagery and corresponding landslide labels. While advances in satellite remote sensing technology have made it possible to acquire vast quantities of landslide remote sensing imagery, the cost of annotating these images with pixel-level landslide labels is extremely high. This is because the landslide labeling process typically requires significant investment in both human and material resources. Insufficient labeled samples significantly reduces the accuracy of deep learning models, making them inadequate for landslide mapping.

[0005] Developing a deep learning-based landslide mapping technique to achieve more accurate and complete landslide identification results in the face of limited labeled samples is undoubtedly a challenging task. The semi-supervised learning (SSL) paradigm can effectively improve the performance of deep learning models by fully leveraging large amounts of unlabeled data. Therefore, applying SLL to landslide mapping is an important approach to addressing the reduced landslide identification accuracy caused by insufficient labeled landslide samples. However, the application of existing SSL to landslide mapping is limited by its consistent learning perspective. Summary of the Invention

[0006] The present invention provides a remote sensing image landslide mapping method and related devices, which solve the problems disclosed in the background technology.

[0007] According to one aspect of the present disclosure, a method for mapping landslides using remote sensing images is provided, comprising:

[0008] Segment the remote sensing images of the disaster-stricken area to obtain remote sensing image blocks, and select some remote sensing image blocks to construct a training set;

[0009] The training set is used to train the student model, and the trained student model is used as the recognition model; when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, This is the result of weak data enhancement perturbation on the landslide images in the training samples without landslide markers. for Results after CutMix data augmentation perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and during the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model;

[0010] All remote sensing image blocks obtained by segmentation are input into the recognition model to obtain the landslide recognition results of each remote sensing image block. Based on the landslide recognition results, the landslide mapping results of the affected area are obtained.

[0011] In some embodiments of the present disclosure, according to ,by Construct an input view consistency loss function for the target, including:

[0012] Will Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation results and ,Will Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation and ;in, and For any pair ;

[0013] Will and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result ;

[0014] Will and Input the student model separately and obtain Landslide identification results and Landslide identification results ,Will and Input the student model separately and obtain Landslide identification results and Landslide identification results ;

[0015] Will and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result ;

[0016] according to ,by Construct the first input view consistency loss function for the target ,according to ,by Construct a second input view consistency loss function for the target .

[0017] In some embodiments of the present disclosure, the first input perspective consistency loss function And the second input perspective consistency loss function , the formula is:

[0018] ;

[0019] ;

[0020] Where, is the Dice loss function, M T_Uis the total number of training samples without landslide labels in the training set, and are the results of weak data enhancement perturbation on the landslide images in the i-th and j-th landslide-free training samples, respectively. and They are respectively after CutMix data enhancement perturbation and , and They are and Input the landslide identification results obtained by the student model, Confidence threshold for filtering weak data to enhance the noise of landslide identification results.

[0021] In some embodiments of the present disclosure, the first CutMix data augmentation perturbation and the second CutMix data augmentation perturbation use anchor point constrained CutMix and have different landslide masks; the formula of the landslide mask is:

[0022] ;

[0023] Where M A is the landslide mask, M is the randomly generated initial landslide mask, and The sizes are the same, Argmax is the Argmax operation, and ⊙ is the Hadamard product operation.

[0024] In some embodiments of the present disclosure, the feature view consistency loss function is formulated as follows:

[0025] ;

[0026] Where, is the feature view consistency loss function, is the Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, is the result of weak data enhancement perturbation on the landslide image in the i-th training sample without landslide markers. for Input the landslide identification results obtained by the student model, for The landslide identification result obtained by the student model is input, and during the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary.

[0027] In some embodiments of the present disclosure, feature perturbation is applied to the features of the hidden layer output at the stage where the overlap between the low-density region and the segmentation category boundary in the student model is the highest, including:

[0028] In the channel dimension, F in Summing and normalizing to get , and construct with Empty matrix F of consistent shape e Among them, F in It is the hidden layer output of the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary;

[0029] Will Compare pixel values ​​with the sampling threshold. For pixels greater than the sampling threshold, e The same position of F is filled with 0, and for pixels smaller than the sampling threshold, e Fill the same position with 1;

[0030] F in With the filled F e Perform Hadamard product operation to obtain the features after applying feature perturbation In some embodiments of the present disclosure, the stage where the low-density region has the highest overlap with the segmentation category boundary is determined before constructing the feature view consistency loss function, and the determination process includes:

[0031] Input any landslide image from the landslide-labeled training sample into the recognition model trained only by the supervised learning loss function to obtain high-dimensional features at the intermediate level of the recognition model;

[0032] Accumulate high-dimensional features in the channel dimension to reduce the high-dimensional features to two-dimensional features;

[0033] The average Euclidean distance between the center pixel of the two-dimensional feature and its surrounding adjacent pixels is calculated by sliding the window to obtain the initial local variation.

[0034] The initial local variation degree is sequentially normalized by maximum and minimum, and visualized as a single-channel grayscale image to obtain a local variation degree map;

[0035] By comparing the local change degree map with the landslide markers, the stage with the highest overlap between the low-density area and the segmentation category boundary is determined.

[0036] In some embodiments of the present disclosure, the model perspective consistency loss function is formulated as follows:

[0037] ;

[0038] Where, is the model perspective consistency loss function, is the binary cross entropy loss function or Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, is the result of weak data enhancement perturbation on the landslide image in the i-th training sample without landslide markers. for Input the landslide identification results obtained by the student model.

[0039] According to another aspect of the present disclosure, a remote sensing image landslide mapping device is provided, comprising:

[0040] The segmentation and construction module segments the remote sensing images of the disaster-stricken area to obtain remote sensing image blocks, and selects some remote sensing image blocks to construct a training set;

[0041] The training and verification module uses the training set to train the student model, and uses the trained student model as the recognition model; when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, This is the result of weak data enhancement perturbation on the landslide images in the training samples without landslide markers. for Results after CutMix data augmentation perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and during the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model;

[0042] The mapping module inputs all the remote sensing image blocks obtained by segmentation into the recognition model to obtain the landslide recognition results of each remote sensing image block, and obtains the landslide mapping results of the affected area based on the landslide recognition results.

[0043] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which stores one or more programs. The one or more programs include instructions. When the instructions are executed by a computing device, the computing device executes a remote sensing image landslide mapping method.

[0044] The beneficial effects achieved by the present invention are as follows: during training, the recognition model adopted by the present invention constructs an input perspective consistency loss function, a feature perspective consistency loss function and a model perspective consistency loss function, forcing the input, feature and model perturbation landslide mapping results to be aligned with the generated weak data enhancement perturbation pseudo-label, thereby achieving the unification of multi-perspective perturbations and broadening the perturbation space, thereby solving the problem of limited consistent learning perspective in landslide mapping. In addition, the present invention can effectively reduce the cost of landslide mapping and provide reliable technical support for meeting the tasks of fast and low-cost disaster relief and geological disaster investigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of the landslide mapping method based on remote sensing images;

[0046] Figure 2 This is a flowchart of the landslide mapping method using remote sensing images;

[0047] Figure 3 Schematic diagram of the training process of the recognition model;

[0048] Figure 4 Schematic diagram of the input perspective consistency learning process;

[0049] Figure 5 Generate schematic diagrams for landslide masks;

[0050] Figure 6 Schematic diagram of the calculation process of the high-dimensional feature local change degree map at the middle level of the model;

[0051] Figure 7 The original visible light remote sensing images and DEM maps of the Bijie landslide dataset cover the area;

[0052] Figure 8 This is a visualization of the local variation of high-dimensional features in the middle layer of some Bijie landslide dataset images calculated based on U-Net;

[0053] Figure 9 The Bijie landslide dataset is a comparison of the algorithm and the landslide mapping results of the present invention under different proportions of landslide-labeled training samples;

[0054] Figure 10 This is a block diagram of the remote sensing image landslide mapping device. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. It is obvious that the embodiments described are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0056] Unless otherwise specified, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0057] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0058] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0059] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0060] It should be noted that like symbols and letters refer to like items in the following figures, so once an item is defined in one figure, it does not need to be further discussed in subsequent figures.

[0061] In order to solve the problem of limited consistent learning perspective in landslide mapping, the present invention proposes a remote sensing image landslide mapping method and related devices. Specifically, when training the model, the input, features, and model perturbation landslide mapping results are forced to be aligned with the generated weak data enhancement perturbation pseudo-labels.

[0062] Figure 1 This is a schematic diagram of an embodiment of the method for mapping landslides using remote sensing images disclosed herein. Figure 1 The embodiment can be executed by a landslide mapping terminal (such as a computer), Figure 2 for Figure 1 Corresponding process diagram.

[0063] like Figure 1 As shown, in step 1 of the embodiment, the remote sensing image of the disaster-stricken area is segmented to obtain remote sensing image blocks, and some remote sensing image blocks are selected to construct a training set.

[0064] It should be noted that a fixed-size sliding window can be used to perform overlapping continuous segmentation of remote sensing images, and when the image at the edge is smaller than the sliding window, it can be supplemented by filling with zero pixels, so that a fixed-size remote sensing image block set of the disaster area can be obtained, which is more conducive to subsequent training and recognition; Among them, the size of the sliding window and the size of the remote sensing image block can be determined according to the actual situation, for example Figure 2 In the figure, the remote sensing image is segmented into remote sensing image blocks with 13 rows and 8 columns.

[0065] It should be noted that when constructing a training set, a validation set and a test set are also needed. The validation set is used to verify the trained model, and the test set is used to test the trained model. This is common sense in model training and will not be described in detail here.

[0066] Specifically, a certain proportion of remote sensing image blocks containing landslide objects can be selected from the remote sensing image block set to construct the training set D T , a certain proportion of remote sensing image blocks to build a validation set D V , the remaining remote sensing image blocks form the test set D I ; Among them, the proportions can generally be 60%, 20%, and 20%.

[0067] To D T Some remote sensing image blocks and D V All remote sensing image blocks in the D T contains training samples with landslide markers and training samples without landslide markers, which can be recorded as 、 , D V The landslide mark verification sample is included in Among them, M T_L is the total number of training samples with landslide labels in the training set, , is the landslide image in the i1th landslide-labeled training sample, is the landslide mark in the i1th training sample with landslide mark, 0 represents the background part, 1 represents the landslide part, M T_U is the total number of training samples without landslide labels in the training set, is the landslide image in the i-th training sample without landslide markers, M V_L is the total number of landslide-marked validation samples in the validation set, is the landslide image in the i2th landslide mark verification sample, Verify the landslide mark in the sample with the i2th landslide mark.

[0068] return Figure 1In step 2 of the embodiment, a student model is trained using a training set, and the trained student model is used as a recognition model; wherein, when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, This is the result of weak data enhancement perturbation on the landslide images in the training samples without landslide markers. for Results after CutMix data augmentation perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and during the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary; for Landslide recognition results obtained by adding Gaussian noise and inputting it into the teacher model.

[0069] It should be noted that before training the model, it is necessary to preset the weak data enhancement perturbation set D W , strong data augmentation perturbation set D S and CutMix data augmentation perturbation D C The specific data enhancement strategies contained in , as well as the probability of each strategy being executed, can be seen in Table 1; among them, CutMix data enhancement perturbation D C The specific CutMix method is anchor point constraint CutMix.

[0070] Table 1 Data augmentation perturbation strategy set

[0071]

[0072] A data enhancement strategy suitable for landslide remote sensing images is designed here to avoid the destruction of the original deep and shallow layer features and micro-topography information in the input image data.

[0073] See Figure 3 , the training process of the student model can be as follows:

[0074] 1) For the training sample S with landslide mark L Perform preset weak data enhancement perturbation to obtain samples after data enhancement perturbation ,Will Landslide images in Input student model , and obtain the landslide identification results ,according to and Landslide markers in Constructing supervised learning loss function .

[0075] It should be noted that S L Follow the weak data enhancement perturbation strategy described in Table 1 in order to obtain ,Will in enter ,right in Perform one-hot encoding and construct with binary cross entropy loss or Dice loss , the formula can be expressed as:

[0076] ;

[0077] Where, is the binary cross entropy loss function or Dice loss function, is the result of weak data enhancement perturbation on the landslide image in the i1th landslide-labeled training sample. for Input the landslide identification results obtained by the student model.

[0078] 2) For the training sample S without landslide mark U Perform preset weak data enhancement perturbation to obtain samples after data enhancement perturbation ,Will Landslide images in Input student model , and obtain the landslide identification results ,Will After strong data enhancement perturbation and CutMix data enhancement perturbation, the data enhancement perturbation results are obtained. ,Will Input student model , and obtain the landslide identification results ,according to , after perturbation with CutMix data enhancement Construct an input view consistency loss function for the target.

[0079] In some embodiments, see Figure 4 , the process of constructing the input perspective consistency loss function can be:

[0080] 21) S U Follow the weak data enhancement perturbation strategy described in Table 1 in order to obtain .

[0081] 22) in Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation results and ,Will middle Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation and ;in, and For any pair .

[0082] 23) and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result .

[0083] It should be noted that the landslide masks of the first CutMix data enhancement perturbation and the second CutMix data enhancement perturbation are different, such as defined as M A1 、M A2 .

[0084] See Figure 5 , can be randomly constructed with Masks M1 and M2 of the same size will be filled with 0 in the part to be replaced and 1 in the part not to be replaced. Input student model Obtain the landslide confidence map, then perform Argmax operation to obtain the landslide anchor point, and perform Hadamard product operation with the inverted mask 1-M1 and 1-M2 to obtain the landslide mask M in the anchor point constraint CutMix enhanced perturbation strategy. A1 、M A2 , which can be expressed as:

[0085] ;

[0086] ;

[0087] Where MA1 and M A2 is the landslide mask, Argmax is the Argmax operation, and ⊙ is the Hadamard product operation.

[0088] 24) and Input the student model separately and obtain Landslide identification results and Landslide identification results ,Will and Input the student model separately and obtain Landslide identification results and Landslide identification results .

[0089] 25) and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result .

[0090] It should be noted that the landslide mask of the first CutMix data enhancement is the above M A1 , the perturbation result needs to be further processed by Argmax operation and One-Hot encoding in order to obtain the hard pseudo label, that is, ; Similarly, the landslide mask of the second CutMix data enhancement is the above M A2 , the perturbation result needs to be further processed by Argmax operation and One-Hot encoding in sequence to obtain .

[0091] 26) According to ,by Construct the first input view consistency loss function for the target ,according to ,by Construct a second input view consistency loss function for the target .

[0092] It should be noted that Dice loss is used here to construct the input perspective consistency loss function and , which can be expressed as:

[0093] ;

[0094] ;

[0095] Where, is the Dice loss function, and are the results of weak data enhancement perturbation on the landslide images in the i-th and j-th landslide-free training samples, respectively. and They are respectively after CutMix data enhancement perturbation and , and They are and Input the landslide identification results obtained by the student model, Confidence threshold for filtering weak data to enhance the noise of landslide identification results.

[0096] It should be noted that input view consistency learning avoids the destruction of the original deep and shallow layer features and micro-topography information in the input image data by adopting a data enhancement strategy suitable for landslide remote sensing images, generates more accurate landslide unlabeled data, and introduces additional information through strong data enhancement perturbation, reducing overfitting of limited labeled landslide data.

[0097] 3) Input the student model and in the landslide identification process, The feature F output by the hidden layer at the stage where the medium and low density areas have the highest overlap with the segmentation category boundary in Apply F-Drop feature perturbation to obtain the features after feature perturbation ,Will Input student model In the subsequent levels, the landslide identification results are obtained ,according to ,by Construct a feature view consistency loss function for the target.

[0098] It should be noted that consistency learning can only learn from unlabeled training samples if the clustering assumption is met. This means that the decision boundary for the segmentation category should be located in a low-density region, thus avoiding classifying the same type of data into different categories. Therefore, perturbing features at the stage where the overlap between the low-density region and the segmentation category boundary is highest ensures that the decision function boundary for consistent learning from the feature perturbation perspective is more likely to reach the low-density region.

[0099] Therefore, in some embodiments, see Figure 6 ,The process of determining the stage where the low-density area has the highest overlap with the segmentation category boundary can include:

[0100] There will be landslide labeled training samples S LAny landslide image input in is only determined by the supervised learning loss function The trained recognition model obtains high-dimensional features of the intermediate layers of the recognition model (because in the neural network training process, the input is usually a visible light image, which has only three dimensions of RGB in the channel dimension, but as the training progresses, each network layer gradually extracts and converts the original input and represents it through complex features. These features usually have multiple dimensions, each dimension represents a different feature or attribute. The features of other network layers except the input and output features of the model can be called high-dimensional features). The high-dimensional features are accumulated in the channel dimension to reduce the high-dimensional features to two-dimensional features. The average Euclidean distance between the center pixel of the two-dimensional feature and its eight surrounding adjacent pixels is calculated through a 3×3 sliding window to obtain the initial local variation degree. The initial local variation degree is sequentially normalized by maximum and minimum and visualized as a single-channel grayscale image to obtain a local variation degree map. The local variation degree map is compared with the landslide mark to determine the stage with the highest overlap between the low-density area and the segmentation category boundary.

[0101] The optimal stage of feature perturbation assignment is determined by observing the local variation of high-dimensional features at the intermediate levels of the recognition model and the overlap of low-density regions of segmented categories.

[0102] It should be noted that feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area has the highest overlap with the segmentation category boundary. The specific process can be:

[0103] In the channel dimension, F in Summing and normalizing to get , and construct with Empty matrix F of consistent shape e ,Will Compare pixel values ​​with the sampling threshold. For pixels greater than the sampling threshold, e The same position of F is filled with 0, and for pixels smaller than the sampling threshold, e Fill the same position of F with 1, and in With the filled F e Perform Hadamard product operation to obtain the features after applying feature perturbation .

[0104] It should be noted that the Dice loss is used here to construct the feature perspective consistency loss function, which can be expressed as follows:

[0105] ;

[0106] Where, is the feature view consistency loss function.

[0107] It should be noted that feature perspective consistency learning enables the model to more fully explore and learn the rich high- and low-dimensional information encoded in the feature map, further improving the accuracy of landslide detection.

[0108] 4) Add Gaussian noise and input to the teacher model , and obtain the landslide identification results ,according to ,by Construct a model view consistency loss function for the target.

[0109] It should be noted that the teacher model and student models The structure is consistent, but the parameter initialization methods are inconsistent. For example, the student model can be manually initialized using a layer-by-layer traversal method, and the teacher model can be initialized using the Kaiming normal distribution. Determine the mean and standard deviation of the Gaussian noise perturbation. Parameters are updated through consistency learning loss, Parameters are passed to The parameters are updated using an exponential sliding average.

[0110] It should be noted that the binary cross entropy loss or Dice loss is used here to construct the model perspective consistency loss function, which can be expressed as:

[0111] ;

[0112] Where, is the model perspective consistency loss function.

[0113] It should be noted that model perspective consistency learning can break the limitations of a single learning perspective and form a virtuous circle through mutual learning between student models, jointly promote the learning of feature distribution, and reduce the prediction oscillation of unlabeled landslide remote sensing images.

[0114] 5) The different perspective consistency learning is implemented in a single learning stream. The supervised learning loss function, input perspective consistency loss function, feature perspective consistency loss function and model perspective consistency loss function are weightedly summed to construct a multi-perspective consistency learning joint loss function, constrain the student model parameter update, and obtain the recognition model.

[0115] It should be noted that the joint loss function of multi-view consistency learning can be expressed as:

[0116] ;

[0117] Where, They represent the supervised learning loss function, the first input perspective consistency loss function, the second input perspective consistency loss function, the feature perspective consistency loss function, and the model perspective consistency loss function corresponding to the landslide image in the i-th landslide-free training sample, respectively. Learning a joint loss function for multi-view consistency, is the weight coefficient, .

[0118] In the above training process, multi-view consistency learning is integrated into a unified objective function, which broadens the perturbation space and achieves the learning of more discriminative features of unlabeled landslide remote sensing images.

[0119] return Figure 1 In step 3 of the embodiment, all the remote sensing image blocks obtained by segmentation are input into the recognition model to obtain the landslide recognition results of each remote sensing image block, and the landslide mapping results of the affected area are obtained based on the landslide recognition results.

[0120] It should be noted that the entire remote sensing image needs to be input for recognition, that is, all remote sensing image blocks are input into the recognition model. If the remote sensing image is a continuous landslide image, the landslide recognition results are sequentially spliced ​​to obtain the landslide mapping result. If it is a discontinuous landslide image, no splicing is required, and the recognition result is the landslide mapping result.

[0121] The above method performs overlapping sliding segmentation on the remote sensing images of the disaster-stricken area to construct a landslide dataset, and selects a certain proportion of landslide image blocks from the training set for manual labeling as labeled data, and the remaining image blocks are used as unlabeled data. Then, by comprehensively applying consistency regularization and self-training methods, the multi-view perturbation is unified by forcibly aligning the input, features, and model perturbation landslide mapping results with the generated weak data enhancement perturbation pseudo-labels, thereby broadening the perturbation space and enabling the model to learn more discriminative features of unlabeled landslide remote sensing images, thereby obtaining a multi-view consistency landslide recognition model. The trained model is used to identify all segmented images, and then the landslide mapping results of the disaster-stricken area are obtained based on the recognition results.

[0122] The above method forcibly aligns the input, features, and model-perturbed landslide mapping results with the generated weak data-enhanced perturbation pseudo-labels during model training, thereby achieving the unification of multi-perspective perturbations and broadening the perturbation space, thereby solving the problem of limited consistent learning perspectives in landslide mapping. In addition, the above method can effectively reduce the cost of landslide mapping and provide reliable technical support for meeting the tasks of fast and low-cost disaster relief and geological disaster investigation.

[0123] The following uses the open-source Bijie landslide dataset as an example, with visible light remote sensing images of the landslide and the corresponding digital elevation model (DEM) as the data source to verify the above method in detail.

[0124] In the above verification process, some parameter values ​​in the method are as follows: M T_L =10, 30, 104, 154 were tested, and the corresponding M T_U =504, 484, 410, 360, and M in the validation set V_L =44, the set to be inspected M I_U =212, H=W=256 represent the length and width of the landslide image respectively, C=4 represents that the landslide image has four bands: red (R), green (G), blue (B), and digital elevation model (DEM). The mask M is 256×256. =0.3, the sampling threshold range is (0.1, 0.3), the mean of the Gaussian noise perturbation is 0, and the standard deviation is 1, and the final value is limited to the range of [-0.1, 0.1], All are 0.25.

[0125] During the above verification process, the original visible light remote sensing images and DEM of the Bijie landslide dataset were used to Figure 7 The visualization results of local variation of high-dimensional features in the middle layer of some Bijie landslide dataset images calculated based on U-Net are shown in Figure 8 ,The stage where the highest overlap between the low-density area and the ,segmentation category boundary is determined by comparing it with the manually ,determined landslide markers is after the fourth upsampling in the decoding ,stage.

[0126] In the above verification process, the set to be tested is directly input into the recognition model to obtain Figure 9 The Bijie landslide dataset was used to compare the algorithm and the landslide mapping results of the present invention under different proportions of landslide-labeled training samples. Because the Bijie landslide dataset provides non-continuous landslide image blocks after segmentation, there is no need to perform a sequential splicing operation on the landslide identification results. Instead, the landslide identification results of the test set are directly used as the landslide mapping results. Through verification, it is proved that the above method achieves the desired effect, that is, it solves the problem of limited consistent learning perspective in landslide mapping, and can effectively reduce the cost of landslide mapping, providing reliable technical support for meeting the requirements of rapid and low-cost disaster relief and geological disaster investigation tasks.

[0127] Figure 10 This is a schematic diagram of an embodiment of the remote sensing image landslide mapping device disclosed in the present invention. Figure 10The embodiment is a virtual device that can be loaded and executed by a landslide mapping terminal (such as a computer), including a segmentation and construction module, a training and verification module, and a mapping module.

[0128] The segmentation and construction module of the embodiment is configured to segment the remote sensing image of the disaster-stricken area to obtain remote sensing image blocks, and select some of the remote sensing image blocks to construct a training set.

[0129] The training verification module of the embodiment is configured to train the student model using the training set and use the trained student model as the recognition model; wherein, when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, This is the result of weak data enhancement perturbation on the landslide images in the training samples without landslide markers. for Results after CutMix data augmentation perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and during the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary; for Landslide recognition results obtained by adding Gaussian noise and inputting it into the teacher model.

[0130] The mapping module of the embodiment is configured to input all the remote sensing image blocks obtained by segmentation into a recognition model to obtain landslide recognition results of each remote sensing image block, and obtain landslide mapping results of the affected area based on the landslide recognition results.

[0131] During training, the recognition model used by the above-mentioned device constructs an input perspective consistency loss function, a feature perspective consistency loss function, and a model perspective consistency loss function, forcing the input, feature, and model perturbation landslide mapping results to be aligned with the generated weak data enhancement perturbation pseudo-labels, thereby achieving the unification of multi-perspective perturbations and broadening the perturbation space, thereby solving the problem of limited consistent learning perspective in landslide mapping.

[0132] Based on the same technical solution, the present disclosure also relates to a computer-readable storage medium, which stores one or more programs. The one or more programs include instructions. When the instructions are executed by a computing device, the computing device executes a remote sensing image landslide mapping method.

[0133] Based on the same technical solution, the present disclosure also relates to a computer device comprising one or more processors and one or more memories, wherein one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for executing a remote sensing image landslide mapping method.

[0134] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0138] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A method for mapping landslides using remote sensing images, characterized in that: include: Segment the remote sensing images of the disaster-stricken area to obtain remote sensing image blocks, and select some remote sensing image blocks to construct a training set; The training set is used to train the student model, and the trained student model is used as the recognition model; when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, This is the result of weak data enhancement perturbation on the landslide images in the training samples without landslide markers. for Results after CutMix data augmentation perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and during the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model; All the remote sensing image blocks obtained by segmentation are input into the recognition model to obtain the landslide recognition results of each remote sensing image block, and the landslide mapping results of the affected area are obtained based on the landslide recognition results; The above-mentioned feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary, including: In the channel dimension, F in Summing and normalizing to get , and construct with Empty matrix F of consistent shape e Among them, F in It is the hidden layer output of the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary; Will Compare pixel values ​​with the sampling threshold. For pixels greater than the sampling threshold, e The same position of F is filled with 0, and for pixels smaller than the sampling threshold, e Fill the same position with 1; F in With the filled F e Perform Hadamard product operation to obtain the features after applying feature perturbation .

2. The remote sensing image landslide mapping method according to claim 1, characterized in that: according to ,by Construct an input view consistency loss function for the target, include: Will Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation results and ,Will Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation and ;in, and For any pair ; Will and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result ; Will and Input the student model separately and obtain Landslide identification results and Landslide identification results ,Will and Input the student model separately and obtain Landslide identification results and Landslide identification results ; Will and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result ; according to ,by Construct the first input view consistency loss function for the target ,according to ,by Construct a second input view consistency loss function for the target .

3. The remote sensing image landslide mapping method according to claim 2, characterized in that: First input perspective consistency loss function And the second input perspective consistency loss function , the formula is: ； ； Where, is the Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, and are the results of weak data enhancement perturbation on the landslide images in the i-th and j-th landslide-free training samples, respectively. and They are respectively after CutMix data enhancement perturbation and , and They are and Input the landslide identification results obtained by the student model, Confidence threshold for filtering weak data to enhance the noise of landslide identification results.

4. The method for mapping landslides using remote sensing images according to claim 2, wherein: The first CutMix data enhancement perturbation and the second CutMix data enhancement perturbation use anchor point constrained CutMix and have different landslide masks; the formula of the landslide mask is: ； Where M A is the landslide mask, M is the randomly generated initial landslide mask, and The sizes are the same, Argmax is the Argmax operation, and ⊙ is the Hadamard product operation.

5. The method for mapping landslides using remote sensing images according to claim 1, wherein: The feature perspective consistency loss function is as follows: ； Where, is the feature view consistency loss function, is the Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, is the result of weak data enhancement perturbation on the landslide image in the i-th training sample without landslide markers. for Input the landslide identification results obtained by the student model, for The landslide identification result obtained by the student model is input, and during the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary.

6. The method for mapping landslides using remote sensing images according to claim 1, wherein: The stage where the low-density region has the highest overlap with the segmentation category boundary is determined before constructing the feature view consistency loss function. The determination process includes: Input any landslide image from the training sample with landslide markers into the recognition model trained only by the supervised learning loss function to obtain the high-dimensional features of the intermediate level of the recognition model; Accumulate high-dimensional features in the channel dimension to reduce the high-dimensional features to two-dimensional features; The average Euclidean distance between the center pixel of the two-dimensional feature and its surrounding adjacent pixels is calculated by sliding the window to obtain the initial local variation. The initial local variation degree is sequentially normalized by maximum and minimum, and visualized as a single-channel grayscale image to obtain a local variation degree map; By comparing the local change degree map with the landslide markers, the stage with the highest overlap between the low-density area and the segmentation category boundary is determined.

7. The method for mapping landslides using remote sensing images according to claim 1, wherein: Model perspective consistency loss function, the formula is: ； Where, is the model perspective consistency loss function, is the binary cross entropy loss function or Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, is the result of weak data enhancement perturbation on the landslide image in the i-th training sample without landslide markers. for Input the landslide identification results obtained by the student model.

8. A remote sensing image landslide mapping device, characterized in that: include: The segmentation and construction module segments the remote sensing images of the disaster-stricken area to obtain remote sensing image blocks, and selects some remote sensing image blocks to construct a training set; The training and verification module uses the training set to train the student model, and uses the trained student model as the recognition model; when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, This is the result of weak data enhancement perturbation on the landslide images in the training samples without landslide markers. for Results after CutMix data augmentation perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and during the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model; The above-mentioned feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary, including: In the channel dimension, F in Summing and normalizing to get , and construct with Empty matrix F of consistent shape e Among them, F in It is the hidden layer output of the stage where the low-density area in the student model has the highest overlap with the segmentation category boundary; Will Compare pixel values ​​with the sampling threshold. For pixels greater than the sampling threshold, e The same position of F is filled with 0, and for pixels smaller than the sampling threshold, e Fill the same position with 1; F in With the filled F e Perform Hadamard product operation to obtain the features after applying feature perturbation ; The mapping module inputs all the remote sensing image blocks obtained by segmentation into the recognition model to obtain the landslide recognition results of each remote sensing image block, and obtains the landslide mapping results of the affected area based on the landslide recognition results.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs include instructions. When the instructions are executed by a computing device, the computing device executes the method according to any one of claims 1 to 7.

Images