Remote sensing image landslide mapping method and related device

Abstract

The invention discloses a remote sensing image landslide mapping method and a related device, and the method comprises the steps: constructing an input visual angle consistency loss function, a feature visual angle consistency loss function and a model visual angle consistency loss function when an adopted recognition model is trained; according to the method, input, feature and model disturbance landslide mapping results are aligned with generated weak data enhanced disturbance pseudo labels forcibly, unification of multi-view disturbance is achieved, disturbance space is widened, the problem that consistent learning view angles are limited in landslide mapping is solved, the cost of landslide mapping is effectively reduced, and the landslide mapping efficiency is improved. And reliable technical support is provided for meeting rapid and low-cost rescue and relief and geological disaster survey tasks.

Description

Technical Field

[0001] The invention relates to a remote sensing image landslide mapping method and related devices, belonging to the technical field of remote sensing identification. Background Art

[0002] Landslide is a natural disaster phenomenon caused by the sliding and destruction of soil, rock or artificial fill along the slope due to human, natural or combined factors. After an extreme disaster event occurs, it is crucial to quickly and accurately map the landslides in the affected area and accurately identify the location and scope of the landslides, which is very important for disaster relief, quantitative assessment of the disaster situation and support for post-disaster reconstruction.

[0003] With the development of earth observation technology, landslide mapping based on remote sensing images has received widespread attention. In recent years, deep learning technology has greatly promoted the development of pixel-by-pixel classification landslide mapping technology with its powerful automatic feature extraction and hierarchical representation learning capabilities. At present, combining deep learning technology with earth observation remote sensing images for landslide mapping has gradually become a hot area in academic research and engineering applications.

[0004] Although deep learning technology has been vigorously developed in landslide mapping tasks, the premise for the success of such methods is a large number of landslide remote sensing images and corresponding landslide labels. With the development of satellite remote sensing technology, it is possible to obtain a large number of landslide remote sensing images, but the corresponding pixel-level landslide label annotation is extremely costly. Because the landslide annotation process is usually accompanied by a large amount of human and material resources. When there are insufficient label samples, the accuracy of the deep learning model will be significantly reduced, and it will not be able to meet the needs of landslide mapping tasks.

[0005] In the face of limited labeled samples, developing a landslide mapping technology based on deep learning to obtain more accurate and complete landslide identification results is undoubtedly a challenging task. The semi-supervised learning (SSL) paradigm can effectively improve the performance of deep learning models by making full use of a large amount of unlabeled data. Therefore, applying SLL to landslide mapping is one of the important ways to solve the problem of reduced landslide identification accuracy caused by insufficient labeled landslide samples, but the application of existing SSL to landslide mapping is still limited by the consistent learning perspective. Summary of the invention

[0006] The invention provides a remote sensing image landslide mapping method and related devices, which solve the problems disclosed in the background technology.

[0007] According to one aspect of the present disclosure, a remote sensing image landslide mapping method is provided, comprising:

[0008] Segment the remote sensing images of the disaster-stricken area to obtain remote sensing image blocks, and select some remote sensing image blocks to construct a training set;

[0009] The training set is used to train the student model, and the trained student model is used as the recognition model; when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, It is the result after weak data enhancement disturbance on the landslide images in the training samples without landslide markers; for The result after CutMix data enhancement perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and in the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the overlap between the low-density area and the separation category boundary in the student model is the highest; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model;

[0010] All remote sensing image blocks obtained by segmentation are input into the recognition model to obtain the landslide recognition results of each remote sensing image block. According to the landslide recognition results, the landslide mapping results of the affected area are obtained.

[0011] In some embodiments of the present disclosure, according to ,by Construct an input perspective consistency loss function for the target, including:

[0012] Will Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation results and ,Will Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation and ;in, and For any pair ;

[0013] Will and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result ;

[0014] Will and Input the student model respectively and obtain Landslide identification results and Landslide identification results ,Will and Input the student model respectively and obtain Landslide identification results and Landslide identification results ;

[0015] Will and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result ;

[0016] according to ,by Construct the first input view consistency loss function for the target ,according to ,by Construct a second input view consistency loss function for the target .

[0017] In some embodiments of the present disclosure, the first input perspective consistency loss function And the second input perspective consistency loss function , the formula is:

[0018] ;

[0019] ;

[0020] In the formula, is the Dice loss function, M T_Uis the total number of training samples without landslide labels in the training set, and are the results of weak data enhancement perturbation on the landslide images in the i-th and j-th training samples without landslide markers, and They are respectively after CutMix data enhancement perturbation and , and They are and Input the landslide identification results obtained by the student model, Confidence threshold for filtering weak data enhancement and noise in landslide identification results.

[0021] In some embodiments of the present disclosure, the first CutMix data enhancement perturbation and the second CutMix data enhancement perturbation use anchor point constraint CutMix, and have different landslide masks; the formula of the landslide mask is:

[0022] ;

[0023] Where M A is the landslide mask, M is the randomly generated initial landslide mask, and The sizes are the same, Argmax is the Argmax operation, and ⊙ is the Hadamard product operation.

[0024] In some embodiments of the present disclosure, the feature view consistency loss function is as follows:

[0025] ;

[0026] In the formula, is the feature view consistency loss function, is the Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, is the result of weak data enhancement perturbation on the landslide image in the i-th training sample without landslide markers. for Input the landslide identification results obtained by the student model, for The landslide identification result obtained by the student model is input, and in the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the boundary of the separation category.

[0027] In some embodiments of the present disclosure, feature perturbation is applied to the features of the hidden layer output at the stage where the overlap between the low-density region and the separation category boundary in the student model is the highest, including:

[0028] In the channel dimension, F in Summing and normalizing to get , and construct with Empty matrix F of consistent shape e Among them, F in It is the hidden layer output of the stage where the low-density area in the student model has the highest overlap with the boundary of the separation category;

[0029] Will Compare the pixel value with the sampling threshold. For pixels greater than the sampling threshold, e The same position of F is filled with 0. For pixels smaller than the sampling threshold, e Fill the same position with 1;

[0030] F in With the filled F e Perform Hadamard product operation to obtain the characteristics after applying characteristic perturbation In some embodiments of the present disclosure, the stage where the low-density region has the highest overlap with the boundary of the separation category is determined before constructing the feature view consistency loss function, and the determination process includes:

[0031] Input any landslide image in the training sample with landslide mark into the recognition model trained only by the supervised learning loss function to obtain the high-dimensional features of the intermediate level of the recognition model;

[0032] Accumulate high-dimensional features in the channel dimension to reduce the high-dimensional features to two-dimensional features;

[0033] The average Euclidean distance between the central pixel of the two-dimensional feature and its surrounding adjacent pixels is calculated by sliding the window to obtain the initial local variation degree;

[0034] The initial local variation degree is normalized by maximum and minimum in sequence and visualized as a single-channel grayscale image to obtain a local variation degree map;

[0035] By comparing the local change degree map with the landslide markers, the stage with the highest overlap between the low-density area and the boundary of the separation category is determined.

[0036] In some embodiments of the present disclosure, the model perspective consistency loss function is formulated as follows:

[0037] ;

[0038] In the formula, is the model perspective consistency loss function, is the binary cross entropy loss function or Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, is the result of weak data enhancement perturbation on the landslide image in the i-th training sample without landslide markers. for Input the landslide identification results obtained by the student model.

[0039] According to another aspect of the present disclosure, a remote sensing image landslide mapping device is provided, comprising:

[0040] The segmentation and construction module segments the remote sensing images of the disaster-stricken area to obtain remote sensing image blocks, and selects some remote sensing image blocks to construct a training set;

[0041] The training and verification module uses the training set to train the student model, and uses the trained student model as the recognition model; when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, It is the result after weak data enhancement disturbance on the landslide images in the training samples without landslide markers; for The result after CutMix data enhancement perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and in the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the overlap between the low-density area and the separation category boundary in the student model is the highest; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model;

[0042] The mapping module inputs all the remote sensing image blocks obtained by segmentation into the recognition model to obtain the landslide recognition results of each remote sensing image block, and obtains the landslide mapping results of the affected area based on the landslide recognition results.

[0043] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which stores one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes a remote sensing image landslide mapping method.

[0044] The beneficial effects achieved by the present invention are as follows: when training, the recognition model adopted by the present invention constructs an input perspective consistency loss function, a feature perspective consistency loss function and a model perspective consistency loss function, and forces the input, feature and model perturbation landslide mapping results to be aligned with the generated weak data enhancement perturbation pseudo-labels, thereby achieving the unification of multi-perspective perturbations and broadening the perturbation space, thereby solving the problem of limited consistent learning perspective in landslide mapping, and the present invention can effectively reduce the cost of landslide mapping, and provides reliable technical support for meeting the tasks of rapid and low-cost disaster relief and geological disaster investigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of the landslide mapping method based on remote sensing images;

[0046] Figure 2 The flowchart of the landslide mapping method using remote sensing images is shown in Figure 1.

[0047] Figure 3 Schematic diagram of the training process of the recognition model;

[0048] Figure 4 Schematic diagram of the input perspective consistency learning process;

[0049] Figure 5 Generate schematic diagrams for landslide masks;

[0050] Figure 6 It is a schematic diagram of the calculation process of the high-dimensional feature local change degree map at the middle level of the model;

[0051] Figure 7 The original visible light remote sensing images and DEM maps of the Bijie landslide dataset cover the area;

[0052] Figure 8 This is a visualization result diagram of the local variation of high-dimensional features at the middle level of some Bijie landslide dataset images calculated based on U-Net;

[0053] Fig. 9 The Bijie landslide data set is a comparison of the algorithm and the landslide mapping result of the present invention under different proportions of landslide marked training samples;

[0054] Fig.10 This is a block diagram of the remote sensing image landslide mapping device. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. It is obvious that the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0056] Unless otherwise specified, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0057] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0058] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0059] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0060] It should be noted that similar symbols and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it does not require further discussion in subsequent figures.

[0061] In order to solve the problem of limited consistent learning perspective in landslide mapping, the present invention proposes a remote sensing image landslide mapping method and related devices, specifically, when training the model, the input, features, model perturbation landslide mapping results are forced to be aligned with the generated weak data enhancement perturbation pseudo-labels.

[0062] Figure 1 Schematic diagram of an embodiment of the method for mapping landslides using remote sensing images disclosed in the present invention. Figure 1 The embodiment of the invention can be executed by a landslide mapping terminal (such as a computer), Figure 2 for Figure 1 The corresponding process diagram.

[0063] like Figure 1 As shown, in step 1 of the embodiment, the remote sensing image of the disaster-stricken area is segmented to obtain remote sensing image blocks, and some remote sensing image blocks are selected to construct a training set.

[0064] It should be noted that a fixed-size sliding window can be used to perform overlapping continuous segmentation of remote sensing images, and when the image at the edge is smaller than the sliding window, it can be supplemented by filling zero pixels, so that a fixed-size remote sensing image block set of the disaster area can be obtained, which is more conducive to subsequent training and recognition; the size of the sliding window and the size of the remote sensing image block can be determined according to the actual situation, for example Figure 2 In the figure, the remote sensing image is segmented into remote sensing image blocks with 13 rows and 8 columns.

[0065] It should be noted that when constructing a training set, you also need to construct a validation set and a test set. The validation set is used to verify the trained model, and the test set is used to test the trained model. This is common sense in model training and will not be described in detail here.

[0066] Specifically, a certain proportion of remote sensing image blocks containing landslide objects can be selected from the remote sensing image block set to construct the training set D T , a certain proportion of remote sensing image blocks to build a validation set D V The remaining remote sensing image blocks form the test set D I ; Among them, the proportions can generally be 60%, 20%, and 20%.

[0067] Right D T Some remote sensing image blocks and D V All remote sensing image blocks in D T contains training samples with landslide marks and training samples without landslide marks, which can be recorded as , , D V The landslide mark verification sample is included in ; Among them, M T_L is the total number of training samples with landslide labels in the training set, , is the landslide image in the i1th landslide labeled training sample, is the landslide mark in the i1th training sample with landslide marks, 0 represents the background part, 1 represents the landslide part, M T_U is the total number of training samples without landslide labels in the training set, is the landslide image in the i-th training sample without landslide markers, M V_L is the total number of landslide-marked validation samples in the validation set, is the landslide image in the i2th landslide mark verification sample, Verify the landslide mark in the i2th sample with a landslide mark.

[0068] return Figure 1In step 2 of the embodiment, a student model is trained using a training set, and the trained student model is used as a recognition model; wherein, when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, It is the result after weak data enhancement disturbance on the landslide images in the training samples without landslide markers; for The result after CutMix data enhancement perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and in the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the overlap between the low-density area and the separation category boundary in the student model is the highest; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model.

[0069] It should be noted that before training the model, it is necessary to preset the weak data enhancement perturbation set D W , Strong data augmentation perturbation set D S and CutMix data augmentation perturbation D C The specific data enhancement strategies contained in and the probability of each strategy being executed can be found in Table 1; among them, CutMix data enhancement perturbation D C The specific CutMix method is anchor point constraint CutMix.

[0070] Table 1 Data augmentation perturbation strategy set

[0071]

[0072] A data enhancement strategy suitable for landslide remote sensing images is designed here to avoid the destruction of the original deep and shallow layer features and micro-topography information in the input image data.

[0073] See Figure 3 , the training process of the student model can be as follows:

[0074] 1) For the training sample S with landslide mark L Perform preset weak data enhancement perturbation to obtain samples after data enhancement perturbation ,Will Landslide images in Input Student Model , and obtain the landslide identification results ,according to and Landslide markers in Constructing supervised learning loss function .

[0075] It should be noted that S L The weak data enhancement perturbation strategy described in Table 1 is performed in sequence to obtain ,Will In enter ,right In One-Hot encoding, constructed with binary cross entropy loss or Dice loss , the formula can be expressed as:

[0076] ;

[0077] In the formula, is the binary cross entropy loss function or Dice loss function, is the result of weak data enhancement perturbation on the landslide image in the i1th landslide-marked training sample. for Input the landslide identification results obtained by the student model.

[0078] 2) For the training sample S without landslide mark U Perform preset weak data enhancement perturbation to obtain samples after data enhancement perturbation ,Will Landslide images in Input Student Model , and obtain the landslide identification results ,Will After strong data enhancement perturbation and CutMix data enhancement perturbation, the data enhancement perturbation results are obtained. ,Will Input Student Model , and obtain the landslide identification results ,according to , after perturbation with CutMix data enhancement Construct an input view consistency loss function for the target.

[0079] In some embodiments, see Figure 4 , the process of constructing the input perspective consistency loss function can be:

[0080] 21) S U The weak data enhancement perturbation strategy described in Table 1 is performed in sequence to obtain .

[0081] 22) In Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation results and ,Will middle Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation and ;in, and For any pair .

[0082] 23) and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result .

[0083] It should be noted that the landslide masks of the first CutMix data enhancement perturbation and the second CutMix data enhancement perturbation are different, such as defined as M A1 、M A2 .

[0084] See Figure 5 , which can be randomly constructed with For masks M1 and M2 of the same size, fill the parts to be replaced with 0 and the parts not to be replaced with 1. Input Student Model The landslide confidence map is obtained, and then the Argmax operation is performed to obtain the landslide anchor point, and the Hadamard product operation is performed with the inverted masks 1-M1 and 1-M2 to obtain the landslide mask M in the anchor point constraint CutMix enhanced perturbation strategy. A1 、M A2 , which can be expressed as:

[0085] ;

[0086] ;

[0087] Where MA1 and M A2 is the landslide mask, Argmax is the Argmax operation, and ⊙ is the Hadamard product operation.

[0088] 24) and Input the student model respectively and obtain Landslide identification results and Landslide identification results ,Will and Input the student model respectively and obtain Landslide identification results and Landslide identification results .

[0089] 25) and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result .

[0090] It should be noted that the landslide mask of the first CutMix data enhancement is the above M A1 , the perturbation result needs to be further processed by Argmax operation and One-Hot encoding in order to obtain the hard pseudo label, that is, ; Similarly, the landslide mask of the second CutMix data enhancement is the above M A2 , the perturbation result needs to be further processed by Argmax operation and One-Hot encoding in order to obtain .

[0091] 26) According to ,by Construct the first input view consistency loss function for the target ,according to ,by Construct a second input view consistency loss function for the target .

[0092] It should be noted that the Dice loss is used here to construct the input perspective consistency loss function and , which can be expressed as:

[0093] ;

[0094] ;

[0095] In the formula, is the Dice loss function, and are the results of weak data enhancement perturbation on the landslide images in the i-th and j-th training samples without landslide markers, and They are respectively after CutMix data enhancement perturbation and , and They are and Input the landslide identification results obtained by the student model, Confidence threshold for filtering weak data enhancement and noise in landslide identification results.

[0096] It should be noted that input view consistency learning avoids the destruction of the original deep and shallow layer characteristics and micro-topography information in the input image data by adopting a data enhancement strategy suitable for landslide remote sensing images, generates more accurate landslide unlabeled data, and introduces additional information through strong data enhancement perturbation, thereby reducing overfitting of limited labeled landslide data.

[0097] 3) Input the student model. During the landslide identification process, the student model The feature F of the hidden layer output at the stage where the overlap between the medium and low density areas and the separation category boundary is the highest in Apply F-Drop feature perturbation to obtain the features after applying feature perturbation ,Will Input Student Model In the subsequent levels, the landslide identification results are obtained ,according to ,by Construct a feature view consistency loss function for the target.

[0098] It should be noted that the premise for consistency learning to learn knowledge from training samples without landslide labels is to meet the clustering hypothesis, that is, the decision boundary of the separation category should be located in the low-density area, so as to avoid dividing the same type of data into different categories. Therefore, the stage where the low-density area and the separation category boundary overlap the most is selected for feature perturbation, which can ensure that the decision function boundary of the feature perturbation perspective consistency learning is easier to reach the low-density area.

[0099] Therefore, in some embodiments, see Figure 6 , the process of determining the stage where the low-density area has the highest overlap with the separation class boundary can include:

[0100] There will be landslide labeled training samples S LAny landslide image input in is only determined by the supervised learning loss function The trained recognition model obtains high-dimensional features of the intermediate layers of the recognition model (because in the process of neural network training, the input is usually a visible light image, which has only three dimensions of RGB in the channel dimension, but as the training progresses, each network layer gradually extracts and converts the original input and represents it through complex features. These features usually have multiple dimensions, each dimension represents a different feature or attribute. The features of other network layers except the input and output features of the model can be called high-dimensional features), accumulates the high-dimensional features in the channel dimension, reduces the high-dimensional features to two-dimensional features, calculates the average Euclidean distance between the central pixel of the two-dimensional feature and its surrounding 8 adjacent pixels through a 3×3 sliding window, and obtains the initial local variation degree. The initial local variation degree is sequentially normalized by maximization and minimization and visualized as a single-channel grayscale image to obtain a local variation degree map. By comparing the local variation degree map with the landslide mark, the stage with the highest overlap between the low-density area and the separation category boundary is determined.

[0101] The optimal stage for feature perturbation assignment is determined by observing the local variation of high-dimensional features at the intermediate levels of the recognition model and the overlap of low-density regions separating classes.

[0102] It should be noted that feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area has the highest overlap with the separation category boundary. The specific process can be:

[0103] In the channel dimension, F in Summing and normalizing to get , and construct with Empty matrix F of consistent shape e ,Will Compare the pixel value with the sampling threshold. For pixels greater than the sampling threshold, e The same position of F is filled with 0. For pixels smaller than the sampling threshold, e Fill the same position of F with 1. in With the filled F e Perform Hadamard product operation to obtain the characteristics after applying characteristic perturbation .

[0104] It should be noted that the Dice loss is used here to construct the feature perspective consistency loss function, which can be expressed as:

[0105] ;

[0106] In the formula, is the feature view consistency loss function.

[0107] It should be noted that feature perspective consistency learning can enable the model to more fully explore and learn the rich high- and low-dimensional information encoded in the feature map, further improving the detection accuracy of landslides.

[0108] 4) Add Gaussian noise and input to the teacher model , and obtain the landslide identification results ,according to ,by Construct a model view consistency loss function for the target.

[0109] It should be noted that the teacher model and student models The structure is consistent, but the parameter initialization methods are inconsistent. For example, the student model can use layer-by-layer traversal to manually initialize the parameters, and the teacher model can use the kaiming normal distribution to initialize the parameters; determine the mean and standard deviation of the Gaussian noise disturbance, The parameters are updated through the consistency learning loss, Parameters are passed to The parameters are updated using an exponential sliding average.

[0110] It should be noted that the binary cross entropy loss or Dice loss is used here to construct the model perspective consistency loss function, which can be expressed as:

[0111] ;

[0112] In the formula, is the model perspective consistency loss function.

[0113] It should be noted that model perspective consistency learning can break the limitations of a single learning perspective and form a virtuous circle through mutual learning between student models, jointly promote the learning of feature distribution, and reduce the prediction oscillation of unlabeled landslide remote sensing images.

[0114] 5) The different perspective consistency learning is implemented in a single learning stream, and the supervised learning loss function, input perspective consistency loss function, feature perspective consistency loss function and model perspective consistency loss function are weightedly summed to construct a multi-perspective consistency learning joint loss function, constrain the student model parameter update, and obtain the recognition model.

[0115] It should be noted that the joint loss function of multi-view consistency learning can be expressed as:

[0116] ;

[0117] In the formula, They represent the above-mentioned supervised learning loss function, the first input perspective consistency loss function, the second input perspective consistency loss function, the feature perspective consistency loss function and the model perspective consistency loss function corresponding to the landslide image in the i-th training sample without landslide mark, respectively. Learning a joint loss function for multi-view consistency, is the weight coefficient, .

[0118] In the above training process, multi-view consistency learning is integrated into a unified objective function, which broadens the disturbance space and realizes the learning of more discriminative features of unlabeled landslide remote sensing images.

[0119] return Figure 1 In step 3 of the embodiment, all the remote sensing image blocks obtained by segmentation are input into the recognition model to obtain the landslide recognition results of each remote sensing image block, and the landslide mapping results of the affected area are obtained according to the landslide recognition results.

[0120] It should be noted that the entire remote sensing image needs to be input for recognition, that is, all remote sensing image blocks are input into the recognition model. If the remote sensing image is a continuous landslide image, the landslide recognition results are sequentially spliced ​​to obtain the landslide mapping result. If it is a discontinuous landslide image, there is no need to splice, and the recognition result is the landslide mapping result.

[0121] The above method constructs a landslide dataset by performing overlapping sliding segmentation on the remote sensing images of the disaster-stricken area, and selects a certain proportion of landslide image blocks from the training set for manual labeling as labeled data, and the remaining image blocks are used as unlabeled data. Then, by comprehensively using consistency regularization and self-training methods, the unification of multi-view disturbances is achieved by forcibly aligning the input, features, and model perturbation landslide mapping results with the generated weak data enhancement perturbation pseudo-labels, thereby broadening the disturbance space and enabling the model to learn more discriminative features of unlabeled landslide remote sensing images, thereby obtaining a multi-view consistency landslide recognition model, using the trained model to identify all segmented images, and then obtaining the landslide mapping results of the disaster-stricken area based on the recognition results.

[0122] The above method forces the input, features, and model-perturbed landslide mapping results to be aligned with the generated weak data-enhanced perturbation pseudo-labels during model training, thereby achieving the unification of multi-perspective perturbations and broadening the perturbation space, thereby solving the problem of limited consistent learning perspective in landslide mapping. In addition, the above method can effectively reduce the cost of landslide mapping and provide reliable technical support for meeting the tasks of rapid and low-cost disaster relief and geological disaster investigation.

[0123] The following uses the open source Bijie landslide dataset as an example, with the landslide visible light remote sensing image and the corresponding digital elevation model (DEM) as the data source to verify the above method in detail.

[0124] In the above verification process, some parameter values ​​in the method are as follows: M T_L = 10, 30, 104, 154 for testing, the corresponding M T_U =504, 484, 410, 360, M in the validation set V_L =44, the set to be tested is M I_U =212, H=W=256 represent the length and width of the landslide image respectively, C=4 represents that the landslide image has four bands: red (R), green (G), blue (B), and digital elevation model (DEM), and the mask M value is 256×256. =0.3, the sampling threshold range is (0.1, 0.3), the mean of Gaussian noise perturbation is 0, and the standard deviation is 1, and the final value is limited to the range of [-0.1, 0.1], All are 0.25.

[0125] In the above verification process, the original visible light remote sensing images and DEM of the Bijie landslide dataset cover the area. Figure 7 ; The visualization results of the local variation of high-dimensional features in the middle layer of some Bijie landslide dataset images calculated based on U-Net are shown in Figure 8 ,The stage of the highest overlap between the low-density area and the boundary of the ,separation category is determined by comparing it with the manually ,determined landslide markers is after the fourth upsampling in the decoding ,stage.

[0126] In the above verification process, the set to be tested is directly input into the recognition model to obtain Fig. 9 The Bijie landslide dataset is compared with the algorithm and the landslide mapping result map of the present invention under different proportions of landslide marked training samples. Since the Bijie landslide dataset provides non-continuous landslide image blocks after segmentation, there is no need to perform sequential splicing operations on the landslide recognition results, and the landslide recognition results of the test set are directly used as the landslide mapping results. Through verification, it is proved that the above method has achieved the desired effect, that is, it solves the problem of limited consistent learning perspective in landslide mapping, and can effectively reduce the cost of landslide mapping, providing reliable technical support for meeting the tasks of rapid and low-cost disaster relief and geological disaster investigation.

[0127] Fig.10 is a schematic diagram of an embodiment of the remote sensing image landslide mapping device disclosed in the present invention, Fig.10The embodiment of the invention is a virtual device which can be loaded and executed by a landslide mapping terminal (such as a computer), and includes a segmentation and construction module, a training and verification module, and a mapping module.

[0128] The segmentation and construction module of the embodiment is configured to segment the remote sensing image of the disaster-stricken area to obtain remote sensing image blocks, and select some remote sensing image blocks to construct a training set.

[0129] The training verification module of the embodiment is configured to train the student model using the training set and use the trained student model as the recognition model; wherein, when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, It is the result after weak data enhancement disturbance on the landslide images in the training samples without landslide markers; for The result after CutMix data enhancement perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and in the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the overlap between the low-density area and the separation category boundary in the student model is the highest; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model.

[0130] The mapping module of the embodiment is configured to input all remote sensing image blocks obtained by segmentation into a recognition model to obtain landslide recognition results of each remote sensing image block, and obtain landslide mapping results of the affected area based on the landslide recognition results.

[0131] During training, the recognition model adopted by the above-mentioned device constructs an input perspective consistency loss function, a feature perspective consistency loss function and a model perspective consistency loss function, forcing the input, feature and model perturbation landslide mapping results to be aligned with the generated weak data enhancement perturbation pseudo-labels, thereby achieving the unification of multi-perspective perturbations and broadening the perturbation space, thereby solving the problem of limited consistent learning perspective in landslide mapping.

[0132] Based on the same technical solution, the present disclosure also relates to a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by a computing device, the computing device executes a remote sensing image landslide mapping method.

[0133] Based on the same technical solution, the present disclosure also relates to a computer device, including one or more processors and one or more memories, one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing a remote sensing image landslide mapping method.

[0134] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0138] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A method for mapping landslides using remote sensing images, characterized in that: include: Segment the remote sensing images of the disaster-stricken area to obtain remote sensing image blocks, and select some remote sensing image blocks to construct a training set; The training set is used to train the student model, and the trained student model is used as the recognition model; when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, It is the result after weak data enhancement disturbance on the landslide images in the training samples without landslide markers; for The result after CutMix data enhancement perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and in the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the overlap between the low-density area and the separation category boundary in the student model is the highest; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model; All remote sensing image blocks obtained by segmentation are input into the recognition model to obtain the landslide recognition results of each remote sensing image block. According to the landslide recognition results, the landslide mapping results of the affected area are obtained.

2. The remote sensing image landslide mapping method according to claim 1, characterized in that: according to ,by Construct an input view consistency loss function for the target, include: Will Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation results and ,Will Perform the first strong data enhancement perturbation and the second strong data enhancement perturbation respectively to obtain the data enhancement perturbation and ;in, and For any pair ; Will and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result ; Will and Input the student model respectively and obtain Landslide identification results and Landslide identification results ,Will and Input the student model respectively and obtain Landslide identification results and Landslide identification results ; Will and Perform the first CutMix data enhancement perturbation to obtain the data enhancement perturbation result ,Will and Perform the second CutMix data enhancement perturbation to obtain the data enhancement perturbation result ; according to ,by Construct the first input view consistency loss function for the target ,according to ,by Construct a second input view consistency loss function for the target .

3. The remote sensing image landslide mapping method according to claim 2, characterized in that: First input perspective consistency loss function And the second input perspective consistency loss function , the formula is: ； ； In the formula, is the Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, and are the results of weak data enhancement perturbation on the landslide images in the i-th and j-th training samples without landslide markers, and They are respectively after CutMix data enhancement perturbation and , and They are and Input the landslide identification results obtained by the student model, Confidence threshold for filtering weak data enhancement and noise in landslide identification results.

4. The method for mapping landslides using remote sensing images according to claim 2, characterized in that: The first CutMix data enhancement perturbation and the second CutMix data enhancement perturbation use anchor point constraint CutMix and have different landslide masks; the formula of the landslide mask is: ； Where M A is the landslide mask, M is the randomly generated initial landslide mask, and The sizes are the same, Argmax is the Argmax operation, and ⊙ is the Hadamard product operation.

5. The method for mapping landslides using remote sensing images according to claim 1, characterized in that: The feature perspective consistency loss function is as follows: ； In the formula, is the feature view consistency loss function, is the Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, is the result of weak data enhancement perturbation on the landslide image in the i-th training sample without landslide markers. for Input the landslide identification results obtained by the student model, for The landslide identification result obtained by the student model is input, and in the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the boundary of the separation category.

6. The method for mapping landslides using remote sensing images according to claim 1 or 5, characterized in that: Apply feature perturbations to the features of the hidden layer output at the stage where the low-density area in the student model has the highest overlap with the boundary of the separation category, including: In the channel dimension, F in Summing and normalizing to get , and construct with Empty matrix F of consistent shape e ; Among them, F in It is the hidden layer output of the stage where the low-density area in the student model has the highest overlap with the boundary of the separation category; Will Compare the pixel value with the sampling threshold. For pixels greater than the sampling threshold, e The same position of F is filled with 0. For pixels smaller than the sampling threshold, e Fill the same position with 1; F in With the filled F e Perform Hadamard product operation to obtain the characteristics after applying characteristic perturbation .

7. The method for mapping landslides using remote sensing images according to claim 1, characterized in that: The stage where the low-density area has the highest overlap with the boundary of the separated categories is determined before constructing the feature view consistency loss function. The determination process includes: Input any landslide image in the training sample with landslide mark into the recognition model trained only by the supervised learning loss function to obtain the high-dimensional features of the intermediate level of the recognition model; Accumulate high-dimensional features in the channel dimension to reduce the high-dimensional features to two-dimensional features; The average Euclidean distance between the central pixel of the two-dimensional feature and its surrounding adjacent pixels is calculated by sliding the window to obtain the initial local variation degree; The initial local variation degree is normalized by maximum and minimum in sequence and visualized as a single-channel grayscale image to obtain a local variation degree map; By comparing the local change degree map with the landslide markers, the stage with the highest overlap between the low-density area and the boundary of the separation category is determined.

8. The method for mapping landslides using remote sensing images according to claim 1, characterized in that: Model perspective consistency loss function, the formula is: ； In the formula, is the model perspective consistency loss function, is the binary cross entropy loss function or Dice loss function, M T_U is the total number of training samples without landslide labels in the training set, is the result of weak data enhancement perturbation on the landslide image in the i-th training sample without landslide markers. for Input the landslide identification results obtained by the student model.

9. A remote sensing image landslide mapping device, characterized in that: include: The segmentation and construction module segments the remote sensing images of the disaster-stricken area to obtain remote sensing image blocks, and selects some remote sensing image blocks to construct a training set; The training and verification module uses the training set to train the student model, and uses the trained student model as the recognition model; when training the student model, according to ,by Construct an input perspective consistency loss function for the target; according to ,by Construct a feature perspective consistency loss function for the target; according to ,by Construct a model view consistency loss function for the target; for Input the landslide identification results obtained by the student model, It is the result after weak data enhancement disturbance on the landslide images in the training samples without landslide markers; for The result after CutMix data enhancement perturbation; for Input the landslide identification results obtained by the student model, for The results after strong data enhancement perturbation and CutMix data enhancement perturbation respectively; for The landslide identification result obtained by the student model is input, and in the landslide identification process, feature perturbation is applied to the features of the hidden layer output at the stage where the overlap between the low-density area and the separation category boundary in the student model is the highest; for Landslide identification results obtained by adding Gaussian noise and inputting it into the teacher model; The mapping module inputs all the remote sensing image blocks obtained by segmentation into the recognition model to obtain the landslide recognition results of each remote sensing image block, and obtains the landslide mapping results of the affected area based on the landslide recognition results.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs include instructions. When the instructions are executed by a computing device, the computing device executes any one of the methods of claims 1 to 8.

Images