A landslide change detection sample enhancement method considering terrain factors

Abstract

This invention provides a landslide change detection sample enhancement method considering topographic factors, belonging to the field of image processing technology. The method includes: cropping landslide samples to obtain pixel clusters with the ground truth label "landslide" as landslide mask samples, and performing random X and Y axis flipping and / or random angle rotation; selecting scene samples without landslide labels, calculating the normalized mutual information between the post-landslide image corresponding to the landslide mask sample and the post-temporal image of the scene sample, obtaining candidate scenes and calculating the slope aspect and slope of each pixel to obtain slope areas prone to landslide development; obtaining mosaic candidate areas in the slope areas prone to landslide development, calculating the average slope aspect, determining the slope aspect, and mosaicking the transformed landslide mask sample onto the mosaic candidate area to obtain a new landslide change detection sample. This invention rapidly generates landslide change detection samples by mosaicking landslide patches onto slope areas with significant differences, ensuring reasonableness.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for enhancing landslide change detection samples that takes into account topographic factors. Background Technology

[0002] Landslides triggered by earthquakes, rainfall, snowmelt, or human engineering disturbances are often highly dangerous and pose a serious threat to the lives and property of people in landslide-prone areas.

[0003] Accurately obtaining key spatial information such as the location and extent of newly formed landslides in the immediate aftermath of a landslide disaster is crucial, as it helps disaster prevention and mitigation departments assess the situation and formulate relief plans.

[0004] Landslide change detection technology based on multi-temporal remote sensing imagery offers significant advantages in efficiency and labor costs compared to field surveys, making it the mainstream technique for identifying clustered landslides after disasters. In recent years, artificial intelligence (AI) technology has developed rapidly, significantly improving the accuracy of landslide change detection. However, high-performance intelligent change models are often structurally complex, requiring large-scale labeled samples for thorough training. Large-scale landslide change detection samples remain scarce, limiting the practical application of cutting-edge intelligent change detection models in landslide change detection tasks.

[0005] Currently, most enhancement methods for landslide change detection samples originate from the field of computer vision. These include geometric enhancement methods such as rotation and flipping, grayscale enhancement methods such as filtering and adding noise points, and multi-sample stitching methods such as Cutmix and Mixup. Using these enhancement methods, the number of training samples can be rapidly expanded, meeting the needs of artificial intelligence model parameters from initialization to full training. This reduces the dependence of artificial intelligence model training on large-scale labeled samples to some extent, allowing researchers to quickly deploy models based on a small number of labeled samples, thus promoting the performance improvement of artificial intelligence models in tasks such as natural image classification and remote sensing image semantic segmentation.

[0006] With the development of advanced artificial intelligence technologies such as large-scale visual models in recent years, the demand for large-scale samples in cutting-edge artificial intelligence models has increased rapidly. Although the above-mentioned augmentation methods can be used in combination to expand the samples exponentially, the sample feature diversity obtained by this method is poor, making it difficult to fully train cutting-edge artificial intelligence models with high feature generalization and expressive capabilities.

[0007] In addition, landslide change detection tasks have distinct geoscientific characteristics. Samples generated using augmentation methods such as Cutmix and Mixup often contain certain irrationalities. For example, augmented landslide samples may be located in flat areas such as urban areas or water bodies where landslides are unlikely to occur. This can mislead the model during training due to irrational samples, negatively impacting the model's training effectiveness and prediction performance.

[0008] Therefore, it is urgent to develop a method that can rapidly generate a large number of landslide change detection samples with significant differences in characteristics and reasonable spatial locations, taking into account the geoscientific characteristics of landslide change detection tasks. Summary of the Invention

[0009] To address the aforementioned issues, this invention provides a landslide change detection sample enhancement method that considers topographic factors. By patching landslide patches onto slope areas with significant differences, a large number of simulated landslide change detection samples with obvious visual and morphological differences are quickly generated, ensuring the rationality of the landslide change detection samples.

[0010] To achieve the above objectives, the present invention provides a method for enhancing landslide change detection samples that takes into account topographic factors, comprising: cropping each landslide sample labeled with a pixel-by-pixel landslide ground truth value to obtain pixel clusters labeled as landslide as landslide mask samples;

[0011] The X and Y axes of each landslide Mask sample are randomly flipped and / or rotated at random angles to obtain the transformed landslide Mask samples.

[0012] Select scene samples without landslide labels, calculate the normalized mutual information between the uncropped post-landslide image corresponding to the landslide Mask sample and the post-temporal image of the scene sample, and obtain candidate scenes that are significantly different from the background of the post-landslide image patches.

[0013] For the obtained candidate scenes, the slope aspect and slope of each pixel are calculated to obtain the slope areas where landslides are prone to develop.

[0014] Extract the skeleton of the transformed landslide mask sample as the landslide sliding trajectory, and calculate the landslide movement direction for each landslide mask sample;

[0015] In slope areas prone to landslide development, candidate areas for patching that can accommodate transformed landslide mask samples are obtained, and the average slope aspect of the candidate areas is calculated.

[0016] If the average slope aspect of the candidate patchwork area differs from the landslide movement direction of the transformed landslide mask sample by no more than a preset threshold, then the transformed landslide mask sample is patched onto the candidate patchwork area to obtain a new landslide change detection sample.

[0017] As a further improvement of the present invention, each landslide mask sample consists of a post-landslide image patch and corresponding label data, and has the same number of landslide instances as its corresponding landslide sample.

[0018] As a further improvement of the present invention, the smaller the calculated normalized mutual information value, the greater the overall difference between the two. If the normalized mutual information value is less than a specified threshold, then the candidate scene is a candidate scene that is significantly different from the background of the landslide image patch.

[0019] As a further improvement of the present invention, if the normalized mutual information value is greater than a specified threshold, a new set of scene samples is selected and the normalized mutual information of the uncropped post-landslide image corresponding to the landslide Mask sample and the post-temporal image of the set of scene samples is recalculated until candidate scenes are obtained.

[0020] As a further improvement of the present invention, based on the digital elevation model, the third-order finite difference method is used to calculate the slope aspect and slope of each pixel in the candidate scene, and the area with the slope magnitude within the specified threshold range is regarded as the slope area where landslides are prone to develop.

[0021] As a further improvement of the present invention, the Zhang-Seun thinning algorithm is used to extract the skeleton of the transformed landslide Mask sample as the landslide sliding trajectory, and the mean value of the skeleton line vector is calculated to obtain the average trajectory direction of the landslide.

[0022] Calculate the distance from the edge of the landslide along the perpendicular direction of the first and fourth quintile points of the landslide trajectory. The side with the larger distance is the starting direction of the landslide.

[0023] The landslide motion direction of each landslide mask sample is obtained based on the average trajectory direction and the starting direction.

[0024] As a further improvement of the present invention, a random search method is adopted to search for slope regions that can completely accommodate the transformed landslide Mask sample in all slope regions where landslides are prone to development, as candidate areas for patching.

[0025] If no candidate area for tiling is obtained after searching more than a specified number of times, a new set of landslide mask samples is selected and the X and Y axes are randomly flipped and / or rotated at random angles to obtain new transformed landslide mask samples. The search is then performed again until a candidate area for tiling is obtained.

[0026] As a further improvement of the present invention, the slope aspect of each pixel in the tiling candidate area is converted into a unit vector, and the average slope aspect is calculated.

[0027] As a further improvement of the present invention, if the average slope direction of the collage candidate area differs from the landslide movement direction of the transformed landslide Mask sample by more than a preset threshold, then the collage candidate area is re-acquired and the average slope direction of the collage candidate area is calculated.

[0028] As a further improvement of the present invention, if the number of times the candidate patching area is re-acquired exceeds a specified number, and the difference between the average slope direction of the candidate patching area and the landslide movement direction of the transformed landslide Mask sample still exceeds a preset threshold, then the transformed landslide Mask sample is rotated clockwise by an angle equal to the preset threshold, and a suitable candidate patching area is searched again until the direction difference is less than the threshold. Then, the clockwise rotated landslide Mask sample is patched onto the candidate patching area to obtain a new landslide change detection sample.

[0029] If a suitable candidate region for tiling is not found after a specified number of attempts using the rotating landslide mask samples, a new set of landslide mask samples will be selected until a suitable candidate region for tiling is found.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] This invention, based on a small number of landslide annotation samples, rapidly generates a large number of simulated landslide change detection samples with obvious visual and morphological differences by pasting landslide patches onto slope areas that are significantly different from them. This provides sample support for the full training and transfer application of intelligent models for complex landslide change detection, alleviates the problems of model training difficulties and poor regional transferability under small sample conditions, and promotes the application and improvement of cutting-edge artificial intelligence models in landslide change detection tasks.

[0032] This invention comprehensively considers topographical factors in landslide development. By piecing together cropped landslide patches onto slope areas with significant differences, it can expand the number of landslide change detection samples to include more visually and morphologically distinct samples. The newly generated landslide change detection samples have stronger spatial location rationality. This patent can provide sample support for the full training and transfer application of intelligent models for complex landslide change detection, alleviating the difficulties in model training and poor regional transferability under small sample conditions, and promoting the practical application and innovative improvement of cutting-edge artificial intelligence models in landslide change detection tasks.

[0033] In this invention, when tiling landslide patches into a new scene, the topographical factors of the new scene, including slope and aspect, are comprehensively considered. The tiled landslide patches are completely located in a slope area within the new scene with a slope suitable for landslide development, and the main sliding direction of the tiled landslide patches is consistent with the overall slope aspect of the area. Attached Figure Description

[0034] Figure 1This is a schematic diagram of a landslide change detection sample enhancement method considering topographic factors, as disclosed in an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] The present invention will now be described in further detail with reference to the accompanying drawings:

[0037] like Figure 1 As shown, the present invention provides a method for enhancing landslide change detection samples considering topographic factors, comprising:

[0038] S1. Each landslide sample labeled with the ground truth value of each pixel is cropped to obtain the pixel clusters with the ground truth label of landslide as landslide Mask samples.

[0039] in,

[0040] Landslide samples with pixel-by-pixel landslide ground truth annotations are sample data with real landslide information annotations, where each pixel in remote sensing images or other geographic data is clearly marked as to whether it belongs to a landslide area.

[0041] For example, on a remote sensing image with a resolution of 0.5 meters, each 0.5m × 0.5m pixel within the landslide area is marked as "landslide", and pixels in non-landslide areas are marked as "non-landslide". This set of marked pixels is the pixel-by-pixel landslide ground truth labeling sample.

[0042] In actual annotation, the pixels of the landslide boundary are precisely defined based on professional judgment. If more than 50% of a pixel on the edge of the landslide area is within the landslide area, it is marked as "landslide"; otherwise, it is marked as "non-landslide".

[0043] Furthermore,

[0044] Each landslide Mask sample consists of a post-landslide image patch and corresponding label data, and has the same number of landslide instances as its corresponding landslide sample.

[0045] All the landslide mask samples obtained from the cropping process together form the landslide mask sample set.

[0046] S2. Randomly flip and / or rotate the X and Y axes of each landslide Mask sample to obtain the transformed landslide Mask samples.

[0047] in,

[0048] A set of landslide mask samples can be randomly selected as a benchmark, and geometric transformations such as random flipping of the X and Y axes and random angle rotation can be performed on them.

[0049] S3. Select scene samples without landslide labels, calculate the normalized mutual information (NMI) between the uncropped post-landslide image corresponding to the landslide Mask sample and the post-temporal image of the scene sample, and obtain candidate scenes that are significantly different from the background of the landslide image patches.

[0050] in,

[0051] A set of scene samples without landslide labels was randomly selected. The scene samples included dual-temporal images and data based on a digital elevation model.

[0052] Dual-temporal imagery refers to remote sensing images of the same geographic area acquired at two different times. Based on digital elevation models, it means that the scene sample contains terrain data derived from digital elevation models. Digital elevation models record the elevation information of each point in the scene sample through raster or vector data, and are a digital representation of the three-dimensional terrain of the earth's surface.

[0053] Furthermore,

[0054] The smaller the calculated normalized mutual information (NMI) value, the greater the overall difference between the two. If the normalized mutual information value is less than the specified threshold, then the candidate scene is a candidate scene that is significantly different from the background of the landslide image patch.

[0055] If the normalized mutual information value is greater than the specified threshold, a new set of scene samples is selected and the normalized mutual information (NMI) of the uncropped post-landslide image corresponding to the landslide Mask sample and the post-temporal image of the same set of scene samples is recalculated until candidate scenes are obtained.

[0056] This step filters out candidate scenes from the scene samples that are significantly different from the background of the landslide patches, so that the background of the subsequently enhanced sample has a large feature difference from the background of the original sample.

[0057] S4. Calculate the slope aspect and slope of each pixel for the obtained candidate scenes to obtain the slope areas where landslides are prone to develop.

[0058] in,

[0059] For the candidate scenes obtained, the slope aspect and slope of each pixel in the candidate scene are calculated using the third-order finite difference method based on the digital elevation model corresponding to the candidate scene. Areas with slope magnitude within a specified threshold range are identified as slope areas prone to landslide development.

[0060] For example, areas with slopes between 15 and 45 degrees can be designated as slope areas prone to landslides.

[0061] Furthermore,

[0062] This step filters out areas in the candidate scenarios where the slope is prone to landslides, ensuring that subsequent landslide patches are not pasted onto flat areas, gentle slopes, or extremely steep areas where landslides will not occur, thus ensuring that the generated landslide samples are reasonable in terms of terrain.

[0063] S5. Extract the skeleton of the transformed landslide mask sample as the landslide sliding trajectory, and calculate the landslide movement direction of each landslide mask sample.

[0064] in,

[0065] The Zhang-Seun thinning algorithm is used to extract the skeleton of the transformed landslide mask sample as the landslide sliding trajectory, and the mean value of the skeleton line vector is calculated to obtain the average trajectory direction of the landslide.

[0066] Calculate the distance from the edge of the landslide along the perpendicular direction of the first and fourth quintile points of the landslide trajectory. The side with the larger distance is the starting direction of the landslide.

[0067] The landslide motion direction of each landslide mask sample is obtained based on the average trajectory direction and the starting direction.

[0068] Furthermore,

[0069] The Zhang-Seun thinning algorithm iteratively removes edge pixels, ultimately compressing the contour of the landslide area into a skeleton with a width of only one pixel.

[0070] For example, for an irregularly shaped landslide area, the refined skeleton will retain its central axis, similar to the supporting role of the human skeleton in the body shape.

[0071] The specific steps of the algorithm include:

[0072] Neighborhood analysis: Define the 8-neighborhood of the center pixel, and calculate the number of foreground pixels in the neighborhood (N(p1)) and the number of 0-1 changes (S(p1)).

[0073] Two scans: The first scan removes edge pixels that meet specific conditions (such as having 2-6 neighboring pixels), and the second scan further optimizes the skeleton structure until there are no pixels left to delete.

[0074] Specifically,

[0075] The sliding path of a landslide usually follows the central axis of its mass distribution. The refined skeleton reflects this axis, which is the potential sliding trajectory.

[0076] To calculate the average trajectory direction, the skeleton needs to be converted into an ordered vector sequence, including: starting from one end of the skeleton, traversing each pixel in sequence and recording its coordinate sequence; for each skeleton point, calculating the coordinate difference (Δx, Δy) between it and the next point to obtain the direction vector of that skeleton segment. For example, if the coordinates of two adjacent points are (10, 20) and (15, 25), then the vector is (5, 5) with a direction of 45°.

[0077] Sum the x and y components of all direction vectors and divide by the total number of vectors to obtain the average direction vector. Calculate the angle of the average vector using the arctangent function. This angle is the average trajectory direction of the landslide.

[0078] Divide the skeleton trajectory into five equal segments according to its length, and take the first (20% position) and fourth (80% position) points as equal division points. For example, if the total length of the trajectory is 100 pixels, the first point is located at 20 pixels and the fourth point is located at 80 pixels. For each five-equal division point, calculate the perpendicular direction of its trajectory tangent. Measure the distance to the landslide edge in both the positive and negative directions of the perpendicular. The direction corresponding to the larger distance in the positive and negative directions of the perpendicular is the candidate starting direction.

[0079] S6. In the slope area where landslides are prone to develop, obtain the candidate patching area that can accommodate the transformed landslide Mask sample, and calculate the average slope aspect of the candidate patching area.

[0080] in,

[0081] Using a random search method, a slope region that can completely accommodate the transformed landslide Mask sample is searched among all the landslide-prone slope regions obtained in step S4 as a mosaic candidate region. The slope aspect of each pixel in the mosaic candidate region is converted into a unit vector, and the average slope aspect is calculated.

[0082] Furthermore,

[0083] If no candidate area for tiling is obtained after searching more than the specified number of times, steps S2-S6 are executed again to randomly flip the X and Y axes and / or rotate the landslide mask samples to obtain new transformed landslide mask samples, and the search is performed again until a candidate area for tiling is obtained.

[0084] S7. If the average slope aspect of the candidate patching area and the landslide movement direction of the transformed landslide Mask sample do not differ from the preset threshold, then the transformed landslide Mask sample is patched onto the candidate patching area to obtain a new landslide change detection sample.

[0085] in,

[0086] The preset threshold is set to 10 degrees, and the difference does not exceed the preset threshold, that is, the directional difference is within 10 degrees.

[0087] Furthermore,

[0088] If the average slope aspect of the candidate patchwork area differs from the landslide movement direction of the transformed landslide Mask sample obtained in step S5 by more than a preset threshold (more than 10 degrees), then steps S6-S7 are executed again to obtain the candidate patchwork area and calculate the average slope aspect of the candidate patchwork area.

[0089] Furthermore,

[0090] If the number of times the candidate patching area is re-acquired (i.e., the number of times step S6 is repeatedly executed) exceeds the specified number, and the difference between the average slope aspect of the candidate patching area and the landslide movement direction of the transformed landslide mask sample still exceeds the preset threshold (more than 10 degrees), then the transformed landslide mask sample is rotated clockwise by an angle equal to the preset threshold, i.e., rotated clockwise by 10 degrees, and a suitable candidate patching area is searched again. This process continues until the direction difference is less than the threshold, and then the clockwise rotated landslide mask sample is patched onto the candidate patching area to obtain a new landslide change detection sample. If the landslide mask sample is rotated more than the specified number of times and a suitable candidate patching area is still not found, then steps S2-S7 are repeated until a suitable candidate patching area is found.

[0091] Specifically, through steps S4, S5, S6, and S7, the sliding direction of the landslide in the newly pieced-together landslide sample can be kept consistent with the slope direction, further ensuring that the landslide sample is reasonable in terms of terrain.

[0092] Advantages of this invention:

[0093] This invention comprehensively considers the topographical factors of landslide development. By patching landslide patches onto slope areas with significant differences, it can expand the number of landslide change detection samples with more significant visual and morphological differences, and the newly generated landslide change detection samples have stronger spatial location rationality. This patent can provide sample support for the full training and transfer application of intelligent models for complex landslide change detection, alleviating the problems of difficult model training and poor regional transferability under small sample conditions, and promoting the practical application and improvement of cutting-edge artificial intelligence models in landslide change detection tasks.

[0094] In this invention, when landslide patches are pieced together to a new scene, the topographical factors of the new scene, including slope and aspect, are taken into account. The pieced landslide patches are completely located in the slope area of ​​the new scene where the slope is suitable for landslide development, and the main sliding direction of the pieced landslide patches is consistent with the overall slope aspect of the area.

[0095] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for enhancing landslide change detection samples considering topographic factors, characterized in that, include: Select scene samples without landslide labels, calculate the normalized mutual information between the uncropped post-landslide image corresponding to the landslide Mask sample and the post-temporal image of the scene sample, and obtain candidate scenes that are significantly different from the background of the landslide image patches. The X and Y axes of the landslide mask samples are randomly flipped and / or rotated at random angles. The skeleton of the transformed landslide mask samples is extracted as the landslide sliding trajectory, and the landslide movement direction of each landslide mask sample is calculated. In slope areas prone to landslide development, candidate areas for patching that can accommodate transformed landslide mask samples are obtained, and the average slope aspect of the candidate areas is calculated. If the average slope aspect of the candidate patchwork area differs from the landslide movement direction of the transformed landslide mask sample by no more than a preset threshold, then the transformed landslide mask sample is patched onto the candidate patchwork area to obtain a new landslide change detection sample.

2. The landslide change detection sample enhancement method considering topographic factors according to claim 1, characterized in that: The smaller the calculated normalized mutual information value, the greater the overall difference between the two. If the normalized mutual information value is less than a specified threshold, then the candidate scene is a candidate scene that is significantly different from the background of the landslide image patch.

3. The landslide change detection sample enhancement method considering topographic factors according to claim 2, characterized in that: If the normalized mutual information value is greater than a specified threshold, a new set of scene samples is selected and the normalized mutual information of the uncropped post-landslide image corresponding to the landslide Mask sample and the post-temporal image of the set of scene samples is recalculated until candidate scenes are obtained.

4. The landslide change detection sample enhancement method considering topographic factors according to claim 1, characterized in that: A random search method was used to search for slope regions that could completely accommodate the transformed landslide mask sample in all landslide-prone slope regions as candidate areas for patching. If no candidate area for tiling is obtained after searching more than a specified number of times, a new set of landslide mask samples is selected and the X and Y axes are randomly flipped and / or rotated at random angles to obtain new transformed landslide mask samples. The search is then performed again until a candidate area for tiling is obtained.

5. The landslide change detection sample enhancement method considering topographic factors according to claim 1, characterized in that: If the average slope direction of the collage candidate area differs from the landslide movement direction of the transformed landslide Mask sample by more than a preset threshold, then the collage candidate area is re-acquired and the average slope direction of the collage candidate area is calculated.

6. The landslide change detection sample enhancement method considering topographic factors according to claim 5, characterized in that: If the number of times the candidate patching area is re-acquired exceeds a specified number, and the difference between the average slope direction of the candidate patching area and the landslide movement direction of the transformed landslide mask sample still exceeds a preset threshold, then the transformed landslide mask sample is rotated clockwise by an angle equal to the preset threshold, and the candidate patching area is searched again until the direction difference is less than the preset threshold. Then, the clockwise rotated landslide mask sample is patched onto the candidate patching area to obtain a new landslide change detection sample.

7. The landslide change detection sample enhancement method considering topographic factors according to claim 6, characterized in that: If no candidate region for tiling is found after a specified number of rotating landslide mask samples, a new set of landslide mask samples will be selected until a candidate region for tiling is found.

Images