Landslide edge identification method and system fusing CNN and ViT

Abstract

The invention discloses a CNN and ViT fused landslide edge identification method and system, and belongs to the field of landslide edge identification, and the method comprises the steps: obtaining a remote sensing image of a landslide region and a corresponding digital elevation model image; inputting the remote sensing image into the CNN, and extracting a first feature map; inputting the digital elevation model image into ViT, and extracting a second feature map; based on the first feature map, constructing an edge response map, and calculating a fuzzy perception map; based on the second feature map, constructing a structure expectation map; carrying out pixel-by-pixel fusion on the fuzzy perception graph and the structure expectation graph to form an occlusion risk thermodynamic graph; based on the occlusion risk thermodynamic diagram, fusion is carried out through a cross attention mechanism, and boundary compensation features are generated; and forming a fusion feature map by the boundary compensation feature and the unshielded region feature map, and inputting the fusion feature map into a decoder to generate a landslide edge probability map. According to the method, the accuracy of landslide boundary recognition under the dust shielding condition is remarkably improved, and the method can serve the fields of primary disaster edge recognition and secondary disaster emergency judgment.

Description

Technical Field

[0001] This invention relates to the field of landslide edge recognition, and more particularly to a landslide edge recognition method and system that integrates CNN and ViT. Background Technology

[0002] Landslides are a typical type of geological hazard, characterized by their suddenness, destructive power, and complex spatial distribution. Accurate identification of landslide boundaries is a core element in landslide hazard monitoring and assessment. With the development of remote sensing technology, landslide identification methods based on remote sensing images have become one of the mainstream research and application directions. Especially after extreme events such as earthquakes and heavy rainfall, timely acquisition of landslide boundary information is crucial for emergency response and disaster analysis. Furthermore, landslides exhibit a certain degree of cascading nature. Particularly after a primary landslide, the instability of residual landslide masses, unrecovered surface damage, or changes in external environmental factors often induce "secondary landslides." A secondary landslide refers to a landslide that occurs after the initial landslide due to unresolved conditions or new triggering factors (such as continuous rainfall, post-earthquake soil loosening, or subsequent construction disturbance). The secondary landslide may occur near the original landslide boundary or extend to a wider area. Timely identification of the boundary range and potential weak points after the first landslide is crucial for assessing the risk of subsequent landslide activity, conducting geological assessments, and developing emergency prevention and control measures. Therefore, improving the accuracy and completeness of landslide boundary identification is not only helpful for post-disaster monitoring and assessment, but also a key foundation for achieving early warning of secondary landslide disasters.

[0003] However, traditional edge recognition methods based on remote sensing images face several challenges. On the one hand, landslides are often accompanied by the raising of large amounts of dust, soil debris, and other materials. On the other hand, landslides often occur after extreme weather and natural events such as heavy rainfall, rainstorms, or earthquakes, in which cloud cover is dense and airflow disturbances are significant. During a landslide, due to the severe destruction of the mountain, the surface soil and rocks slide at high speed under the influence of gravity, resulting in intense friction with the air and easily raising large amounts of dust and fine particulate matter. At the same time, water vapor and low-altitude clouds during rainfall often mix with the dust, forming a dense local occlusion layer. These phenomena manifest in remote sensing images as regional weakening of visible light reflection, blurred texture details, or complete occlusion, causing some landslide boundary areas in the image to appear as "blurred patches" or "broken bands," severely interfering with the normal judgment of edge detection models and reducing the continuity and completeness of boundary recognition. This occlusion not only causes landslide boundaries to exhibit "weak response" or "missing" features in visible light images but also affects the model's judgment of edge continuity, leading to false detections or missed detections. Existing methods often struggle to accurately identify these blurred or broken areas caused by occlusion, resulting in insufficient integrity and reduced reliability of boundary detection results.

[0004] On the other hand, identification methods relying solely on single-modal data have inherent limitations. While two-dimensional texture features based on remote sensing imagery are helpful in extracting apparent information about landslide edges, these features are easily lost in areas with dramatic topographic relief, indistinct textures, or vegetation cover. Digital elevation models (DEMs), as important data sources reflecting changes in topographic elevation, have the potential to reveal the internal structural relationships of landslides, especially in areas with blurred textures or severe occlusion, providing additional spatial morphological information. However, identification methods using only DEMs as input also suffer from insufficient detail representation and limited edge localization accuracy.

[0005] In recent years, deep learning methods have been widely applied to semantic segmentation and object detection tasks in remote sensing scenarios. Among them, Convolutional Neural Networks (CNNs) excel at capturing texture edges and local details due to their excellent local perception capabilities; while Visual Transformers (ViTs), based on self-attention mechanisms, possess strong global modeling capabilities and can capture long-distance dependencies and structural features. In landslide recognition tasks, these two types of models each have their advantages. However, in complex real-world scenarios (including disaster relief, disaster assessment, geological surveys, and early warning scenarios for secondary landslides), a single structural model struggles to simultaneously ensure accurate localization of edge details and global judgment of structural consistency. This is especially problematic when facing scenarios with occlusion or abrupt changes in elevation, easily leading to an imbalance in feature representation.

[0006] Therefore, a landslide edge recognition method that can integrate multimodal features and take into account both local details and global structure is needed to improve the accuracy and stability of boundary recognition in complex occluded environments. Summary of the Invention

[0007] The purpose of this invention is to overcome the problems existing in the prior art and provide a landslide edge recognition method and system that integrates CNN and ViT.

[0008] The objective of this invention is achieved through the following technical solution: Firstly, a landslide edge recognition method integrating CNN and ViT is provided, including: Remote sensing images and corresponding digital elevation model images of the landslide area are acquired, and resolution registration and geometric alignment are performed. The remote sensing images are input into a convolutional neural network to extract the first feature map reflecting the local details of the edges and textures. Input the digital elevation model image into the visual Transformer to extract the second feature map that reflects the terrain structure relationship; Based on the first feature map, an edge response map is constructed and a fuzzy perception map is calculated to identify weak edge response areas that may be obscured by dust. Based on the second feature map, a structural expectation map is constructed to identify regions in the image that should exist structurally but lack edge detection. The fuzzy perception map and the structural expectation map are fused pixel by pixel to form an occlusion risk heat map; Based on the occlusion risk heatmap, local regions of the first feature map and the second feature map are extracted in the high occlusion area, and fused through a cross-attention mechanism to generate boundary compensation features. The boundary compensation features, together with the first and second feature maps directly spliced ​​in the unobstructed area, form a fused feature map, which is then input into the decoder to generate a landslide edge probability map.

[0009] In some embodiments, the method further includes: Thresholding and morphological post-processing operations are performed on the landslide edge probability map to output the final landslide boundary map.

[0010] In some embodiments, the steps of constructing an edge response map and calculating a fuzzy perception map include: The first feature map is input into the edge prediction sub-network to generate an edge response map. The edge response map is a probability map with the same size as the remote sensing image, where the value of each pixel position represents the prediction confidence that the position is the edge of a landslide. For pixel locations in the edge response map where the confidence level is lower than a preset edge response threshold, the corresponding local texture region in the remote sensing image is extracted, and the texture gradient change rate of the local texture region is calculated. Further obtain the slope gradient value in the digital elevation model image corresponding to the pixel position; When a pixel location meets the following occlusion conditions simultaneously, that pixel location is marked as a high-risk area for dust occlusion: Its edge response confidence is lower than the first threshold, its texture gradient change rate is less than the second threshold, and its slope gradient value is greater than the third threshold. A blurred perception map is generated based on the positions of all pixels that meet the occlusion conditions. The blurred perception map is a risk probability map with the same size as the remote sensing image, used to represent the distribution of areas where dust occlusion causes weak edge response.

[0011] In some embodiments, the step of constructing the structural expectation diagram includes: The second feature map is input into the structural attention analysis module, which calculates the structural relevance score between each image location and its global context region based on the self-attention mechanism in the visual Transformer, and generates a structural relevance map. Pixel locations with edge response confidence scores higher than a fourth threshold are extracted from the edge response map and defined as high-response edge regions. In the structural correlation map, image locations that have significant structural correlation with multiple high-response edge regions are identified as structurally strongly correlated regions. If the confidence level of a pixel location in the strongly correlated region of the structure is lower than the fifth threshold in the edge response map, it is marked as a region where the structure is expected to be missing. A structure prediction map is constructed based on the locations of all expected missing structural regions. The structure prediction map is a mask map with the same size as the original remote sensing image.

[0012] In some embodiments, the step of fusing the blurred perception map and the structural expectation map pixel by pixel to form an occlusion risk heat map includes: The values ​​of corresponding pixel positions in the fuzzy perception map and the structure prediction map are weighted and summed to generate an occlusion risk heatmap, where the occlusion risk value of each pixel is determined by the following formula: ; in, The heatmap showing occlusion risk is located at pixel coordinates. The value, Represents the coordinates of the fuzzy perceptual map at the corresponding pixel position. pixel values, This indicates the coordinates of the expected structure map at the corresponding pixel position. pixel values, and To preset the fusion weight coefficients, satisfying ; The occlusion risk heatmap is used to represent the probability value of each pixel being in a dust-occluded area.

[0013] In some embodiments, the step of generating boundary compensation features includes: In the occlusion risk heatmap, pixel locations with occlusion probability values ​​greater than a preset threshold are selected as high occlusion areas. Extract local feature sub-regions corresponding to the highly occluded regions from the first feature map and the second feature map, respectively; The local features in the corresponding local feature sub-region of the first feature map are used as query vectors, and the local features in the corresponding local feature sub-region of the second feature map are used as key vectors, which are then input into the cross-attention fusion module. In the cross-attention fusion module, attention weights are calculated based on the similarity between the query vector and the key vector, and the value vectors are weighted and summed to generate the fused boundary compensation features. The boundary compensation feature is used to replace the feature representation in the original region.

[0014] In some embodiments, the step of generating a landslide edge probability map includes: The boundary compensation features are mapped back to their corresponding positions in the original remote sensing image space to replace the feature representations corresponding to the highly occluded areas in the occlusion risk heatmap. The features of the locations not covered by the occluded risk area in the first feature map and the second feature map are concatenated along the channel dimension to form the fused features of the unoccluded area. The boundary compensation feature and the unoccluded area fusion feature are spliced ​​together according to their spatial positions to form a fusion feature map; The fused feature map is input into the decoder network to generate a landslide edge probability map with the same size as the original remote sensing image. Each pixel value of the probability map represents the confidence level that it belongs to the landslide edge.

[0015] In some embodiments, the step of outputting a landslide boundary map includes: The landslide edge probability map is subjected to threshold segmentation processing, and pixels with probability values ​​higher than a preset edge confidence threshold are marked as edge regions to generate a binary edge mask map. Morphological post-processing operations, including at least one of erosion, dilation, opening and closing operations, are applied to the edge mask image to eliminate isolated noise, fill boundary breaks, or smooth edge contours. The morphologically processed results are output as a landslide boundary map, which is used to represent the final boundary distribution of the landslide area.

[0016] Secondly, a landslide edge recognition system integrating CNN and ViT is provided, including: The input and preprocessing module is used to acquire remote sensing images of the landslide area and the corresponding digital elevation model images, and to perform resolution registration and geometric alignment. The feature extraction module is used to input remote sensing images into a convolutional neural network to extract a first feature map that reflects the local details of edges and textures; and to input digital elevation model images into a visual Transformer to extract a second feature map that reflects the topographic structure relationships. The fuzzy perception map generation module is used to construct an edge response map and calculate a fuzzy perception map based on the first feature map, which is used to identify weak response areas at the edges that may be obscured by dust. The structure prediction map generation module is used to construct a structure prediction map based on the second feature map, which is used to identify regions in the image that should exist structurally but whose edge detection is missing; The occlusion risk heat map generation module is used to fuse the fuzzy perception map and the structural expectation map pixel by pixel to form an occlusion risk heat map. The boundary compensation feature generation module is used to extract local regions of the first feature map and the second feature map in the high occlusion area based on the occlusion risk heat map, and fuse them through a cross-attention mechanism to generate boundary compensation features. The landslide edge probability map generation module is used to combine the boundary compensation features with the first feature map and the second feature map directly spliced ​​in the unobstructed area to form a fused feature map, which is then input into the decoder to generate the landslide edge probability map.

[0017] In some embodiments, it also includes: The landslide boundary map generation module is used to perform thresholding and morphological post-processing operations on the landslide edge probability map and output the final landslide boundary map.

[0018] It should be further noted that the technical features corresponding to the above embodiments can be combined or substituted with each other to form new technical solutions without conflict.

[0019] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention employs multi-source feature complementarity to enhance edge discrimination capability. Specifically, by inputting remote sensing images into a convolutional neural network (CNN) to extract texture and detail features, and inputting digital elevation model images into a visual Transformer (ViT) to extract terrain structure relationship features, it can fully integrate local edge features and global structural information, overcome the problem that single models are easily affected by texture weakening, scale deformation, etc., and improve the robustness and accuracy of landslide boundary recognition.

[0020] 2. This invention enhances edge perception capabilities by employing an occlusion area identification mechanism. Specifically, for the common problem of dust occlusion at landslide sites, a fuzzy perception map is constructed to uncover weak edge response areas caused by dust or vegetation cover. Combined with slope changes at corresponding locations in the DEM, it effectively identifies potential risk areas that "should have edges but insufficient response," thereby improving the system's sensitivity and processing capabilities for occlusion areas.

[0021] 3. This invention uses a structural prediction map to assist in edge loss inference. Specifically, it uses the Transformer self-attention mechanism to construct a structural prediction map, which can locate regions that should appear in the global structural relationship but are missing edge responses in the actual image. This effectively compensates for edge loss caused by occlusion or texture degradation, and enhances the system's structural reasoning ability and boundary continuity.

[0022] 4. This invention employs pixel-by-pixel fusion to construct an occlusion risk heatmap, thereby enhancing region perception. Specifically, by weighted fusion of the fuzzy perception map and the structural expectation map at the pixel level, an occlusion risk heatmap is constructed, which can accurately identify high-risk occlusion areas and achieve more discriminative local area focus in complex environments.

[0023] 5. This invention introduces a cross-attention mechanism for regional feature compensation, which effectively restores the occluded boundary information. Specifically, cross-attention fusion is performed on multimodal features within the occluded area to achieve adaptive generation of boundary compensation features, significantly improve the edge recognition accuracy in the occluded area, and enhance the model's expressive integrity and local consistency.

[0024] 6. This invention uses splicing and fusion of feature maps to reconstruct the entire image information, ensuring the continuity of boundary output. Specifically, the compensated occluded area features and unoccluded area features are uniformly fused and input into the decoder to generate a continuous, smooth landslide edge probability map with complete contour, which significantly improves the spatial consistency of boundary recognition.

[0025] 7. This invention is not only applicable to remote sensing image analysis after a landslide, but can also automatically identify landslide boundaries and quantify landslide range, providing data support for post-disaster relief, disaster assessment, and geological surveys. It is also applicable to early warning scenarios for secondary landslide disasters, especially when there are active faults or potential instability zones at the edge of the initial landslide area. The generated boundary probability map and occlusion risk heat map can be used to determine risk areas where further sliding may occur, assisting decision-makers in conducting ground inspections, equipment deployment, and preventive evacuation. It has significant emergency response value and practical application prospects. Attached Figure Description

[0026] Figure 1 This is a flowchart of a landslide edge recognition method that integrates CNN and ViT according to the present invention. Detailed Implementation

[0027] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] It should be noted that the defects in the solutions in the prior art are all the results of the inventors' practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of this application in the following text should be the inventors' contributions to this application in the process of invention and creation, and should not be understood as technical content known to those skilled in the art.

[0029] In response to the technical problems mentioned in the background art, this invention integrates convolutional neural networks (CNN) and visual transformers (ViT) for landslide edge recognition, aiming to improve the robustness and accuracy of landslide boundary recognition. By combining multi-source feature complementarity and occlusion area recognition mechanisms, it makes full use of information from remote sensing images and digital elevation model (DEM) images.

[0030] In one exemplary embodiment, such as Figure 1 As shown, a landslide edge recognition method integrating CNN and ViT includes the following steps: Remote sensing images and corresponding digital elevation model images of the landslide area are acquired, and resolution registration and geometric alignment are performed. The remote sensing images are input into a convolutional neural network to extract the first feature map reflecting the local details of the edges and textures. Input the digital elevation model image into the visual Transformer to extract the second feature map that reflects the terrain structure relationship; Based on the first feature map, an edge response map is constructed and a fuzzy perception map is calculated to identify weak edge response areas that may be obscured by dust. Based on the second feature map, a structural expectation map is constructed to identify regions in the image that should exist structurally but lack edge detection. The fuzzy perception map and the structural expectation map are fused pixel by pixel to form an occlusion risk heat map; Based on the occlusion risk heatmap, local regions of the first feature map and the second feature map are extracted in the high occlusion area, and fused through a cross-attention mechanism to generate boundary compensation features. The boundary compensation features, together with the first and second feature maps directly spliced ​​in the unobstructed area, form a fused feature map, which is then input into the decoder to generate a landslide edge probability map.

[0031] In practice, the first step is to acquire remote sensing images of the landslide area and corresponding digital elevation model (DEM) images. Remote sensing images, typically taken by satellites or drones, contain rich texture and detail information, such as vegetation cover and ground color variations, which are crucial for identifying landslide boundaries. The DEM images record elevation data of the terrain, reflecting geomorphic features such as slope and aspect, which is extremely helpful in understanding the overall morphology of the landslide.

[0032] To enable these two images to work together, resolution registration and geometric alignment are necessary. Resolution registration involves adjusting the pixel dimensions of the two images to ensure they have the same resolution. For example, if the remote sensing image has a resolution of 0.5 meters per pixel and the DEM image has a resolution of 1 meter per pixel, the DEM image can be upsampled to 0.5 meters per pixel using interpolation methods (such as bilinear interpolation). Geometric alignment requires using geographic coordinate information (such as latitude and longitude) and affine or projective transformations to ensure the two images completely overlap spatially. This alignment operation is typically achieved using GIS software or image processing libraries (such as GDAL).

[0033] The preprocessed remote sensing image is input into a convolutional neural network (CNN) to extract features reflecting local details of edges and textures; this is called the first feature map. CNNs, through multiple layers of convolution and pooling operations, can capture low-level features (such as edges and corners) and mid-to-high-level features (such as texture patterns) in images. For example, ResNet-50 can be chosen as the network architecture, as it includes residual connections, which can effectively alleviate the vanishing gradient problem in deep networks. In this case, the input remote sensing image (assuming a size of 512×512 pixels, 3-channel RGB) after processing by convolutional layers might output a smaller feature map (e.g., 64×64 pixels, 256 channels), which is the first feature map.

[0034] To make feature extraction more suitable for landslide edge recognition tasks, fine-tuning can be performed on a pre-trained ResNet-50 model using a dataset with landslide edge annotations. During training, the cross-entropy loss function is commonly used to optimize network parameters. Data augmentation techniques (such as random cropping, rotation, and brightness adjustment) can be used to increase data diversity, thereby improving the model's generalization ability.

[0035] Meanwhile, the DEM image is input into a visual Transformer model to extract features reflecting terrain structure relationships, known as the second feature map. The visual Transformer, by segmenting the image into small patches (e.g., 16×16 pixels) and utilizing a self-attention mechanism to capture the global relationships between these patches, is well-suited for processing the spatial structure of terrain data. For example, for a 512×512 pixel DEM image, ViT divides it into 1024 patches (each 16×16 pixels), then generates a feature representation containing global information through a multi-head self-attention layer, ultimately outputting a second feature map (e.g., 64×64 pixels, 384 channels).

[0036] In implementation, the standard ViT-B / 16 model (basic version, 16×16 pixel block size) can be selected, as it has a moderate number of parameters and stable performance. To adapt to the landslide recognition task, fine-tuning with labeled data is also required, and the Dice loss function can be selected to better optimize the extraction of terrain boundary-related features.

[0037] Based on the first feature map, an edge response map is further constructed. This step is achieved by inputting the first feature map into an edge prediction sub-network. The edge prediction sub-network can be a simple convolutional layer (e.g., a 3×3 convolutional kernel with one output channel) or an encoder branch of a more complex U-Net structure. The output of the edge prediction sub-network is a probability map consistent with the size of the remote sensing image (e.g., 512×512 pixels), where the value of each pixel is between 0 and 1, representing the prediction confidence that the pixel is a landslide boundary.

[0038] Next, a blurred perception map is calculated to identify weakly responding edge regions that may be obscured by dust. Specifically, this involves analyzing pixel locations with low confidence in the edge response map and combining this with texture information from the remote sensing image and slope information from the DEM image for a comprehensive assessment. For pixels with confidence below a certain preset threshold (e.g., 0.3), their local texture region in the remote sensing image (e.g., a 9×9 pixel window centered on that pixel) is extracted, and the texture gradient change rate of that region is calculated. The texture gradient change rate can be used to calculate edge strength using the Sobel operator, and then the average value is taken. If the change rate is less than a certain threshold (e.g., 0.1), it indicates that the texture in that region is smooth, which may be due to dust obscuration.

[0039] Simultaneously, the slope gradient value corresponding to that pixel in the DEM image is obtained. The slope gradient can be calculated using elevation difference, for example, for a pixel. , They are pixels in x and y The coordinates of the direction and the slope gradient can be approximated by the following formula: in ,in This represents the elevation difference in the x-direction of the DEM image. This represents the elevation difference along the y-direction in the DEM image. This represents the elevation value of the DEM image at pixel location (i,j). This represents the elevation value of the DEM image at pixel location (i+1,j). This represents the elevation value of the DEM image at pixel location (i-1,j). This represents the elevation value of the DEM image at pixel location (i,j+1). This represents the elevation value of the DEM image at pixel location (i,j-1).

[0040] If the slope gradient is greater than a certain threshold (e.g., 5 degrees), it indicates that the terrain in the area has significant changes and there should be edges that may be obscured.

[0041] A pixel is marked as a high-risk area for dust occlusion when it meets all of the following conditions: The edge response confidence level is lower than the first threshold (e.g., 0.3); The texture gradient change rate is less than the second threshold (e.g., 0.1); The slope gradient value is greater than the third threshold (e.g., 5 degrees).

[0042] Based on the locations of all pixels that meet the conditions, a blurred perception map is generated. This is a risk probability map with the same size as the remote sensing image (e.g., 512×512 pixels), where the value of each pixel represents its probability of being obscured by dust. It can simply take the value of 0 (no risk) or 1 (high risk), or it can be normalized to obtain continuous values ​​(between 0 and 1).

[0043] Based on the second feature map, a structural prediction map is constructed to identify regions in the image that should structurally exist but lack edge detection. First, the second feature map is input into a structural attention analysis module. This module, based on ViT's self-attention mechanism, calculates the structural relevance score between each image location and its global context. For example, for each feature vector in the second feature map (corresponding to a location in a 64×64 pixel matrix), its relevance with all other locations is calculated through self-attention, generating a structural relevance map (still 64×64 pixels in size).

[0044] Then, pixel locations with confidence scores higher than a certain fourth threshold (e.g., 0.7) are extracted from the edge response map and defined as high-response edge regions. These regions are typically reliable parts of landslide boundaries. In the structural correlation map, locations that are significantly correlated with multiple high-response edge regions (e.g., correlation scores greater than 0.8) are identified and marked as structurally strongly correlated regions. These regions should be closely related to the edge in terms of topographic structure.

[0045] For pixels in structurally strongly correlated regions, if their confidence score in the edge response map is below a certain fifth threshold (e.g., 0.4), it indicates that these locations should have edges in terms of structure but are actually missing in detection, possibly due to occlusion or texture degradation. Therefore, these are marked as regions with expected structural missingness. Based on the location of these regions, a structural expectation map is constructed, which is a mask map with the same size as the remote sensing image (e.g., 512×512 pixels). In some embodiments, a value of 0 or 1 is selected, representing no missing and missing, respectively. Alternatively, continuous values ​​(between 0 and 1) can be obtained through normalization.

[0046] In the process of generating the occlusion risk heatmap, the fuzzy perception map and the structural expectation map are fused pixel by pixel to generate the occlusion risk heatmap.

[0047] For each pixel position The fusion formula is as follows: ; in, The heatmap showing occlusion risk is located at pixel coordinates. The value, Represents the coordinates of the fuzzy perceptual map at the corresponding pixel position. pixel values, This indicates the coordinates of the expected structure map at the corresponding pixel position. pixel values, and To preset the fusion weight coefficients, satisfying .

[0048] The significance of this formula lies in its ability to jointly model two types of occlusion risks—dust occlusion and structural loss—by weighted summing of the values ​​at each corresponding pixel position in the blurred perception map and the structure prediction map. The formula contains... This reflects the possibility that dust obstruction may lead to a weakening of edge response. This indicates edge regions that should exist but are actually missing from the perspective of global structural relationships. The weighting coefficients α and β are used to adjust their influence on the final occlusion risk assessment according to the actual scene. Through this pixel-by-pixel fusion operation, the risk representation of multi-source information can be effectively superimposed and unified, generating a more discriminative occlusion risk heatmap, providing a reliable basis for subsequent feature enhancement and boundary compensation of high-risk areas.

[0049] Weight and The value of is between 0 and 1, and can usually be set to ( ). =0.6), ( =0.4), to slightly emphasize the local information of the blurred perception map. The specific value can be determined through cross-validation, for example, by testing different combinations on the validation set, such as... =0.5, =0.5 or =0.7, =0.3, selecting the combination that maximizes edge detection accuracy. In the generated occlusion risk heatmap, the value of each pixel represents the probability that it is in a dust-occluded area, typically between 0 and 1.

[0050] In the generation of boundary compensation features, based on the occlusion risk heatmap, pixel locations with occlusion probability values ​​greater than a certain threshold (e.g., 0.6) are selected as high-occlusion regions. For these regions, corresponding local feature sub-regions (e.g., 9×9 pixel feature blocks) are extracted from the first and second feature maps. Then, they are fused using a cross-attention mechanism.

[0051] Specifically, local feature sub-regions of the first feature map are used as query vectors, and local feature sub-regions of the second feature map are used as key and value vectors, respectively, and input into a cross-attention fusion module. This module calculates the similarity between the query and the key (e.g., through dot product and normalization) to obtain attention weights, and then performs a weighted summation of the value vectors to generate the fused boundary compensation features. The advantage of this mechanism is that it can adaptively enhance the occluded edge information in the first feature map while introducing terrain structure information from the second feature map, achieving feature complementarity.

[0052] For example, suppose the local features of the first feature map are a 9×9×256 tensor and the local features of the second feature map are a 9×9×384 tensor. After cross-attention, a 9×9×256 boundary compensation feature is output to replace the original feature representation of the highly occluded region.

[0053] In the generation of the fused feature map and edge probability map, the boundary compensation features are mapped back to their corresponding positions in the original remote sensing image space, replacing the feature representation of highly occluded areas. For unoccluded areas (i.e., areas with a probability below the threshold in the occlusion risk heatmap), the first and second feature maps are concatenated along the channel dimension. For example, if the first feature map is 512×512×256 and the second feature map is 512×512×384, the concatenated feature map will be 512×512×640, which will be used as the fused feature for the unoccluded area.

[0054] Then, the boundary compensation features and the fused features of the unoccluded area are concatenated spatially to form a complete fused feature map (still 512×512×640 pixels). This feature map is input into a decoder network to generate a landslide edge probability map. The decoder can use the decoding branch of U-Net, and through upsampling and convolution operations, restore the feature map to the original image size (512×512 pixels), outputting a single-channel probability map, where each pixel value represents its confidence level (between 0 and 1) in belonging to the landslide boundary.

[0055] Threshold segmentation is performed on the landslide edge probability map, marking pixels with probability values ​​higher than a certain edge confidence threshold (e.g., 0.5) as edge regions, generating a binary mask image (values ​​of 0 or 1). This threshold can be adjusted according to the specific task; for example, it can be set to 0.7 in scenarios requiring higher precision.

[0056] Then, morphological post-processing operations are applied, including erosion, dilation, opening, and closing operations. For example, an opening operation (erosion followed by dilation) is first performed using a 3×3 structuring element to remove isolated noise points; then a closing operation (dilation followed by erosion) is performed to fill boundary breaks. These operations can be implemented using the OpenCV library, and specific parameters (such as the structuring element size) can be adjusted according to the image resolution and noise level.

[0057] Finally, the morphologically processed binary mask image is output as a landslide boundary map, used to represent the boundary distribution of landslide areas. For example, the result can be visualized as a red outline on a remote sensing image for easy user understanding.

[0058] In practical applications, all thresholds can be adjusted according to the characteristics of the dataset. For example, for landslide data in mountainous areas with significant dust obscuring the data, the first threshold can be lowered to 0.2 to improve sensitivity; for flat terrain areas, the third threshold can be raised to 10 degrees to reduce false positives.

[0059] For model training, some embodiments may use landslide datasets with edge annotations, such as real data obtained from geological survey agencies. During training, the loss function may combine cross-entropy loss and Dice loss. This combination can balance the accuracy and completeness of edge regions.

[0060] To illustrate with an example, suppose we are processing a 512×512 pixel remote sensing image of a landslide, which contains areas partially obscured by dust. After the steps described above, the first feature map extracts blurred edge information, the second feature map captures changes in terrain slope, and the blurred perception map and the structure prediction map identify obscured and missing areas, respectively. The resulting landslide boundary map accurately delineates the obscured edges, improving recognition accuracy compared to a single CNN method.

[0061] This invention, in the process of landslide edge identification, constructs a fuzzy perception map and a structural expectation map, and fuses them to generate an occlusion risk heat map. This not only improves the accuracy of edge extraction under occlusion conditions but also indirectly reveals potential "residual unstable body" regions in the image. These regions typically exhibit edges that should exist structurally but lack texture response, and show significant abrupt changes in terrain slope, making them highly likely to be dangerous blocks that have not yet fully slid, and important potential sources of secondary landslides. Therefore, the structural map and risk map generated by this invention can be used to assist in identifying such high-risk areas, providing important judgment criteria for subsequent disaster monitoring.

[0062] Building upon this foundation, the present invention can further integrate external triggering factors such as terrain slope, rainfall intensity, and lithological conditions to expand and construct a landslide trend simulation module, enabling prediction and early warning of potential landslide development paths and landslide areas. Based on the distribution characteristics of risk indicators in the fused image, a three-tiered warning system (red, yellow, and blue) is established. This helps emergency management departments to promptly implement response measures such as population evacuation, traffic control, deployment of monitoring equipment, and allocation of rescue resources, thereby achieving a closed-loop linkage from image recognition to disaster response and significantly improving the comprehensive prevention and control capabilities for landslide disasters.

[0063] In summary, the method of this invention, by fusing multi-source features from CNN and ViT and combining occlusion recognition and boundary compensation mechanisms, not only overcomes problems such as dust occlusion and texture degradation, but also significantly improves the robustness and spatial consistency of landslide boundary recognition, thus possessing high practical value.

[0064] In another exemplary embodiment, a landslide edge recognition system integrating CNN and ViT is provided, comprising: The input and preprocessing module is used to acquire remote sensing images of the landslide area and the corresponding digital elevation model images, and to perform resolution registration and geometric alignment. The feature extraction module is used to input remote sensing images into a convolutional neural network to extract a first feature map that reflects the local details of edges and textures; and to input digital elevation model images into a visual Transformer to extract a second feature map that reflects the topographic structure relationships. The fuzzy perception map generation module is used to construct an edge response map and calculate a fuzzy perception map based on the first feature map, which is used to identify weak response areas at the edges that may be obscured by dust. The structure prediction map generation module is used to construct a structure prediction map based on the second feature map, which is used to identify regions in the image that should exist structurally but whose edge detection is missing; The occlusion risk heat map generation module is used to fuse the fuzzy perception map and the structural expectation map pixel by pixel to form an occlusion risk heat map. The boundary compensation feature generation module is used to extract local regions of the first feature map and the second feature map in the high occlusion area based on the occlusion risk heat map, and fuse them through a cross-attention mechanism to generate boundary compensation features. The landslide edge probability map generation module is used to combine the boundary compensation features with the first feature map and the second feature map directly spliced ​​in the unobstructed area to form a fused feature map, which is then input into the decoder to generate the landslide edge probability map.

[0065] For example, the system also includes: The landslide boundary map generation module is used to perform thresholding and morphological post-processing operations on the landslide edge probability map and output the final landslide boundary map.

[0066] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.

Claims

1. A landslide edge recognition method integrating CNN and ViT, characterized in that, include: Acquire remote sensing images and corresponding digital elevation model images of the landslide area, and perform resolution registration and geometric alignment; The remote sensing image is input into a convolutional neural network to extract the first feature map that reflects the local details of edges and textures. Input the digital elevation model image into the visual Transformer to extract the second feature map that reflects the terrain structure relationship; Based on the first feature map, an edge response map is constructed and a fuzzy perception map is calculated to identify weak edge response areas that may be obscured by dust. Based on the second feature map, a structural expectation map is constructed to identify regions in the image that should exist structurally but lack edge detection. The fuzzy perception map and the structural expectation map are fused pixel by pixel to form an occlusion risk heat map; Based on the occlusion risk heatmap, local regions of the first feature map and the second feature map are extracted in the high occlusion area, and fused through a cross-attention mechanism to generate boundary compensation features. The boundary compensation features, together with the first and second feature maps directly spliced ​​in the unobstructed area, form a fused feature map, which is then input into the decoder to generate a landslide edge probability map.

2. The landslide edge recognition method integrating CNN and ViT according to claim 1, characterized in that, The method further includes: Thresholding and morphological post-processing operations are performed on the landslide edge probability map to output the final landslide boundary map.

3. The landslide edge recognition method integrating CNN and ViT according to claim 1, characterized in that, The steps for constructing the edge response map and calculating the fuzzy perception map include: The first feature map is input into the edge prediction sub-network to generate an edge response map. The edge response map is a probability map with the same size as the remote sensing image, where the value of each pixel position represents the prediction confidence that the position is the edge of a landslide. For pixel locations in the edge response map where the confidence level is lower than a preset edge response threshold, the corresponding local texture region in the remote sensing image is extracted, and the texture gradient change rate of the local texture region is calculated. Further obtain the slope gradient value in the digital elevation model image corresponding to the pixel position; When a pixel location meets the following occlusion conditions simultaneously, that pixel location is marked as a high-risk area for dust occlusion: Its edge response confidence is lower than the first threshold, its texture gradient change rate is less than the second threshold, and its slope gradient value is greater than the third threshold. A blurred perception map is generated based on the positions of all pixels that meet the occlusion conditions. The blurred perception map is a risk probability map with the same size as the remote sensing image, used to represent the distribution of areas where dust occlusion causes weak edge response.

4. The landslide edge recognition method integrating CNN and ViT according to claim 3, characterized in that, The steps to construct the expected structure diagram include: The second feature map is input into the structural attention analysis module, which calculates the structural relevance score between each image location and its global context region based on the self-attention mechanism in the visual Transformer, and generates a structural relevance map. Pixel locations with edge response confidence scores higher than a fourth threshold are extracted from the edge response map and defined as high-response edge regions. In the structural correlation map, image locations that have significant structural correlation with multiple high-response edge regions are identified as structurally strongly correlated regions. If the confidence level of a pixel location in the strongly correlated region of the structure is lower than the fifth threshold in the edge response map, it is marked as a region where the structure is expected to be missing. A structure prediction map is constructed based on the locations of all expected missing structural regions. The structure prediction map is a mask map with the same size as the original remote sensing image.

5. The landslide edge recognition method integrating CNN and ViT according to claim 4, characterized in that, The steps of fusing the fuzzy perception map and the structural expectation map pixel by pixel to form an occlusion risk heatmap include: The values ​​of corresponding pixel positions in the fuzzy perception map and the structure prediction map are weighted and summed to generate an occlusion risk heatmap, where the occlusion risk value of each pixel is determined by the following formula: ； in, The heatmap showing occlusion risk is located at pixel coordinates. The value, Represents the coordinates of the fuzzy perceptual map at the corresponding pixel position. pixel values, This indicates the coordinates of the expected structure map at the corresponding pixel position. pixel values, and To preset the fusion weight coefficients, satisfying ; The occlusion risk heatmap is used to represent the probability value of each pixel being in a dust-occluded area.

6. The landslide edge recognition method integrating CNN and ViT according to claim 5, characterized in that, The step of generating boundary compensation features includes: In the occlusion risk heatmap, pixel locations with occlusion probability values ​​greater than a preset threshold are selected as high occlusion areas. Extract local feature sub-regions corresponding to the highly occluded regions from the first feature map and the second feature map, respectively; The local features in the corresponding local feature sub-region of the first feature map are used as query vectors, and the local features in the corresponding local feature sub-region of the second feature map are used as key vectors, which are then input into the cross-attention fusion module. In the cross-attention fusion module, attention weights are calculated based on the similarity between the query vector and the key vector, and the value vectors are weighted and summed to generate the fused boundary compensation features. The boundary compensation feature is used to replace the feature representation in the original region.

7. A landslide edge recognition method integrating CNN and ViT according to claim 6, characterized in that, The steps to generate a landslide edge probability map include: The boundary compensation features are mapped back to their corresponding positions in the original remote sensing image space to replace the feature representations corresponding to the highly occluded areas in the occlusion risk heatmap. The features of the locations not covered by the occluded risk area in the first feature map and the second feature map are concatenated along the channel dimension to form the fused features of the unoccluded area. The boundary compensation feature and the unoccluded area fusion feature are spliced ​​together according to their spatial positions to form a fusion feature map; The fused feature map is input into the decoder network to generate a landslide edge probability map with the same size as the original remote sensing image. Each pixel value of the probability map represents the confidence level that it belongs to the landslide edge.

8. A landslide edge recognition method integrating CNN and ViT according to claim 2, characterized in that, The steps to output a landslide boundary map include: The landslide edge probability map is subjected to threshold segmentation processing, and pixels with probability values ​​higher than a preset edge confidence threshold are marked as edge regions to generate a binary edge mask map. Morphological post-processing operations, including at least one of erosion, dilation, opening and closing operations, are applied to the edge mask image to eliminate isolated noise, fill boundary breaks, or smooth edge contours. The morphologically processed results are output as a landslide boundary map, which is used to represent the final boundary distribution of the landslide area.

9. A landslide edge recognition system integrating CNN and ViT, characterized in that, include: The input and preprocessing module is used to acquire remote sensing images of the landslide area and the corresponding digital elevation model images, and to perform resolution registration and geometric alignment. The feature extraction module is used to input remote sensing images into a convolutional neural network to extract a first feature map that reflects the local details of edges and textures; and to input digital elevation model images into a visual Transformer to extract a second feature map that reflects the topographic structure relationships. The fuzzy perception map generation module is used to construct an edge response map and calculate a fuzzy perception map based on the first feature map, which is used to identify weak response areas at the edges that may be obscured by dust. The structure prediction map generation module is used to construct a structure prediction map based on the second feature map, which is used to identify regions in the image that should exist structurally but whose edge detection is missing; The occlusion risk heat map generation module is used to fuse the fuzzy perception map and the structural expectation map pixel by pixel to form an occlusion risk heat map. The boundary compensation feature generation module is used to extract local regions of the first feature map and the second feature map in the high occlusion area based on the occlusion risk heat map, and fuse them through a cross-attention mechanism to generate boundary compensation features. The landslide edge probability map generation module is used to combine the boundary compensation features with the first feature map and the second feature map directly spliced ​​in the unobstructed area to form a fused feature map, which is then input into the decoder to generate the landslide edge probability map.

10. A landslide edge recognition system integrating CNN and ViT according to claim 9, characterized in that, Also includes: The landslide boundary map generation module is used to perform thresholding and morphological post-processing operations on the landslide edge probability map and output the final landslide boundary map.

Images