A remote sensing landslide and surrounding building detection method and system based on ViTAEv2-S model

Abstract

The present invention discloses a remote sensing landslide and surrounding building detection method and system based on the ViTAEv2-S model, belonging to the field of image processing. The method comprises: constructing a detection model: using a lightweight segmentation framework, including an encoder and a decoder, the encoder using the ViTAEv2-S backbone network, combining multi-scale dilated convolution with a self-attention mechanism to extract multi-scale features; the decoder simplifying the U-Net structure, removing the pyramid pooling module, adding a spatial attention module to focus on the landslide and building areas, and adding a channel attention module to enhance the feature channels corresponding to the landslide and building; training the detection model; and using the trained detection model to synchronously segment the landslide and the building. The present invention proposes a lightweight improvement scheme for the segmentation framework, improves semantic segmentation accuracy, significantly reduces computing resource consumption, enhances small sample training capabilities, and achieves synchronous detection of landslides and buildings through multi-task learning, suitable for real-time disaster monitoring scenarios.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and disaster monitoring, and in particular to a remote sensing landslide and surrounding building detection method and system based on the ViTAEv2-S model. Background Art

[0002] With the rapid development of remote sensing technology, landslide detection has become a critical task in geological hazard monitoring. Compared to other data sources, optical remote sensing offers sub-meter resolution, providing more detailed surface information. Research has shown that the reliability of detection results is highly dependent on the quality of the dataset and the factors considered. Furthermore, high-resolution satellite data can provide daily or even hourly updates, providing crucial support for real-time monitoring and emergency response, such as disasters. Consequently, optical remote sensing imagery has gained widespread application in geological exploration, environmental protection, and earthquake hazard monitoring.

[0003] Currently, remote sensing landslide detection primarily relies on semantic segmentation techniques, which typically employ convolutional neural networks (CNNs) or Transformer architectures for image segmentation. These techniques employ multi-layer feature extraction and pixel-by-pixel classification to detect landslides and buildings in remote sensing images. These models employ convolutional layers or self-attention mechanisms during feature extraction. Convolutional neural networks use convolution operations to extract low-level and high-level features from images, restore resolution through pooling and upsampling layers, and finally perform pixel-level classification. Compared to CNNs, the Transformer architecture effectively models long-range contextual relationships and utilizes self-attention mechanisms to process global information in remote sensing images.

[0004] The main method of semantic segmentation technology includes the following steps:

[0005] 1. Data preprocessing: Perform preprocessing operations such as cropping and normalization on remote sensing images to ensure that the input data is suitable for model training.

[0006] 2. Feature extraction: Use convolutional layers or Transformer layers to extract image features and construct semantic representations of landslides and buildings.

[0007] 3. Segmentation decoding: The decoder maps the extracted features back to the original image resolution to generate a segmented image.

[0008] 4. Loss calculation: Evaluate the difference between the model output and the true label through common loss functions (such as cross entropy loss, Dice coefficient loss, etc.), and optimize the model parameters.

[0009] 5. Inference and post-processing: After training, the model is used for inference to generate the final segmentation results, which can be optimized through post-processing methods such as conditional random fields (CRF).

[0010] Existing semantic segmentation technology has the following defects:

[0011] 1. High computing resource consumption: Traditional CNN or Transformer models have large number of parameters, making them difficult to deploy on low-computing devices and unsuitable for remote sensing detection scenarios with limited computing resources.

[0012] 2. High data dependency: Large-scale labeled data is required, and remote sensing data labeling is costly;

[0013] 3. Poor multi-task performance: Unbalanced training for landslide and building detection tasks leads to insufficient model stability.

[0014] 4. Conflict between efficiency and accuracy: Lightweight models have low accuracy, high-precision models cannot run in real time, and the inference speed is not fast enough, which does not meet the needs of disaster monitoring and emergency response scenarios that require rapid response. Summary of the Invention

[0015] The purpose of the present invention is to overcome the technical problems existing in the prior art, and provide a remote sensing landslide and surrounding building detection method and system based on the ViTAEv2-S model, propose a lightweight improvement scheme for the segmentation framework, and improve the semantic segmentation accuracy.

[0016] The object of the present invention is achieved through the following technical solutions:

[0017] First, a remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model is provided, comprising:

[0018] Detection model construction: A lightweight segmentation framework is used, including an encoder and decoder. The encoder uses the ViTAEv2-S backbone network, combining multi-scale dilated convolution with a self-attention mechanism to extract multi-scale features. The decoder simplifies the U-Net structure, removing the UPerNet pyramid pooling module, adding a spatial attention module to focus on landslides and buildings, and a channel attention module to enhance the feature channels corresponding to landslides and buildings.

[0019] Training the detection model;

[0020] The trained detection model is used to simultaneously segment landslides and buildings.

[0021] In some embodiments, the spatial attention module includes a 3×3 convolutional layer and a sigmoid activation function, and the channel attention module includes a global average pooling layer and a fully connected layer.

[0022] In some embodiments, the training of the detection model includes:

[0023] The detection model is trained based on self-supervised pre-training and multi-task learning strategies.

[0024] In some embodiments, the multi-task learning strategy includes:

[0025] The landslide and building segmentation tasks are integrated into a multi-task model with a shared backbone network.

[0026] In some embodiments, the loss functions used by the multi-task model include Dice loss and cross entropy loss, and a GradNorm loss weight adaptation method and an alternating training strategy are adopted.

[0027] In some embodiments, it further includes:

[0028] Construct remote sensing landslide dataset and surrounding building dataset;

[0029] The images in the remote sensing landslide dataset and the surrounding building dataset were preprocessed respectively.

[0030] In some embodiments, before training the detection model, the method further includes:

[0031] Normalize the input image and convert it to the detection model input format.

[0032] In some embodiments, the synchronous segmentation of landslides and buildings using the trained detection model includes:

[0033] The generated probability map is thresholded and segmented, and the landslide or building binary mask is output after threshold segmentation.

[0034] Secondly, a remote sensing landslide and surrounding building detection system based on the ViTAEv2-S model is provided, including:

[0035] A detection model construction module is used to build a detection model using a lightweight segmentation framework, including an encoder and a decoder. The encoder uses the ViTAEv2-S backbone network, combining multi-scale dilated convolution with a self-attention mechanism to extract multi-scale features. The decoder simplifies the U-Net structure, removing the UPerNet pyramid pooling module, adding a spatial attention module to focus on landslides and buildings, and adding a channel attention module to enhance the feature channels corresponding to landslides and buildings.

[0036] A training module, used for training the detection model;

[0037] The detection module is used to synchronously segment landslides and buildings using the trained detection model.

[0038] It should be further explained that the technical features corresponding to the above embodiments can be combined or replaced with each other to form a new technical solution if there is no conflict.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. Lightweight model

[0041] (1) Reduced parameter and computational complexity: By introducing a simplified U-Net framework and combining spatial and channel attention mechanisms, the model's parameter and computational complexity are reduced by 52% and 80%, respectively. This enables the present invention to run on resource-constrained devices (such as mobile terminals and embedded platforms) without sacrificing segmentation accuracy. (2) Reduced memory usage: Compared to the original U-Net segmentation framework, the present invention significantly reduces memory usage and is suitable for embedded platforms with smaller memory, reducing storage space requirements. This enables highly efficient semantic segmentation of landslides and buildings on platforms with limited computing power and storage capacity.

[0042] 2. Higher inference efficiency

[0043] (1) Improved inference speed: The optimized model not only reduces computing resource consumption but also increases inference speed. This is particularly important for disaster monitoring and emergency response applications that require rapid response. (2) Strong adaptability: The present invention can operate efficiently in environments with limited computing resources and storage space, solving the problem of existing high-precision models being unable to run effectively on devices with limited computing and storage capabilities.

[0044] 3. Reduce dependence on large amounts of labeled data and speed up the training cycle

[0045] By applying self-supervised pre-training (such as MAE and SparK methods), effective training can be performed on small-scale unlabeled datasets. This is particularly important for scenarios where labeling is difficult and expensive in remote sensing data, helping to reduce data labeling costs and improve the generalization ability of the model.

[0046] 4. Improve downstream task performance

[0047] Experimental results show that the F1 score of the model based on self-supervised pre-training in landslide and building detection tasks increased by 3.15 percentage points (on the landslide dataset) and 2.87 percentage points (on the building dataset), respectively, demonstrating the advantage of the pre-training method in accuracy.

[0048] 5. Improve multi-task learning capabilities

[0049] (1) Synchronous segmentation of landslides and buildings: This paper integrates the landslide and building segmentation tasks into a multi-task model. By sharing the backbone network, the model's computational and storage requirements are reduced, while improving the model's operational efficiency and practicality. (2) Solving the overfitting problem: By using the GradNorm method and an alternating training strategy, the problem of the landslide task affecting the accuracy of the building task due to rapid convergence is alleviated, ensuring that the segmentation accuracy of landslides and buildings is balanced and avoiding interference between tasks during separate training. This achieves synchronous detection of landslides and buildings, improving detection efficiency.

[0050] 6. Real-time and applicable

[0051] The optimization presented in this paper enables semantic segmentation tasks to be run in real time on platforms with limited computing resources, making it suitable for practical applications, particularly for real-time monitoring and early warning of landslide disasters. This feature allows the technology to be better applied in real-world disaster prevention and control efforts, with significant social and economic value. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of a remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model of the present invention;

[0053] Figure 2 Schematic diagram of the segmentation framework of the detection model of the present invention;

[0054] Figure 3 Activate heatmap for the present invention category;

[0055] Figure 4 This is a comparison chart of the loss changes of the four model structures of the present invention during the pre-training stage. DETAILED DESCRIPTION

[0056] The technical solutions of the present invention are described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings herein can be arranged and designed in various different configurations. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] It should be noted that the defects existing in the solutions in the above-mentioned prior art are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above-mentioned problems and the solutions proposed in the embodiments of this application below for the above-mentioned problems should be the contributions made by the inventor to this application in the process of invention and creation, and should not be understood as technical contents known to technical personnel in this field.

[0058] In response to the technical problems pointed out in the background technology, the embodiments provided by the present invention are as follows: In response to the technical problems pointed out in the background technology, the embodiments provided by the present invention are as follows:

[0059] Reference Figure 1 In an exemplary embodiment, a remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model includes:

[0060] Detection model construction: A lightweight segmentation framework is used, including an encoder and decoder. The encoder uses the ViTAEv2-S backbone network, combining multi-scale dilated convolution with a self-attention mechanism to extract multi-scale features. The decoder simplifies the U-Net structure, removing the UPerNet pyramid pooling module, adding a spatial attention module to focus on landslides and buildings, and a channel attention module to enhance the feature channels corresponding to landslides and buildings.

[0061] Training the detection model;

[0062] The trained detection model is used to simultaneously segment landslides and buildings.

[0063] Among them, the detection model proposes a lightweight segmentation framework based on the ViTAEv2-S backbone network, and adds spatial attention and channel attention mechanisms on the basis of simplifying the U-Net decoder to suppress spatial interference information and channel interference information in the feature map respectively. Figure 2 As shown, the model inputs a remote sensing image (RGB or multispectral), with an example resolution of 224×224 pixels. The encoder uses the ViTAEv2-S backbone network, with four stages of progressive downsampling (resolution 56×56 → 28×28 → 14×14 → 7×7). Each stage contains a reduction cell (multi-scale dilated convolution) and a normal cell (convolution and self-attention branches in parallel), outputting multi-scale feature maps. The decoder uses a simplified U-Net architecture, consisting of upsampling layers and cross-layer connections. The upsampling layers gradually restore the resolution to the input size, and the cross-layer connections fuse the features of the corresponding encoder stages. A spatial attention module (SAM) is added after the output to suppress spatial noise, and a channel attention module (CAM) is added before the prediction head to enhance the weights of key features.

[0064] This segmentation framework removes the pyramid pooling module from the UPerNet segmentation framework to reduce computational complexity. The spatial attention module includes a 3×3 convolutional layer with a sigmoid activation function, while the channel attention module includes a global average pooling layer and a fully connected layer. SAM (3×3 convolution + sigmoid) focuses on landslides and building areas, while CAM (global average pooling + MLP) suppresses invalid channels. This dual spatial and channel attention mechanism addresses the issues of "ambiguous target region positioning" and "redundant feature channels" in remote sensing imagery. In the lightweight ViTAEv2-S framework, their combined application achieves the following:

[0065] 1. Spatial dimension: Accurately locate landslide boundaries and building outlines, and suppress background noise;

[0066] 2. Channel dimension: Screen key spectral features to enhance the distinction between target and background;

[0067] 3. Efficiency: Improve accuracy at extremely low computing costs, suitable for remote sensing detection scenarios with limited computing resources.

[0068] The datasets used in this invention are as follows:

[0069] (1) Remote sensing landslide dataset

[0070] Data source: Remote sensing RGB images of a city, including 770 landslide samples and 2003 negative samples (no landslide), with a resolution of 0.8m.

[0071] Preprocessing: 600 negative samples and 770 positive samples were randomly selected to form 1370 samples, which were divided into training set (961 samples) and validation set (409 samples) in a 7:3 ratio.

[0072] Data augmentation: random scaling (maintaining aspect ratio), random cropping, horizontal flipping, photometric distortion (brightness / contrast adjustment).

[0073] Input size: 224×224 pixels.

[0074] (2) Samples_BuiUrbanVill building dataset

[0075] Data source: Remote sensing RGB image of an urban village in a city in northern China, with a resolution of 0.11m and a total of 2328 samples (512×512 pixels).

[0076] Preprocessing: Images with large blank areas were removed, 1370 samples were randomly selected, and the training set (961 images) and the validation set (409 images) were divided into 7:3.

[0077] Label processing: Convert instance segmentation labels into semantic segmentation labels.

[0078] Data augmentation: Similar to the landslide dataset, but without photometric distortion (avoiding color interference).

[0079] (3) Landslide4Sense multispectral landslide dataset

[0080] Data source: Multispectral images (Sentinel-2 band + B13 slope + B14DEM) from 4 regions around the world, with a total of 3799 samples (128×128 pixels).

[0081] Preprocessing: Only the training set is used, and the training set (2661 images) and the validation set (1138 images) are divided into a 7:3 ratio.

[0082] Input features: 14 bands (including multispectral, slope, and DEM). Data augmentation is random upside-down flipping (no photometric operation).

[0083] Furthermore, the detection model is trained based on self-supervised pre-training and multi-task learning strategies. Specifically, a pre-training model structure and corresponding training settings are proposed based on the ViTAEv2-S backbone network. Experiments are conducted on small-scale unlabeled remote sensing imagery, using the SparK self-supervised method with the backbone network unchanged.

[0084] For the above dataset, the landslide and building segmentation tasks are integrated into a multi-task model based on the ViTAEv2-S backbone network, and a multi-task model training strategy is proposed. The landslide branch mainly improves the accuracy of each task in multi-task training by sharing auxiliary training heads and other network layers in the backbone network except the BN layer, and uses Dice loss in the landslide branch. The building branch uses the GradNorm loss weight adaptation method and the proposed alternating training strategy to further improve the accuracy of each task and alleviate the problem of the rapid convergence of the landslide task leading to a large drop in accuracy during the subsequent convergence of the building task. Finally, experiments were conducted on datasets of varying sizes. The experimental results show that the proposed multi-task training strategy is still effective. By sharing the backbone network, the multi-task model can significantly reduce the amount of computation, the number of parameters, and the memory consumption, and accelerate its operation speed, especially the operation speed of Pytorch weights on GPUs.

[0085] Before training the detection model, the input image is normalized and converted into the detection model input format. The specific training process is as follows:

[0086] 1. Initialization and parameter setting

[0087] Pre-trained weights: Use the ImageNet-1K pre-trained ViTAEv2-S weights (official) and convert them to the MMSegmentation framework.

[0088] Optimizer: AdamW, initial learning rate 0.0006, weight decay 0.01, β1=0.9, β2=0.999.

[0089] Learning rate schedule:

[0090] Warm-up phase: LinearLR is used for the first 500 iterations, and the learning rate is linearly increased from 0.0006×0.005 to 0.0006.

[0091] Attenuation stage: PolyLR is used later, and the learning rate decays to 0 according to the polynomial, with the decay exponent power=1.

[0092] Batch size: 48 (GPU: RTX 2080Ti / RTX A4000).

[0093] Loss function: Dice loss + cross entropy loss (CE).

[0094] 2. Training details and improvement points

[0095] Lightweighting improves efficiency: Compared to the original UPerNet, the number of parameters has been reduced from 48.43M to 23.27M (down 52%), and the computational overhead has been reduced from 45.60GB to 8.92GB (down 80%). GPU training memory consumption has been reduced from approximately 12GB to 8GB, and inference speed has been increased from 32 images / s to 43 images / s.

[0096] The role of the attention mechanism: SAM suppresses background noise at the shallow level of the encoder, such as vegetation, roads and other non-landslide areas. CAM enhances the feature channels corresponding to landslides / buildings (for example, the near-infrared band is sensitive to landslides) before the decoder prediction head to improve category differentiation. Figure 3 As shown in Figure 1, the heat map is generated by Grad-CAM, and the target feature layer is the SiLU layer in the last convolution branch in the last stage of the backbone network.

[0097] Multi-scale test (MS): During the test, the input size was adjusted to 256×256, and 0.5 / 0.75 / 1.0 times scaling and horizontal flipping were performed. The multi-scale prediction results were integrated, and the F1 score increased by 0.37% (remote sensing landslide dataset).

[0098] The following is a comparison of several pre-training model structures:

[0099] like Figure 4As shown in the figure, the loss changes of four model structures in the pre-training stage are compared. Among them, the pre-training structure 1 is the Swin U-Net decoder using the SparK self-supervision method, the pre-training structure 1 (+SE) is the pre-training structure 1 that introduces the SE attention module into its backbone network, the pre-training structure 2 is the classic U-Net decoder using the SparK self-supervision method, and the pre-training structure 3 is a pure Transformer architecture based on the Swin MAE self-supervision method. Figure 4 In the figure, the horizontal axis represents the number of iterations (iters), indicating the number of updates in model training, with a total of 50,000 iterations. The vertical axis represents the self-supervised loss, which measures the error in the model's reconstruction of features in unobstructed areas. Lower values ​​indicate higher reconstruction accuracy. As can be seen from the figure, pre-trained structure 1 has the fastest training loss decrease and the lowest final loss, indicating the highest self-supervised learning efficiency and the strongest ability to reconstruct features from remote sensing imagery. The loss of pre-trained structure 1 (+SE) is significantly higher than that of pre-trained structure 1, due to the increased model complexity of the SE module, which reduces training stability. The loss of pre-trained structure 2 decreases slightly faster and reaches a higher final value than that of structure 1, due to the increased computational complexity of the concatenation operation in the U-Net decoder, resulting in slightly lower convergence efficiency. The loss of pre-trained structure 3 decreases more slowly and reaches a higher final value, due to the lack of compatibility of the pure Transformer architecture with convolutional features, resulting in poor feature reconstruction in obscured areas.

[0100] Key conclusion: Spark-based pre-training architecture 1 performs best in self-supervised pre-training. Adding the SE module actually degrades performance, validating the conclusion that simple architectures are more suitable for small-sample pre-training. Tables 1-5 present the pre-training results for each model architecture. IoU stands for Intersection over Union (IoU), which measures the difference between the predicted and ground-truth bounding boxes in segmentation tasks.

[0101]

[0102]

[0103]

[0104]

[0105]

[0106] As can be seen, the proposed model performs well, significantly reducing the number of parameters and computational overhead while also achieving higher semantic segmentation accuracy than existing models. The SparK self-supervised approach combined with the SwinU-Net decoder pre-training model is optimal. While its improved accuracy on downstream tasks is somewhat lower than fully supervised pre-training using the large-scale ImageNet-1K dataset, it is effective and significantly reduces storage and time costs.

[0107] Furthermore, the trained detection model is used to simultaneously segment the landslide and the building. The specific detection process and experimental results are as follows:

[0108] 1. Reasoning Process

[0109] Input preprocessing: single-band / multi-band image normalization (mean / variance standardization), conversion to model input format (such as RGB or multispectral tensor).

[0110] Forward propagation: The encoder extracts multi-scale features, and the decoder generates pixel-by-pixel classification probabilities through upsampling and attention modules.

[0111] Post-processing: Threshold segmentation of the probability map (e.g. 0.5) to generate a binary mask (landscape / building is 1, background is 0).

[0112] Optional operation: Morphological filtering (dilation / erosion) to optimize the boundary.

[0113] 2. Experimental Results

[0114] Accuracy comparison (see Tables 6 and 7): The improved model achieved an F1 score of 86.9% on the remote sensing landslide dataset, outperforming Swin-T-UPerNet (86.72%) and ViTAEv2-S-UPerNet (86.14%). On the building dataset, it achieved an F1 score of 92.47% and an IoU of 85.99%, surpassing U-Net (87.77% F1) and ConvNeXt-S (90.91% F1). This demonstrates the effectiveness of the proposed model, significantly reducing the number of parameters and computational overhead while also achieving higher semantic segmentation accuracy than other existing model architectures. The SparK self-supervised approach combined with the Swin U-Net decoder pre-trained model achieved optimal performance. While its improved accuracy on downstream tasks was somewhat lower than fully supervised pre-training using the large-scale ImageNet-1K dataset, it was effective and significantly reduced storage and time costs.

[0115]

[0116]

[0117] In another exemplary embodiment, based on the same inventive concept as the method embodiment, a remote sensing landslide and surrounding building detection system based on the ViTAEv2-S model is provided, comprising:

[0118] A detection model construction module is used to build a detection model using a lightweight segmentation framework, including an encoder and a decoder. The encoder uses the ViTAEv2-S backbone network, combining multi-scale dilated convolution with a self-attention mechanism to extract multi-scale features. The decoder simplifies the U-Net structure, removing the UPerNet pyramid pooling module, adding a spatial attention module to focus on landslides and buildings, and adding a channel attention module to enhance the feature channels corresponding to landslides and buildings.

[0119] A training module, used for training the detection model;

[0120] The detection module is used to synchronously segment landslides and buildings using the trained detection model.

[0121] The above specific implementation methods are detailed descriptions of the present invention. It cannot be considered that the specific implementation methods of the present invention are limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, they can make several simple deductions and substitutions without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model, characterized in that: include: Detection model construction: A lightweight segmentation framework is used, including an encoder and decoder. The encoder uses the ViTAEv2-S backbone network, combining multi-scale dilated convolution with a self-attention mechanism to extract multi-scale features. The decoder simplifies the U-Net structure, removing the UPerNet pyramid pooling module, adding a spatial attention module to focus on landslides and buildings, and a channel attention module to enhance the feature channels corresponding to landslides and buildings. The encoder includes four progressive downsampling stages, each of which includes multi-scale dilated convolution and self-attention mechanisms. The decoder includes upsampling layers and cross-layer connections. The upsampling layers gradually restore the resolution to the input size. The cross-layer connections fuse the features of the corresponding encoder stages. A spatial attention module is added after the output to suppress spatial noise, and a channel attention module is added before the prediction head to enhance the weight of key features. The spatial attention module includes a 3×3 convolutional layer and a sigmoid activation function, and the channel attention module includes a global average pooling layer and a fully connected layer. The output of the first stage of the encoder is processed by a spatial attention module, and the output of the second stage is processed by another spatial attention module after upsampling. The output of the fourth stage is spliced ​​with the output of the third stage after upsampling, and then processed by a channel attention module. The output of the channel attention module is upsampled and spliced ​​with the output of the first stage spatial attention module and the output of the second stage spatial attention module, and the spliced ​​output is processed by another channel attention module. The detection model is trained by integrating the landslide and building segmentation tasks into a multi-task model based on the ViTAEv2-S backbone network. A multi-task model training strategy is proposed. The landslide segmentation task is mainly improved by sharing auxiliary training heads and other network layers in the backbone network except the BN layer, and the landslide branch uses Dice loss to improve the accuracy of each task in multi-task training. The building segmentation task is further improved by using the GradNorm loss weight adaptation method and the proposed alternating training strategy, and the problem of rapid convergence of the landslide task leading to a large drop in accuracy during the later convergence of the building task is alleviated. The trained detection model is used to simultaneously segment landslides and buildings.

2. The remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model according to claim 1 is characterized in that: The training of the detection model includes: The detection model is trained based on self-supervised pre-training and multi-task learning strategies.

3. The remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model according to claim 2 is characterized in that: The multi-task learning strategy includes: The landslide and building segmentation tasks are integrated into a multi-task model with a shared backbone network.

4. The remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model according to claim 3 is characterized in that: The loss functions adopted by the multi-task model include Dice loss and cross entropy loss, and the GradNorm loss weight adaptation method and alternating training strategy are adopted.

5. The remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model according to claim 1 is characterized in that: Also includes: Construct remote sensing landslide dataset and surrounding building dataset; The images in the remote sensing landslide dataset and the surrounding building dataset were preprocessed respectively.

6. The remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model according to claim 1 is characterized in that: Before training the detection model, the method further includes: Normalize the input image and convert it to the detection model input format.

7. The remote sensing landslide and surrounding building detection method based on the ViTAEv2-S model according to claim 1 is characterized in that: The method of synchronously segmenting the landslide and the building using the trained detection model includes: The generated probability map is thresholded and segmented, and the landslide or building binary mask is output after threshold segmentation.

8. A remote sensing landslide and surrounding building detection system based on the ViTAEv2-S model, characterized in that: include: A detection model construction module is used to construct a detection model. The detection model adopts a lightweight segmentation framework, including an encoder and a decoder. The encoder adopts the ViTAEv2-S backbone network, combined with multi-scale void convolution and self-attention mechanism to extract multi-scale features; the decoder simplifies the U-Net structure, removes the pyramid pooling module, adds a spatial attention module to focus on landslide and building areas, and adds a channel attention module to enhance the feature channels corresponding to landslides and buildings; the encoder includes four step-by-step downsampling stages, each stage contains multi-scale void convolution and self-attention mechanism; the decoder includes upsampling layers and cross-layer connections, the upsampling layers gradually restore the resolution to the input size, the cross-layer connections fuse the features of the corresponding stages of the encoder, add a spatial attention module after the output to suppress spatial noise, and add a channel attention module before the prediction head to enhance the weight of key features; the spatial attention module includes a 3×3 convolution layer and a sigmoid activation function, and the channel attention module includes a global average pooling layer and a fully connected layer; The output of the first stage of the encoder is processed by a spatial attention module, and the output of the second stage is processed by another spatial attention module after upsampling. The output of the fourth stage is spliced ​​with the output of the third stage after upsampling, and then processed by a channel attention module. The output of the channel attention module is upsampled and spliced ​​with the output of the first stage spatial attention module and the output of the second stage spatial attention module, and the spliced ​​output is processed by another channel attention module. A training module is provided for training the detection model. Based on the ViTAEv2-S backbone network, the landslide and building segmentation tasks are integrated into a multi-task model, and a multi-task model training strategy is proposed. Specifically, the landslide segmentation task is mainly improved by sharing auxiliary training heads and other network layers in the backbone network except the BN layer, and the landslide branch uses Dice loss to improve the accuracy of each task in the multi-task training. The building segmentation task is further improved by using the GradNorm loss weight adaptation method and the proposed alternating training strategy, and the problem of the rapid convergence of the landslide task causing a large drop in accuracy when the building task converges is alleviated. The detection module is used to synchronously segment landslides and buildings using the trained detection model.

Images