Long-term landslide identification method and system for various terrains

Abstract

The application discloses a kind of long time sequence landslide identification method and system for multiple terrain environments, comprising the following steps: for multiple terrains, long time sequence landslide dataset construction and dataset enhancement are carried out;Long time sequence landslide identification model is constructed based on long time sequence landslide dataset;Long time sequence landslide identification model is used to extract and classify landslide for input remote sensing image;Finally, data backflow is carried out, and whether it needs to increase and expand label or delete wrong label is judged through computer post-processing and artificial interactive correction, and this mechanism can adjust and optimize model according to feedback.This application has excellent multi-temporal long time sequence landslide identification capability, good scalability and easy to deploy, maintain characteristics, improves long time sequence landslide identification precision and efficiency under multiple terrains, especially suitable for long-term monitoring and disaster prevention of landslide in earthquake disaster area and other special geographical environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and remote sensing image processing, in particular to a long-time sequence landslide recognition algorithm, and more particularly to a long-time sequence landslide recognition method and system for various terrains. BACKGROUND

[0002] In natural disaster monitoring and geological disaster prevention, accurately identifying and monitoring different types and evolution processes of landslides is crucial. Landslides are a common geological disaster that poses a serious threat to human society and the natural environment. Traditional long-time sequence landslide recognition and monitoring methods mainly rely on manual interpretation and field investigation, which is time-consuming and labor-intensive and prone to subjective errors, making it difficult to meet the monitoring and management needs of large-scale areas. With the development of remote sensing technology, it has become easier to obtain high-quality and high-resolution remote sensing images, which provides new possibilities for landslide monitoring, geological disaster assessment, and environmental protection. However, accurately identifying and classifying landslides in these images, especially in diverse terrain conditions, remains a challenge.

[0003] Firstly, the problem of landslide recognition in remote sensing images is complex and diverse. Different types of landslides (such as collapse, debris flow, landslide, etc.) exhibit different characteristics in images. These characteristics are influenced by various factors, including the type of landslide, the covered terrain, seasonal changes, lighting conditions, etc. For example, collapse usually presents as steep cliffs and broken rocks, while debris flow may form obvious gullies and sediments due to water erosion. These differences limit the recognition effect based on traditional image processing methods, making it difficult to meet the requirements of high precision and efficiency.

[0004] Secondly, the problem of small size target recognition in remote sensing images is also very prominent. In large-scale remote sensing images, some targets such as landslides are relatively small and often mixed with the surrounding natural environment and other geological structures. This not only increases the difficulty of target recognition, but also requires the recognition model to have stronger feature extraction capability.

[0005] In addition, the problem of sample imbalance is particularly significant in long-time sequence landslide recognition of remote sensing images. In practical applications, landslide areas are usually more sparse than other ground objects (such as forests, farmland, buildings, etc.), resulting in a serious imbalance in the number of landslide samples in the training data set. This imbalance can affect the training effect of the model, making the model perform poorly in identifying landslide types.

[0006] In addition, the limitations of existing remote sensing image processing methods in dealing with complex backgrounds or similar features are also a major problem. Remote sensing images often contain complex background information, such as land, vegetation, buildings, etc., which may have visual similarities with landslides, increasing the risk of misclassification. At the same time, traditional rule-based methods often have limited effectiveness in dealing with these complex scenarios, making it difficult to adapt to changing environmental conditions. SUMMARY

[0007] Therefore, it is necessary to develop a long-term landslide recognition method for various terrains, which can adapt to various terrain conditions, process small-sized targets, solve the problem of sample imbalance, and effectively distinguish long-term landslides in complex backgrounds.

[0008] The recognition method proposed by the present application is born in response to the above problems. It fuses SwinTransformer encoder, PSA attention mechanism module and FCUperNet decoder, and changes the loss function to focal binary cross-entropy loss function, which plays an important role in solving the problem of sample imbalance, improving the recognition ability of the model for minority classes and small areas of landslide regions, and building a powerful recognition module. Finally, data backflow is performed to determine whether to increase and expand the label or delete the wrong label. This mechanism enables the algorithm to adjust and optimize the model according to feedback during actual application, which is crucial for accurately classifying a small number of important targets such as landslides in images. The present application not only improves the recognition accuracy, but also significantly improves the processing efficiency, providing a new idea and method for solving a series of technical problems in long-term landslide recognition.

[0009] To achieve the above objectives, the technical method of the present application is as follows: a long-term landslide recognition method for various terrains, comprising the following steps:

[0010] Step 1, data set construction, obtaining remote sensing images of multiple years before and after earthquakes, and marking the landslide phenomena on the images to obtain labeled images;

[0011] Step 2, using data augmentation methods to augment the images in the data set;

[0012] Step 3, construction of a long-term landslide recognition model, including an encoder composed of Swin Transformer Blocks for obtaining different scale feature maps of input images, a pyramid pooling module for adjusting the scale of feature maps, a decoder composed of FPN and performing full-order long connection with the corresponding scale feature maps in the encoder, an attention mechanism module composed of PSA attention mechanism for fusing the output features of the decoder, and a classification head for pixel-level classification prediction;

[0013] Step 4, using the trained long-time sequence landslide identification model to extract and identify the landslide in the remote sensing image.

[0014] Further, the data enhancement method in step 2 includes contrast disturbance, image rotation, overexposure processing, color disturbance, hue separation, contrast enhancement, brightness adjustment, image sharpening, image shear, weather filter, haze processing and random patching.

[0015] Further, the encoder includes k structurally identical Swin Transformer Blocks. First, the input image is converted into smaller patch blocks after linear embedding, and then processed by Swin Transformer Block1. Then, through the Patch Merging technology, the number and feature dimension of the patch blocks gradually change. Finally, the final output is obtained after k-1 Swin Transformer Block processing.

[0016] Further, the input of the pyramid pooling module comes from the final output of the encoder. First, different scale pooling operations, convolution operations and up-sampling operations are performed to capture different scale context information and obtain multiple feature maps. Then, the multiple feature maps are fused and combined through addition operation to form the final fusion feature map.

[0017] Further, the full-stage long connection is realized through a full-stage long connection conversion layer. The processing process of the full-stage long connection is as follows:

[0018] For different scale feature maps output by the encoder, first, a 3x3 convolution layer and a ReLU activation function are applied to process each scale feature map to change the channel number of the feature map while keeping the spatial size unchanged. Then, a max-pooling layer is used for down-sampling, and the number of down-sampling is determined by the scale of the feature map. The deepest scale feature map does not need to be down-sampled, and the shallowest scale feature map is down-sampled k-1 times, where k is the number of Swin Transformer Blocks. The number of down-sampling decreases with the increase of the scale of the feature map. After each down-sampling operation, the spatial size is halved.

[0019] Further, the decoder integrates the feature maps from all levels of the encoder, the final output of the pyramid pooling module PPM, and the output of the full-stage long connection conversion layer.

[0020] The decoder comprises k FPN layers, wherein k is the number of Swin Transformer Blocks; the processing procedure of each FCN layer is as follows: receiving the feature map of the deepest layer from the pyramid pooling module, and up-sampling the feature map, then adding the feature map to the feature map of the corresponding scale of the encoder and the feature map of the same dimension after the full-length connection conversion processing; finally, k feature maps with the same number of channels are output.

[0021] Further, the processing procedure of the pyramid segmentation attention PSA is as follows:

[0022] After the input feature map x is processed by the spatial convolution, it is re-grouped, each group is processed in its corresponding spatial convolution layer, and the feature map of each group is calculated by the corresponding SE block to generate the group weight; in the SE block, first, the local average pooling is performed to pool the feature map of each channel to generate the channel-level global feature descriptor; after the global average pooling, the dimension reduction convolution layer reduces the dimension of the feature by 1x1 convolution; the feature after the ReLU activation is upgraded by the 1x1 convolution layer; and the importance weight is limited between 0 and 1 by the Sigmoid activation function; then, focusing is performed, that is, the normalized weight is multiplied by the output of the spatial convolution processing element by element to focus on important spatial features; finally, the last output feature map is reorganized to match the dimension of the original input feature map.

[0023] The output feature map after the processing of each FCN layer by the PSA is finally reshaped and spliced, and the spliced feature map is classified and predicted by the classification head at the pixel level to output the probability distribution of each pixel belonging to each category.

[0024] Further, the long-time sequence landslide recognition model is trained using a focal binary cross-entropy loss function, the focal binary cross-entropy loss function takes the binary cross-entropy loss as the basis, and the weight is adjusted by applying the focal loss function; wherein the calculation formula of the binary cross-entropy loss function BCELoss is as follows:

[0025]

[0026] In the formula, N is the number of samples, y i is the true label of the i-th sample, taking the value of 0 or 1, p i is the probability that the model predicts the i-th sample as class 1;

[0027] The calculation formula of the focal loss function FLoss is as follows:

[0028] FLoss(p t )=-α t (1-p t ) γ log(p t )

[0029] In the formula, p t is the probability predicted by the model, gamma is a regulation parameter for reducing the weight of easy-to-classify samples, alpha t is a class-level weight parameter for further solving the class imbalance problem;

[0030] The binary cross-entropy loss and the focal loss are combined to obtain the overall formula of the focal binary cross-entropy loss function:

[0031]

[0032] In the formula, FocalBCELoss represents the focal binary cross-entropy loss function, N is the total number of samples, is the prediction probability of the model corresponding to the true label of the i-th sample, represents the class weight of the i-th sample, according to its true label t i The weight alpha1 of the positive class or the weight alpha0 of the negative class is selected to balance the difference in the number of classes, wherein alpha0 and alpha1 are hyperparameters.

[0033] Further, the confidence map post-processing model is optimized through a computer post-processing and artificial interaction correction mechanism, specifically: an accuracy threshold a is set, and it is automatically evaluated whether the recognition result meets the standard, if the test index does not reach the threshold a, the test data is backflowed for artificial re-interpretation and labeling, to construct new labels, expand the existing data set or delete the wrong labels.

[0034] The application also provides a long-time sequence landslide identification system for various terrains, comprising:

[0035] A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the long-time sequence landslide identification method for various terrains as described in the above technical solution.

[0036] In summary, the application develops a long-time sequence landslide identification method for various terrains, which can adapt to various terrain conditions, process small-size targets, solve the sample imbalance problem and effectively distinguish complex backgrounds, and solves the problem of accurately identifying and classifying landslides under various terrain conditions. Moreover, through the mechanism of the data backflow module, the algorithm can adjust and optimize the model according to the feedback in the actual application process, which is very crucial for accurately classifying a small number of important targets such as landslides in images. The application not only improves the recognition accuracy, but also significantly improves the processing efficiency, provides a new idea and method for solving a series of technical problems in landslide identification, and can better support decision-making in the fields of geological disaster monitoring, land resource planning and environmental monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0038] Figure 1 The flowchart of the embodiment of the present application is shown in the figure.

[0039] Figure 2 The flowchart of the overall model of the embodiment of the present application is shown in the figure.

[0040] Figure 3 The structural diagram of the PSA module provided by the embodiment of the present application is shown in the figure.

[0041] Figure 4 The flowchart of the PPM module provided by the embodiment of the present application is shown in the figure.

[0042] Figure 5 The landslide identification result map of the partial model of the embodiment of the present application in the test year is shown in the figure.

[0043] Figure 6 The partial diagram of the landslide identification result map of the partial model of the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0045] It should be understood that the schematic drawings are not drawn according to the actual object. The flowchart in the present application shows the operations realized according to some embodiments of the present application. It should be understood that the operations of the flowchart can not be realized in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowchart or removed from the flowchart by those skilled in the art under the guidance of the content of the present application. Some block diagrams shown in the drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be realized in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.

[0046] The terms "first", "second", and the like, as used in the description and in the claims, do not imply any particular order, but are used for descriptive purposes only. Thus, a feature specified with "first" and "second" can include at least one of the features. The term "and / or", as used herein, describes association between or among any elements linked by the term, and can exist in three states: A and / or B, A or B, and A and B. The term "comprising" is used herein to mean that the compositions and methods include the recited elements, but not excluding others.

[0047] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearance of the phrase in various places in the specification is not necessarily all referring to the same embodiment, nor is it necessarily referring to a separate or alternative embodiment to the other embodiments. It is explicitly and implicitly contemplated that embodiments described herein can be combined with each other.

[0048] The main technical difficulties solved by the present application are:

[0049] 1. Constructing a large-scale landslide data set containing various terrains and scenes, by collecting remote sensing images of multiple years before and after earthquakes, and labeling the landslide phenomena on the images to obtain labeled images. These images cover landslides under different time periods, different seasons, and different weather conditions, ensuring the diversity and representativeness of the data set;

[0050] 2. The algorithm can adapt to various terrain conditions, process small-sized targets, solve the problem of sample imbalance, and effectively distinguish complex background landslide types. The present application fuses a Swin Transformer encoder, a PSA attention mechanism module, and an FCUperNet decoder, and changes the loss function to a focal binary cross-entropy loss function, to construct a powerful recognition module;

[0051] 3. Landslide type recognition generally requires professional knowledge training, manual recognition and review, and has high cost. The algorithm proposed by the present application can automatically identify different types of landslides, reducing the dependence on professional knowledge and reducing the cost of manual recognition and review. Through the data backflow mechanism, the algorithm can adjust and optimize the model according to the feedback, further improving the accuracy and efficiency of the recognition.

[0052] As shown in Figure 1 The technical scheme of the present application is a long-time sequence landslide recognition method for various terrains, including: landslide data set construction, landslide data set enhancement, long-time sequence landslide recognition algorithm construction, training to generate a long-time sequence landslide recognition model, recognizing landslides in long-time sequence remote sensing images, and judging whether the data backflows. The specific process is as follows:

[0053] First step: long time series landslide dataset construction

[0054] Second step: long time series landslide dataset enhancement

[0055] Third step: construction of long time series landslide recognition algorithm

[0056] By fusing the Swin Transformer encoder, PSA attention mechanism module and FCUperNet decoder, and changing the loss function to the focal binary cross entropy loss function, a powerful recognition module is constructed.

[0057] Fourth step: training and generation of long time series landslide recognition model

[0058] Fifth step: long time series landslide recognition model for identifying corresponding targets on remote sensing images

[0059] Sixth step: data backflow, through computer post-processing and manual interaction correction, judge whether the returned sample needs to build a new label, expand the existing dataset or delete the wrong label, set the threshold a, according to the judgment, whether the predicted long time series landslide recognition model test index reaches the threshold, if not, the test data backflow, manual re-interpretation and labeling.

[0060] The specific implementation of the complete technical solution of the application is described as follows:

[0061] 1. Data set construction process: collect remote sensing images of multiple years before and after the earthquake, and mark the landslide phenomena on the images to obtain marked images. These images should cover landslides in different time periods, different seasons and different weather conditions to ensure the diversity and representativeness of the data set.

[0062] 2. Data set enhancement process: different kinds of augmentation are performed on the data set through image algorithms to simulate the imaging effect under different times and seasons. The specific enhancement methods include contrast disturbance, image rotation, overexposure processing, color disturbance, hue separation, contrast enhancement, brightness adjustment, image sharpening, image shear, weather filter, haze processing and random patching, etc. to enrich the diversity of the data set and improve the generalization ability and robustness of the model.

[0063] 3. The long time series landslide recognition algorithm of the application for multiple terrains is mainly composed of three modules as shown in Figure 2 The first module is an encoder composed of Swin Transformer, the second module is an attention mechanism module composed of PSA attention mechanism, and the third module is a decoder composed of FCUperNet. These three modules are connected in series and the loss function is changed to the focal binary cross entropy loss function. The explanation of each module is as follows:

[0064] (1) Using Swin-Transformer as an encoder, features are gradually extracted and refined through a hierarchical method. At different network levels, features are down-sampled and converted to form feature representations of different scales. Swin Transformer as an encoder provides strong feature extraction capability. Swin Transformer includes several stacked Swin Transformer blocks, each of which includes an MLP layer. The calculation process of the MLP layer is described as follows:

[0065] Z1 = XU + b1

[0066] A = σ(Z1)

[0067] Y = AV + b2

[0068] In the formula, Z1 is the first linear transformation formula, X is the input, U is the weight matrix of the first linear mapping, b1 is the bias term corresponding to U, A is the activation function, σ is the ReLU function, Y is the output, V is the weight matrix of the second linear mapping, and b2 is the bias term corresponding to V.

[0069] In this embodiment, the Swin Transformer encoder includes four Swin Transformer blocks with the same structure. First, the input image is converted into smaller patch blocks after linear embedding, with a feature dimension of C. The output resolution of the Swin Transformer Block1 processing is [8, 128, 32, 32] (corresponding to [batch size, channel number, height, width], respectively). As the model processes stage by stage, the number of patch blocks and the feature dimension gradually change through the Patch Merging technology, until the final output dimension reaches [8, 1024, 4, 4].

[0070] (2) The FCUperNet decoder addresses the limitations of traditional feature pyramid decoders in processing multi-scale feature maps and preserving rich semantic information, such as Figure 2 As shown in Figure 4 , the improved full-stage long connection feature pyramid network (FPN) effectively captures different scale context information and supplements low-dimensional features. As shown in Figure 3 , the pyramid segmentation attention (PSA) mechanism is further adopted to fuse these features. This design not only solves the problem of detail loss caused by the reduction of feature map resolution, but also overcomes the problem of low-dimensional feature information loss and semantic information reduction.

[0071] As a feature extraction component, the pyramid pooling module (PPM) generates a set of feature layers rich in high semantic information by implementing multi-scale pooling and down-sampling operations. Subsequently, these feature layers are up-sampled to adapt to large-sized feature maps to facilitate small target detection. This module integrates the results of each pooling to form a more comprehensive feature representation, effectively capturing multi-scale context information, and thus improving the segmentation accuracy of the model.

[0072] In the FCUperNet model, the pooling scales of the PPM part are set to (1, 2, 3, 6), and the input feature map comes from the final output of the encoder, with a dimension of [8, 1024, 4, 4] (corresponding to [batch size, channel number, height, width] respectively). First, a global pooling operation with a scale of 1 is performed on the feature map, so that the size of the processed feature map becomes [8, 1024, 1, 1]. Then, a 1x1 convolution is used to adjust the channel number to 512 (the specific channel number is implementation-dependent), and the resulting feature map dimension remains [8, 512, 1, 1], which is then up-sampled to the initial input dimension [8, 512, 4, 4]. Next, a pooling operation with a scale of 2 is performed, using a 2x2 size, resulting in a feature map size adjustment of [8, 1024, 2, 2]. After 1x1 convolution, the channel number becomes 512, and the feature map size is [8, 512, 2, 2], which is then up-sampled to the original input dimension [8, 512, 4, 4]. For the pooling operation with a scale of 3, a 3x3 size is used for pooling, so that the feature map size becomes [8, 1024, 3, 3]. Using a 1x1 convolution, the channel number is adjusted to 512, and the feature map dimension remains [8, 512, 3, 3], which is then up-sampled to the original input dimension [8, 512, 4, 4]. In the pooling operation with a scale of 6, although the input size is only 4x4, 6x6 pooling is achieved through padding, so that the feature map size becomes [8, 1024, 6, 6]. After 1x1 convolution, the channel number is adjusted to 512, maintaining the size as [8, 512, 6, 6], and then down-sampling is performed to match the original input dimension [8, 512, 4, 4]. After the above processing, four feature maps with a size of [8, 512, 4, 4] are obtained, which capture context information at different scales. Subsequently, these feature maps are fused by adding operation, and the channel number remains unchanged, maintaining the size as [8, 512, 4, 4]. The feature of each pixel position is the sum of the features of the same position in different scale feature maps, forming the final fused feature map.

[0073] Based on the design concept of U3+, a full-stage long connection module is designed to transform all output feature maps of the backbone, i.e., the encoder. For example, for the input [8, 128, 32, 32], the channel number is increased from 128 to 512 by applying a 3x3 convolution layer and a ReLU activation function, while the spatial size remains unchanged, i.e., [8, 512, 32, 32]. Next, three consecutive downsampling operations are performed using a MaxPool2d(kernel_size=2, stride=2) max pooling layer, and the spatial size is halved after each operation, resulting in [8, 512, 16, 16], [8, 512, 8, 8], and [8, 512, 4, 4], respectively. For the input [8, 256, 16, 16], the channel number is increased from 256 to 512 by applying a 3x3 convolution and a ReLU activation function, while the spatial size remains unchanged, i.e., [8, 512, 16, 16]. Next, two rounds of max pooling downsampling are performed, resulting in feature maps with sizes [8, 512, 8, 8] and [8, 512, 4, 4], respectively. For the input [8, 512, 8, 8], the channel number is 512, and the spatial size is halved by one max pooling downsampling operation, resulting in [8, 512, 4, 4]. For the input [8, 1024, 4, 4], the channel number is reduced from 1024 to 512 by applying a 3x3 convolution layer and a ReLU activation function, while the spatial size remains unchanged, i.e., [8, 512, 4, 4]. Since this input is already the smallest spatial size, no pooling operation is performed.

[0074] The feature pyramid network (FPN) design is inspired by the human visual system, which can analyze visual information at multiple scales and abstraction levels. The feature pyramid network adjusts the channel number of the bottom-up features through 1x1 convolution and fuses them with the pyramid pooling module output in the top-down path. Through 3x3 convolution, it further extracts features and connects them horizontally to generate a series of feature maps rich in semantic information. In multi-scale feature fusion, FPN may reduce the resolution of feature maps through multiple downsampling and upsampling operations. In the upsampling part of FPN, only high-level features are fused with the previous layer features through nearest neighbor upsampling and addition, which may cause loss of low-dimensional feature information. The reduction of channels in feature fusion may lead to loss of semantic information, which in turn affects the segmentation accuracy. To address this issue, we borrow the full-stage long connection strategy of Unet3+ and convert the output feature maps of each layer of the backbone network to the upsampling layers of FPN through convolution and pooling and addition, which helps the model to obtain more low-dimensional feature information and improve the segmentation accuracy.

[0075] In this embodiment, we improved the feature pyramid network (FPN) to integrate the data features from all levels of the backbone, the final output of the PPM, and the output of the full-stage long connection conversion layer. First, the FPN receives the deepest feature map [8, 512, 4, 4] from the PPM and up-samples it to [8, 512, 8, 8], then adds it to the [8, 512, 8, 8] of the backbone and the same dimension feature map output by the full-stage long connection conversion processing, next, the fused feature map is further up-sampled to [8, 512, 16, 16], then added to the [8, 256, 16, 16] of the backbone feature map adjusted by 1x1 convolution and the [8, 512, 16, 16] output by the full-stage long connection conversion layer, finally, the up-sampling process continues to [8, 512, 32, 32], and is added to the [8, 128, 32, 32] of the backbone feature map adjusted by 1x1 convolution and the [8, 512, 32, 32] output by the full-stage long connection conversion layer. Finally, the deepest feature map [8, 512, 4, 4] of the PPM is added to the output of the full-stage long connection conversion layer.

[0076] Each fusion produces a feature map with 512 channels, and all feature maps are unified to 512 channels before fusion, so the final output feature map of the FPN also maintains 512 channels.

[0077] (3) The feature map output by the FPN needs to be fused by the PSA self-attention mechanism before splicing. In the PSA self-attention, the input feature map x is first re-grouped into the shape b, self.S, c / / self.S, h, w by spatial convolution processing (SPC module), each group is processed in its corresponding spatial convolution layer, c / / self.S is the channel number of each group. The feature map of each group calculates the SE weight through the corresponding SE block to generate the weight of each group. In the SE block, the global average pooling (GAP) is first performed on each channel feature map to generate a channel-level global feature descriptor. After global average pooling, the dimension reduction convolution layer reduces the dimension of the feature by 1x1 convolution, and the feature after ReLU activation is up-sampled by 1x1 convolution layer. The Sigmoid activation function limits the importance weight between 0 and 1. Enter the spatial attention focusing (SPA module), multiply the normalized weight (processed by Softmax) and the output of the SPC module element by element, and focus on important spatial features. Finally, the PSA output feature map is reorganized to match the dimension of the original input feature map.

[0078] The PSA output feature map is finally resized to [8, 512, 32, 32] and spliced into [8, 2048, 32, 32]. The spliced feature map is subjected to pixel-level classification prediction by a classification head, and the probability distribution of each pixel belonging to each category is output.

[0079] The pyramid segmentation attention (PSA) module aggregates the convolution results generated by different size convolution kernels to construct a pyramid-shaped feature map, and applies an attention mechanism on the basis of the pyramid-shaped feature map, so as to mine rich feature information. The core of the PSA module is that it can make each element not only associated with adjacent elements, but also associated with all other elements in the sequence in the processing of sequence data, and by evaluating the relative importance between elements, long-distance dependencies can be adaptively captured. The highlights of the PSA module design include: first, polarization filtering, which maintains a high resolution in the channel and spatial dimensions to reduce information loss; second, an enhancement mechanism that accurately simulates the output distribution of fine-grained regression by combining nonlinear direct fitting. In the FCUperNet decoder, the PSA module is used to perform attention reconstruction on the channel and spatial dimensions of the FPN and PPM outputs after full-order long connection, so as to optimize feature fusion and effectively integrate multi-dimensional and multi-scale context information.

[0080] (4) Improvement of the loss function. The original cross-entropy loss function may cause the model to pay excessive attention to the majority classes in a class-imbalanced dataset, which will lead to a decrease in the recognition performance of the model for minority classes. In this paper, the focal binary cross-entropy loss function (FocalBCE) is used, which is based on the binary cross-entropy loss function (BCELoss) and applies the weight adjustment of the focal loss function (FL). The binary cross-entropy loss function is as follows:

[0081]

[0082] where N is the number of samples, y i is the true label of the i-th sample, taking the value 0 or 1, p i is the probability that the model predicts the i-th sample to be class 1.

[0083] The Focal loss loss function is as follows:

[0084] FLoss(p t )=-α t (1-p t ) γ log(p t )

[0085] where p t is the model predicted probability for the correct class. γ is a tuning parameter that reduces the weight of easy-to-classify samples. α t is a class-level weight parameter that addresses the data distribution imbalance problem by directly amplifying / reducing the loss values of different classes. It works together with the γ parameter (sample difficulty weight) to improve the model's recognition ability for minority classes or difficult samples.

[0086] Combining the binary cross-entropy loss and the focal loss, the overall formula of the focal binary cross-entropy loss function can be obtained as follows:

[0087]

[0088] where N is the total number of samples, is the adjusted prediction probability for the correct class. γ is a tuning parameter that reduces the weight of easy-to-classify samples. represents the class weight of the i-th sample, which is determined according to its true label t i The weight of the positive class α1 or the weight of the negative class α0 is selected to balance the difference in the number of classes, where α0 and α1 are hyperparameters.

[0089] When combining BCELoss and Focal Loss, the weight adjustment mechanism of Focal Loss is used to modify the basic binary cross-entropy loss of each sample. First, the basic binary cross-entropy loss of each sample is calculated. Second, for each sample, a weight factor (1-p t ) γ and α t is calculated according to the Focal Loss formula. This weight factor reduces the loss of easy-to-classify samples (i.e., p t close to 0 or 1), while increasing the loss of difficult-to-classify samples. Finally, the calculated weight factor is applied to the binary cross-entropy loss of each sample. In this way, the model is guided to pay more attention to difficult-to-classify samples during training, while giving less attention to easy-to-classify samples. Such a loss function is particularly suitable for class-imbalanced datasets and can help improve the model's recognition ability for minority classes.

[0090] In summary, the long-time series landslide recognition algorithm for multiple terrains of the present application mainly consists of three modules: Module 1 is an encoder composed of Swin Transformer, Module 2 is an attention mechanism module composed of PSA attention mechanism, and Module 3 is a decoder composed of FCUperNet. These three modules are connected in series and the loss function is changed to the focal binary cross-entropy loss function.

[0091] 4. The application is based on the enhanced data set using long-term landslide recognition algorithm for various terrains to train, and obtain a long-term landslide recognition model.

[0092] 5. The application uses the trained long-term landslide recognition model to infer remote sensing images to obtain long-term landslide recognition results in different years, such as Figure 5 The landslide recognition result map of the embodiment of the application in the test year is shown in part of the model, Figure 6 The landslide recognition result map of the embodiment of the application in part of the model is shown in part of the model.

[0093] 6. The application judges whether the returned sample needs to be separately constructed into a new label, expanded into an existing data set or deleted by computer post-processing and manual interaction. This mechanism can adjust and optimize the model according to the feedback. Manual intervention is used to judge whether the returned sample needs to be separately constructed into a new label, expanded into an existing data set or discarded, and then enters the first step. Specifically, a threshold a is set, and whether the predicted long-term landslide recognition model test index reaches the threshold is judged. If not, the test data is returned to the flow, and manual re-interpretation and labeling are performed. The value of the threshold a can be set to control the workload of manual secondary labeling.

[0094] 7. Finally, the long-term landslide recognition is deployed to the mobile terminal or the cloud.

[0095] On the other hand, the embodiment of the application also provides a long-term landslide recognition system for various terrains, which comprises:

[0096] A processor and a memory, the memory is used to store program instructions, and the processor is used to call the storage instructions in the memory to execute the long-term landslide recognition method for various terrains as described in the above technical solution.

[0097] Sample data is used to train and verify the performance of a deep learning model, which is an important prerequisite for landslide recognition. The number and quality of sample images play a key role in the results of training neural network models. The model test area of the application is 471 square kilometers between a certain town and a certain town. Within this range, in addition to the time sequence landslide inventory data for more than 10 years, we also obtained Landsat images (orbit number 130 / 38) from 2008 to 2018, as shown in Table 1. Using Landsat TM and Landsat OLI data, we can crop landslide sample images to train and test deep learning models.

[0098] Table 1 Satellite remote sensing data table

[0099]

[0100] Meanwhile, comparative test experiments of SwinFCUperNet model (the model of the application) and UNet, UNet3+, TransUNet, Pspnet, DeepLabv3+, Mask2Fomer, SwinUpernet, Segformer and other typical semantic segmentation models are carried out to verify whether SwinFCUperNet has excellent performance. In the test, a single year (2008) of images is used as a sample to train the model, and the performance of the model in other years (2009-2018) is tested. If a model can achieve excellent performance without training in other years, it indicates that the model has good migration ability and can effectively reduce the workload of long-term landslide mapping.

[0101] Table 2 Time migration ability test of various models

[0102]

[0103]

[0104] The data in the table are the average values of the accuracy indicators of the test years (2009-2018). Table 2 shows that the SwinFCUperNet model exhibits excellent performance in almost all accuracy indicators. The Precision of the model reaches 74.33%, the Recall is 80.18%, the mIoU is 76.49%, and the F1Score is 77.14%, which are the highest among all the compared models. This proves that the SwinFCUperNet model not only can accurately identify landslide events, but also can maintain high identification ability in a long time range, showing excellent time migration ability.

[0105] Further testing the performance of the model in each test year shows that SwinFCUperNet has excellent average performance in multiple years, especially in Precision, Recall, mIoU and F1Score.

[0106] Table 3 Accuracy performance of some models in specific test years

[0107]

[0108]

[0109] The above method describes the examples of the present application in combination with the drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, which all belong to the protection of the present application.

Claims

1. A long-term landslide identification method for various terrains, characterized in that, Comprising the following steps: Step 1, data set construction, obtain remote sensing images of multiple years before and after the earthquake, and mark the landslide phenomenon on the image to obtain a marked image; Step 2, using data enhancement method to enhance the images in the data set; Step 3, construction of long time series landslide identification model, including an encoder composed of Swin Transformer Block for obtaining different scale feature maps of input images, a pyramid pooling module for adjusting the scale of the feature map, a decoder composed of FPN and performing full-order long connection with the feature map of the corresponding scale in the encoder, an attention mechanism module composed of PSA attention mechanism for fusing the output features of the decoder, and a classification head for pixel-level classification prediction; Full-order long connection is realized through a full-order long connection conversion layer, and the processing process of the full-order long connection is as follows: For different scale feature maps output by the encoder, first apply a 3x3 convolution layer and a ReLU activation function to process each scale feature map, change the channel number of the feature map, and keep the spatial size unchanged; Then use the maximum pooling layer for down-sampling, the number of down-sampling times is determined by the scale of the feature map, the feature map of the deepest scale does not need to be down-sampled, and the feature map of the shallowest scale is down-sampled k-1 times, where k is the number of Swin Transformer Block, and the number of down-sampling times decreases with the increase of the scale of the feature map, and the spatial size is halved after each down-sampling operation; The decoder integrates the feature maps from all levels of the encoder, the final output of the pyramid pooling module PPM, and the output of the full-order long connection conversion layer; The decoder includes k FPN layers, where k is the number of Swin Transformer Block; The processing process of each FPN layer is: receiving the deepest feature map from the pyramid pooling module and up-sampling it, then adding it to the corresponding scale feature map of the encoder and the same dimension feature map processed by the full-order long connection conversion; Finally, output k feature maps with the same channel number; Step 4, using the trained long time series landslide identification model to extract and identify landslides in remote sensing images. 2.The multi-terrain long-term landslide identification method of claim 1, wherein: The data enhancement method in step 2 includes contrast disturbance, image rotation, overexposure processing, color disturbance, hue separation, contrast enhancement, brightness adjustment, image sharpening, image shear, weather filter, haze processing and random patching.

3. The multi-terrain long-term landslide identification method of claim 1, wherein: The encoder contains k Swin Transformer Blocks with the same structure, first, the input image is linearly embedded into smaller patch blocks, then processed by Swin Transformer Block1, then the number and feature dimension of the patch blocks are gradually changed through the Patch Merging technology, and finally processed by k-1 Swin Transformer Blocks to obtain the final output. 4.The multi-terrain long-term landslide identification method of claim 1, wherein: The input of the pyramid pooling module is derived from the final output of the encoder, and first, different scale pooling operations, convolution operations and up-sampling operations are performed to capture different scale context information and obtain multiple feature maps, and then the multiple feature maps are fused and combined through addition operation to form the final fusion feature map. 5.The multi-terrain long-term landslide identification method of claim 1, wherein: The processing process of the pyramid segmentation attention PSA is as follows: After the input feature map x is processed by spatial convolution, it is re-grouped, each group is processed in its corresponding spatial convolution layer, and the feature map of each group is calculated by the corresponding SE block to generate the group weight; in the SE block, first, the local average pooling is performed to pool the feature map of each channel to generate the channel-level global feature descriptor; after global average pooling, the dimension reduction convolution layer reduces the dimension of the feature by 1x1 convolution, the feature after ReLU activation is upgraded by 1x1 convolution layer, and the importance weight is limited between 0 and 1 by Sigmoid activation function; then focusing is performed, that is, the normalized weight is multiplied by the output of the spatial convolution processing element by element, focusing on important spatial features; finally, the last output feature map is reorganized to match the dimension of the original input feature map. The output feature map processed by the PSA after each FPN layer is finally reshaped and spliced, and the spliced feature map is classified and predicted by the classification head at the pixel level to output the probability distribution of each pixel belonging to each category.

6. The multi-terrain long-term landslide identification method of claim 1, wherein: The long-term landslide identification model is trained using the focal binary cross-entropy loss function. This function is based on the binary cross-entropy loss and applies the focal loss function for weight adjustment. The calculation formula is as follows: where N is the number of samples, is the true label of the th sample, taking values 0 or 1, is the probability that the model predicts the th sample to be of class 1; Focal loss function The calculation formula is as follows: wherein, is the model predicted probability, γ is a tuning parameter to reduce the weight of easy-to-classify samples, is a class level weight parameter to further address the class imbalance problem; The binary cross entropy loss and the focal loss are combined to obtain the overall formula of the focal binary cross entropy loss function: In the formula, Let N represent the focal binary cross-entropy loss function, where N is the total number of samples. The model is for the first The predicted probability corresponding to the true label of each sample. Indicates the first The class weights of each sample are based on its true label. Choose the weight of the positive class. Or the weight of the negative class This is used to balance the differences in the number of categories, where and This is a hyperparameter.

7. The multi-terrain long-term landslide identification method of claim 1, wherein: Further comprising, the confidence map post-processing model is optimized by computer post-processing and artificial interactive correction mechanism, specifically: setting an accuracy threshold a, automatically evaluating whether the recognition result meets the standard, if the test index does not reach the threshold a, the test data is backflowed for artificial re-interpretation and labeling to build new labels, expand the existing data set or delete the wrong labels.

8. A long-term landslide identification system for various terrains, characterized by, Comprise: A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the long-time sequence landslide identification method for multiple terrains as claimed in any one of claims 1-7.

Images