Earthquake landslide identification method and system based on multi-feature cross fusion twin network

Abstract

The application discloses a kind of based on multi-feature cross fusion twin network earthquake landslide identification method and system, the method of the present application includes using multi-feature cross fusion twin network based on the SAR image before and after earthquake in same scene to carry out earthquake landslide identification, the multi-feature cross fusion twin network includes encoder and decoder, the encoder includes two feature extractors of twin structure and shared parameter, the decoder includes multi-scale difference feature and aggregation feature generation module MDAG, multi-feature cross fusion module MFCM and convolution upsampling module.The present application aims to use multi-feature cross fusion twin network to introduce change detection into earthquake landslide identification task, the surface change information provided by the SAR image of different time phases before and after earthquake is extracted by feature, finally realizes the identification of earthquake landslide boundary, to solve the problems, such as SAR image resolution is lower in traditional change detection, it is difficult to extract landslide feature and landslide scale is various.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an earthquake landslide identification method and system based on a multi-feature cross-fusion twin network. Background Technology

[0002] Earthquake landslides are a serious natural disaster triggered by seismic activity. Due to factors such as surface loosening, soil liquefaction, and landslides, soil, rocks, and other materials slide rapidly down slopes or mountain surfaces. Because of their destructive power, large affected area, and suddenness, they pose a significant threat to human survival and the natural environment. Therefore, accurate detection and identification of earthquake landslides are crucial for disaster reduction, emergency response, and post-disaster reconstruction. In recent decades, with the rapid development of remote sensing technology and the surge in the number of remote sensing satellites, a wealth of data has been provided for research. Consequently, research using optical remote sensing imagery or synthetic aperture radar (SAR) images has increased year by year. Optical remote sensing imagery, in particular, is widely used by researchers in various fields such as agriculture, military, and geological environment monitoring due to its high resolution and multispectral characteristics. However, optical remote sensing images are highly susceptible to natural conditions. In earthquake and landslide detection tasks, earthquakes are often accompanied by adverse weather conditions such as clouds, rain, and fog, which limits the effectiveness of optical remote sensing images. SAR images, on the other hand, can penetrate clouds and fog, making them invaluable for landslide detection. Since a single post-earthquake SAR image contains only limited temporal resolution and cannot provide historical information on surface changes, many researchers utilize multi-temporal SAR image data to capture more detailed information on changes in land cover detection tasks.

[0003] In traditional remote sensing image change detection, a difference map is typically constructed first from two images, and then analyzed using unsupervised classification or clustering methods. Common methods for constructing difference maps include image differencing, ratio methods, mean ratio methods, and logarithmic ratio methods. Unsupervised classification typically uses threshold segmentation to classify images, with methods including KI, EM, and Otsu's method. Clustering methods commonly employ K-means clustering, fuzzy C-means clustering, and fuzzy local information C-means clustering. However, most of these methods require manual feature filtering based on thresholds, ignoring the semantic relationships between pixels. When constructing the difference map, they can only obtain changes through shallow pixel information from the two images, making it difficult to capture deeper information. In recent years, deep learning has developed rapidly. The high-dimensional features extracted by deep convolutional neural networks possess abstract semantic information, while the low-dimensional features contain rich texture details. Therefore, they are widely used in SAR image classification, change detection, and target recognition. The 2016 technical paper (Zhang Puzhao, Gong Maoguo, Su Linzhi, et al. Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing. 2016, 116: 24-41.) uses a denoising coding structure to perform change detection on multi-source remote sensing image data, and its accuracy is better than traditional change detection methods. The 2018 technical paper (Wang Qing, Zhang Xiaodong, Chen Guanzhou, Change detection based on Faster R-CNN for high-resolution remote sensing images). Remote sensing letters.(2018, 9(10), 923-932.) Faster R-CNN was used to detect changes in high-resolution remote sensing images. A 2019 technical paper (Gao Feng, Wang Xiao, Gao Yunhao, et al. Sea ice change detection in SAR images based on convolutional-wavelet neural networks[J]. IEEE Geoscience and Remote Sensing Letters. 2019, 16(8): 1240-1244.) introduced dual-tree complex wavelet transform into convolutional neural networks to reduce the influence of speckle noise and performed sea ice layer change detection on SAR. However, most landslide identification based on deep learning methods is currently performed on optical remote sensing images; no application of SAR images for landslide boundary identification has yet been found. Therefore, how to apply landslide identification based on deep learning methods to SAR images remains a key technical problem that urgently needs to be solved. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide an earthquake landslide identification method and system based on a multi-feature cross-fusion twin network, which addresses the above-mentioned problems in the prior art. This invention aims to introduce change detection into the earthquake landslide identification task by utilizing a multi-feature cross-fusion twin network. By extracting features from surface change information provided by SAR images at different time phases before and after the earthquake, the boundary of the earthquake landslide is finally identified, thus solving the problems of low SAR image resolution, difficulty in extracting landslide features, and diverse landslide scales in traditional change detection.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: An earthquake landslide identification method based on a multi-feature cross-fusion twin network includes using a multi-feature cross-fusion twin network to identify earthquake landslides based on SAR images of the same scene before and after an earthquake. The multi-feature cross-fusion twin network includes an encoder and a decoder. The encoder includes two feature extractors with twin structures and shared parameters. The decoder includes a multi-scale differential feature and aggregate feature generation module (MDAG), a multi-feature cross-fusion module (MFCM), and a convolutional upsampling module. The earthquake landslide identification process includes: using the two feature extractors to extract high-dimensional features at multiple scales from the SAR images of the same scene before and after an earthquake; using the multi-scale differential feature and aggregate feature generation module (MDAG) to extract differential and aggregate features of the earthquake landslide from the high-dimensional features at multiple scales of the SAR images of the same scene before and after an earthquake; using the multi-feature cross-fusion module (MFCM) to fuse the differential and aggregate features of the earthquake landslide into a fused feature map; and using the convolutional upsampling module to sequentially perform convolutional upsampling on the fused feature map to obtain the landslide identification result.

[0006] Optionally, the multi-scale differential feature and aggregation feature generation module MDAG includes multiple differential feature and aggregation feature generation modules DAG that correspond one-to-one with the scale of the high-dimensional features. Extracting differential and aggregation features of earthquake landslides from high-dimensional features at multiple scales of SAR images before and after an earthquake in the same scene using the multi-scale differential feature and aggregation feature generation module MDAG includes using the differential feature and aggregation feature generation modules DAG to extract differential and aggregation features from high-dimensional features of SAR images before and after an earthquake at corresponding scales. and The differential and aggregated features of earthquake-induced landslides are extracted, where the differential features refer to the high-dimensional features of SAR images before and after the earthquake at this scale. and The differences between them are aggregated features, which are high-dimensional features of SAR images before and after the earthquake at this scale. and Obtained by aggregation.

[0007] Optionally, the calculation function expression for the difference feature is: , In the above formula, To highlight the differences, a and b represent SAR images of the same scene before and after an earthquake. For matrix subtraction, and These are the high-dimensional features of SAR images before and after the earthquake at this scale.

[0008] Optionally, the calculation function expression for the aggregation feature is: , In the above formula, As an aggregation feature, For channel stacking, For Hadama product, For attention feature maps, and These are the high-dimensional features of the SAR images before and after the earthquake at this scale, and the function expression for extracting the attention feature map is: , In the above formula, For the Sigmoid function, Indicates similarity Adding dimensions, similarity High-dimensional features of SAR images before and after the earthquake at this scale and The similarity between them.

[0009] Optionally, the high-dimensional features of the SAR images before and after the earthquake at this scale and similarity between The expression for the computation function is: , In the above formula, and respectively and The corresponding element in the i-th dimension, and These are the high-dimensional features of the SAR images before and after the earthquake at this scale. for and Dimensions.

[0010] Optionally, when using the Multi-Feature Cross-Fusion Module (MFCM) to fuse the differential and aggregated features of earthquake landslides into a fused feature map, for each scale, fusing the differential and aggregated features into a fused feature map at that scale includes: firstly, extracting the tensor matrix of each dimension according to the following formula: , In the above formula, For the first i The first dimension j Tensor matrices, For the first Aggregated features in multiple dimensions For the first i The system identifies the differential features in each dimension; then, it upsamples the tensor matrices of each dimension layer by layer using deconvolution and then stacks the channels to obtain the fused feature map.

[0011] Optionally, the step of sequentially performing convolutional upsampling on the fused feature map to obtain the landslide recognition result includes: performing pixel binary classification and 4x upsampling on the fused feature map using a 3×3 convolution module and bilinear interpolation to obtain a landslide recognition result with the same size as the original image input to the network.

[0012] Optionally, the feature extractor is a ResNet50 network.

[0013] Furthermore, the present invention also provides an earthquake landslide identification system based on a multi-feature cross-fusion twin network, comprising a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the earthquake landslide identification method based on the multi-feature cross-fusion twin network.

[0014] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program for being programmed or configured by a microprocessor to execute the earthquake landslide identification method based on a multi-feature cross-fusion twin network.

[0015] Compared with the prior art, the present invention has the following main advantages: 1. The method of this invention includes using a multi-feature cross-fusion twin network to identify earthquake landslides based on SAR images before and after an earthquake in the same scene. The multi-feature cross-fusion twin network includes an encoder and a decoder. The encoder includes two feature extractors with twin structures and shared parameters. The decoder includes a multi-scale differential feature and aggregated feature generation module (MDAG), a multi-feature cross-fusion module (MFCM), and a convolutional upsampling module. This invention introduces change detection into the earthquake landslide identification task using a multi-feature cross-fusion twin network. By extracting features from surface change information provided by SAR images at different time phases before and after the earthquake, the method ultimately identifies the boundary of the earthquake landslide, thus solving the problems of low SAR image resolution, difficulty in extracting landslide features, and diverse landslide scales in traditional change detection.

[0016] 2. The encoder twin structure of the present invention and the two feature extractors sharing parameters enable the two twin networks to share the same feature representation capabilities, thereby reducing the number of network parameters and improving the training efficiency and feature extraction performance of the model. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the network structure of DACS-Net in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0019] Figure 3This is a diagram comparing early-stage and late-stage fusion.

[0020] Figure 4 This is a schematic diagram of the network structure of the differential feature and aggregation feature generation module DAG in an embodiment of the present invention.

[0021] Figure 5 This is a schematic diagram of the fusion principle of the Multi-Feature Cross-fusion Module (MFCM) in an embodiment of the present invention.

[0022] Figure 6 The results are from the landslide identification experiment in scenario I of this invention.

[0023] Figure 7 The results are from the landslide identification experiment in scenario II of this invention. Detailed Implementation

[0024] like Figure 1 As shown, this embodiment of the earthquake landslide identification method based on a multi-feature cross-fusion Siamese network includes using a Difference and Aggregated Feature Cross Fusion Siamese Network (DACS-Net) to identify earthquake landslides based on SAR images (Image1 and Image2) of the same scene before and after an earthquake. The DAS-Net includes an encoder and a decoder. The encoder includes two feature extractors with twin structures and shared parameters. The decoder includes a multi-scale difference feature and aggregation feature generation module (MDAG, shown in the figure), a multi-feature cross-fusion module (MFCM, shown in the figure), and a convolutional upsampling module (shown as two frames for convolution and upsampling in the figure). In this embodiment, the SAR images of the same scene before and after an earthquake specifically use two SAR images provided by Sentinel-1 at different times. Alternatively, SAR images from other satellites can be used as needed. Figure 2 As shown, in this embodiment, DACS-Net performs earthquake landslide identification based on SAR images of the same scene before and after an earthquake, including: using two feature extractors to extract high-dimensional features at multiple scales from SAR images of the same scene before and after an earthquake, see [link to relevant documentation]. Figure 1Specifically, this embodiment involves extracting high-dimensional features at four scales (Level 1 to Level 4) from SAR images of the same scene before and after an earthquake; using the Multi-Scale Differential Feature and Aggregate Feature Generation Module (MDAG) to extract differential and aggregate features of earthquake landslides from the high-dimensional features of SAR images of the same scene before and after an earthquake at multiple scales; using the Multi-Feature Cross-Fusion Module (MFCM) to fuse the differential and aggregate features of earthquake landslides into a fused feature map; and using the convolutional upsampling module to sequentially perform convolutional upsampling on the fused feature map to obtain the landslide identification result.

[0025] Change detection refers to comparing data from multiple time points or multiple sources to identify changes in a scene, used to monitor natural or man-made changes. Deep learning methods capable of end-to-end training can currently be mainly divided into two types: early fusion and late fusion. For example... Figure 3 As shown in (a), early fusion refers to fusing data from multiple temporal images during the input stage of the deep network to form a multi-channel input. This method allows the network model to directly learn the complex relationships between multi-temporal data, helping to extract richer feature information. However, in this method, data fusion is completed at the very beginning, which means that features from different temporal phases may influence each other. Figure 3As shown in (b), late-stage fusion refers to fusing feature maps from multiple time phases through feature splicing or other methods in the middle or end stages of a deep network. This links the feature extraction and difference recognition functions of the original image into a single network through multiple layers. This method fully utilizes the feature information extracted at different stages of the network, increasing feature diversity and expressive power. Late-stage fusion often uses a Siamese network structure to extract features from image data from different time phases. In change detection tasks, this structure can effectively learn the differences and similarities between images, thereby achieving change detection between two images. Therefore, this embodiment adopts the late-stage fusion method, meaning that fusion is not performed in the encoder but in the decoder. The encoder only needs to use a feature extractor for feature encoding. It should be noted that the feature extractor can use the required backbone network as needed. For example, as an optional implementation, the feature extractor in this embodiment is a ResNet50 network. ResNet, as a deep convolutional neural network, has multiple levels of feature representation capabilities and can learn richer and more complex features. It transmits information from the original input through skip connections (also known as residual connections), enabling the network to better learn residual features and effectively alleviate the vanishing gradient problem. It also allows ResNet to train deeper networks, improving network performance and expressive power. In this embodiment, the proposed DACS-Net uses a Siamese ResNet50 as a shared feature extractor. This parameter sharing allows the two Siamese networks to share the same feature representation capabilities, thereby reducing the number of network parameters and improving model training efficiency and feature extraction performance.

[0026] like Figure 1As shown, the multi-scale difference and aggregated feature generation module (MDAG) in this embodiment includes multiple difference and aggregated feature generation modules (DAGs) corresponding one-to-one with the scale of the high-dimensional features, denoted as DAG1 to DAG4 respectively. Since the high-dimensional features in this embodiment include four scales, it specifically includes four difference and aggregated feature generation modules (DAGs). The main function of the difference and aggregated feature generation modules (DAGs) is to transform the feature information of the same scale at different time phases obtained from the encoder into difference and aggregated feature extraction. This embodiment utilizes the multi-scale difference and aggregated feature generation module (MDAG) to extract the difference and aggregated features of earthquake landslides from the high-dimensional features of SAR images before and after an earthquake at multiple scales of the same scene. This includes using the difference and aggregated feature generation modules (DAGs) to extract the high-dimensional features of SAR images before and after an earthquake at corresponding scales. and The differential and aggregated features of earthquake-induced landslides are extracted, where the differential features refer to the high-dimensional features of SAR images before and after the earthquake at this scale. and The differences between them are aggregated features, which are high-dimensional features of SAR images before and after the earthquake at this scale. and Obtained by aggregation.

[0027] In traditional image change detection, methods such as difference, ratio, and mean-ratio are often used to construct difference maps that reflect change characteristics, followed by unsupervised or supervised methods for change feature detection. Therefore, the Dissimilarity Aspect (DAG) module generates difference and aggregate features by performing difference operations on features from different time phases to capture differences and change information between images. Comparing corresponding pixels of two feature maps through difference operations to obtain a difference map that reflects image changes helps us more clearly identify and locate changed areas. This operation transforms pre- and post-earthquake feature information into more representative change information, providing more valuable input for subsequent change detection tasks and improving the accuracy and robustness of change detection. See also... Figure 4 In this embodiment, the expression for the calculation function of the differential features is: , In the above formula, To highlight the differences, a and b represent SAR images of the same scene before and after an earthquake. For matrix subtraction, and These are the high-dimensional features of SAR images before and after the earthquake at this scale.

[0028] Aggregated features refer to features obtained by fusing features from different time phases. By introducing an attention mechanism, greater attention and weight are given to areas of drastic change to obtain feature information more conducive to landslide identification. See also Figure 4 In this embodiment, the calculation function expression for the aggregation feature is: , In the above formula, As an aggregation feature, For channel stacking, For Hadama product, For attention feature maps, and These are the high-dimensional features of the SAR images before and after the earthquake at this scale, and the function expression for extracting the attention feature map is: , In the above formula, For the Sigmoid function, Indicates similarity Adding dimensions, similarity High-dimensional features of SAR images before and after the earthquake at this scale and The similarity between them.

[0029] Similarity can be calculated using the desired method, for example, as an optional implementation, in this embodiment, high-dimensional features of SAR images before and after the earthquake at this scale. and similarity between The expression for the computation function is: , In the above formula, and respectively and The corresponding element in the i-th dimension, and These are the high-dimensional features of the SAR images before and after the earthquake at this scale. for and The dimension. In this embodiment, the specific operation of calculating the aggregated features is as follows: First, the cosine similarity of the two temporal feature matrices is calculated to obtain the similarity information of the two images. This method is used to measure the similarity between two vectors in a multidimensional space. Then, the obtained similarity features are transformed into attention feature maps through dimensionality enhancement and the Sigmoid function. The generated attention feature map Size is , and These represent the height and width of the image, respectively. The features extracted before and after the earthquake are then stacked, and attention feature maps are used. The extraction of aggregated features is completed. The Multi-Feature Cross-Fusion (MFCM) module processes the above steps to obtain four sets of differential features from four sets of features at different resolutions. and aggregation features Let them be denoted as {D1,A1}, {D2,A2}, {D3,A3}, and {D4,A4}, for example... Figure 1 As shown, D1 to D4 are four differential features. A1 to A4 are four aggregation features. .

[0030] The U-Net decoding structure combines multi-resolution features extracted by the encoder through skip connections, fully utilizing semantically rich high-dimensional features while considering the details and texture information of low-dimensional features, enabling the network to better capture details and boundary information in change detection tasks. However, in semantic segmentation tasks, features extracted at different levels during the encoding stage have varying importance. Inspired by UNet++ and SNUNet, and combining the characteristics of cross-level connections, this embodiment utilizes a multi-feature cross-fusion module (MFCM) to fuse differential and aggregated features of earthquakes and landslides into a fused feature map. During the decoding process, differential features at different resolutions extracted by the MFCM are added layer by layer. Figure 1 and Figure 5 As shown, Figure 5 middle, and These represent the differential and aggregated features extracted by the DAG module, respectively, and the subscripts in the DAG module are... n These represent feature dimensions at different resolutions. In this embodiment, when using the Multi-Feature Cross-Fusion Module (MFCM) to fuse the differential and aggregated features of earthquakes and landslides into a fused feature map, for each scale, fusing the differential and aggregated features at that scale into a fused feature map includes: firstly, extracting the tensor matrix for each dimension according to the following formula: , In the above formula, For the first i The first dimension j Tensor matrices, For the first Aggregated features in multiple dimensions For the first i The system identifies the differential features in each dimension; then, the tensor matrices of each dimension are upsampled layer by layer using deconvolution and then stacked through channels to obtain a fused feature map, such as... Figure 1 As shown, this includes: hopping D4 to A4, upsampling it using deconvolution, and then hopping it to D3 to obtain the fused feature Y. 3,1 , will fuse feature Y 3,1 The fused feature Y is obtained by upsampling via deconvolution and then skipping connections with D3. 2,2 , will fuse feature Y 2,2 The fused feature Y is obtained by upsampling via deconvolution and then connecting it with a D1 skip connection. 1,3 The fused feature Y is obtained by skipping D3 to A3, upsampling it using deconvolution, and then skipping it to D2. 2,1 , will fuse feature Y 2,1 The fused feature Y is obtained by upsampling via deconvolution and then connecting it with a D1 skip connection. 1,2 The fused feature Y is obtained by skipping D2 to A2, upsampling it using deconvolution, and then skipping it to D1. 1,1 , fuse A1 and feature Y 1,1 Fusion feature Y 1,2 and fusion feature Y 1,3 Channel stacking yields a fused feature map used as a convolutional upsampling module.

[0031] In this embodiment, the decoding structure of the Multi-Feature Cross-fusion Module (MFCM) can better identify landslides at different scales and adaptively adjust the weight ratio of features at different scales to the prediction results. This cross-level connection method effectively fuses low-level and high-level features, improving the stability and robustness of the model while solving the feature loss problem. It enhances segmentation accuracy by gradually fusing richer semantic information and changing features while preserving high-resolution features. The MFCM upsampling operation is implemented by deconvolution, which upsamples the low-resolution feature map. Compared to upsampling using traditional interpolation, deconvolution has stronger learning capabilities. It can adjust the weights of the convolution kernel through backpropagation, allowing the model to automatically learn an upsampling method more suitable for a specific task. This enables the model to adaptively upsample according to the characteristics of the data, thereby improving the model's representation ability and performance. Furthermore, during the upsampling process, it can better capture the spatial relationships between features, achieving feature fusion and helping to retain more semantic information, thus improving the quality and accuracy of the upsampled image. The upsampled feature map and the transformed feature map are fused through cross-level links, thereby preserving global and local contextual information.

[0032] In this embodiment, a convolutional upsampling module is used to process the fused feature map (size: ...). The landslide recognition results are obtained by sequentially performing convolutional upsampling, including: using a 3×3 convolution module and bilinear interpolation to perform pixel binary classification and 4x upsampling on the fused feature map to obtain landslide recognition results with the same size as the original image input to the network.

[0033] This embodiment presents an experimental study on the numerous landslides triggered by a magnitude 7.5 thrust earthquake that occurred in Papua New Guinea in February 2018. Sentinel-1 data was used, with a 5m... The experimental study area used SAR images stitched together from Sentinel-1 data. First, Sentinel-1 data and Sentinel-2 remote sensing images of the Papua New Guinea earthquake landslide area were registered. Then, the SAR images were matched with the landslide catalog in ArcGIS software. Next, the pre-earthquake and post-earthquake SAR images and their corresponding landslide catalogs were cropped into 512×512 pixel samples. A validation set was created by dividing the training set into a 10:1 ratio. Data augmentation was performed on the training samples, including horizontal and vertical flipping, 90-degree clockwise rotation, 180-degree clockwise rotation, and 90-degree counter-clockwise rotation—a total of five data augmentation methods—to generate the final dataset. In addition, a portion of the landslide area was reserved as an independent test area and not included in the dataset creation. To evaluate the performance of the proposed method in this embodiment, precision (…) was used. Recall, F1 score, and overall accuracy. The model uses the mean Intersection over Union (mIoU) ratio as evaluation metrics. Precision is the proportion of samples that the model predicts to be positive but are actually positive. This represents the proportion of samples that are truly positive and are correctly predicted as positive by the model. F1 score is a comprehensive evaluation metric that considers both precision and recall. It describes the proportion of samples correctly predicted by the model out of the total number of samples. mIoU measures the overlap between the predicted results and the true labels, used to measure segmentation accuracy. The calculation functions for the above evaluation metrics are shown in the following formulas: , , , , , In the above formula, A true positive is the number of pixels that are correctly predicted as a landslide. A true negative is the number of pixels correctly predicted as background. This is a false positive, meaning the number of pixels incorrectly predicted as a landslide. The number of pixels that are incorrectly predicted as negative is called a false negative. This indicates the category of the target. In this embodiment, the experiments were implemented using the PyTorch framework. The software environment was PyTorch 1.7.1, CUDA 11.3, and Python 3.8; the hardware environment was a single NVIDIA RTX 3090 GPU. All experiments used a random number seed of 20, employed the Adam optimization algorithm, set the initial learning rate to 0.0001, and used the ReduceOnPlateau learning rate strategy with a Patience parameter of 15. The batch size for training the model was set to 16, and all samples were trained for 200 epochs. Validation was performed after each training epoch, and the optimal model weights were selected based on the F1-Score of the validation results.

[0034] To verify the effectiveness of the proposed method in this embodiment, comparative experiments were conducted with earlier fusion semantic segmentation methods such as DeepLabV3+ and UNet, as well as later fusion methods such as ChangeNet, SNUNet, and FC-Siam-diff networks. The experimental results are as follows: Figure 6 and Figure 7 As shown, the white area represents the landslide area, the red area represents false landslide warnings, and the blue area represents missed landslide warnings.

[0035] Figure 6 In the image, (a) is a post-earthquake SAR image; (b) is a landslide catalog; (c) is the DeepLabV3+ landslide identification result; (d) is the U-Net landslide identification result; (e) is the ChangeNet landslide identification result; (f) is the SNU-Net landslide identification result; (g) is the FC-Siam-diff landslide identification result; and (h) is the DACS-Net landslide identification result. The landslide catalog ( Figure 6 As shown in (b)), the landslide area in this scenario is small and the locations are relatively scattered. Due to the low resolution of the Sentinel-1 system (approximately 5m), (20m), many landslides are only a few dozen pixels in SCL imagery. This requires the network to have good detail feature extraction capabilities. The detection results show that ChangeNet's detection performance is poor. Figure 6 In (e) of the data, there are large areas of false alarms and missed alarms, and the ability to extract details of landslide features is poor. DeepLabV3+ network ( Figure 6(c) and U-Net network ( Figure 6 The detection results (d) in the SNU-Net network show small differences, and the performance in extracting the boundary areas and details of landslides is not ideal, resulting in many false alarms. Figure 6 (f) and FC-Siam-diff network ( Figure 6 The two networks (g) in the above examples show relatively good detection performance, but the FC-Siam-diff network has more false alarms in the lower right corner area, while SNU-Net has more false alarms. The DACS-Net proposed in this example has significantly better detection results than other networks. Although there are still some false alarms in some small landslides, the shape and location of most landslide areas are closer to the landslide catalog, indicating that this network has better ability to extract detailed features than other networks.

[0036] Figure 7 The images show the landslide identification results for Scene II, where (a) is a post-earthquake SAR image; (b) is a landslide catalog; (c) is the DeepLabV3+ landslide identification result; (d) is the U-Net landslide identification result; (e) is the ChangeNet landslide identification result; (f) is the SNU-Net landslide identification result; (g) is the FC-Siam-diff landslide identification result; and (h) is the DACS-Net landslide identification result. Figure 6 Unlike other regions, the landslides in this area are relatively concentrated, but their structure and background are more complex, thus requiring the network to have better global semantic information extraction capabilities. The results show that the ChangeNet network (… Figure 7 The landslide identification result for (e) was the worst, with a large area of ​​false alarms. DeepLabV3+ network ( Figure 7 (c) in the U-Net network also has a relatively large number of false alarms. Figure 7 While the false alarm rate (d) in the SNUNet network is lower than that of the DeepLabV3+ network, the false alarm rate is significantly higher. Figure 7 (f) and FC-Siam-diff network ( Figure 7 The landslide identification results (g) in the example are similar, with numerous false alarms and missed detections of varying scales occurring at various locations. In contrast, the DACS-Net network ( Figure 7 The (h) network achieved the best results in detail extraction, and the recognition results had clear boundaries. The number of missed detections and false detections were much less than other networks, indicating that the network has a stronger ability to extract global semantic information.

[0037] according to Figure 6 and Figure 7The landslide recognition experiments in the two scenarios show that different network models produce significantly different detection results in the same area. To better analyze the results, Table 1 presents the average evaluation metrics for the two landslide recognition experiments.

[0038] Table 1 Different network detection indicators

[0039] Table 1 clearly shows that SNUNet and FC-Siam-diff achieved good performance on the F1 score, with little difference in accuracy and recall. ChangeNet performed poorly on all metrics except recall. Combined with the graph detection results, it can be seen that while this network achieves higher recall due to large-area false positives, its accuracy decreases. It has many problems in target localization, handling of target size variations, and background regions; it also has many false positives when identifying target categories, misclassifying some background regions as slippery areas, indicating significant room for improvement in background false positive elimination. The DeepLabV3+ and U-Net networks, using two early fusion methods, achieved similar results in overall accuracy and F1 score, but both suffered from many false positives leading to low accuracy. This suggests that in the early fusion methods, the data from the two time phases were fused before entering the network, and the features from different time phases may interfere with each other during feature extraction, resulting in poor detection performance. The proposed DACS-Net in this embodiment demonstrates significant superiority across several key metrics. Its Precision metric reaches 70.75%, far exceeding other networks, indicating highly accurate identification and prediction of positive samples. The F1 score, a harmonic average of Precision and Recall, also achieves the best result at 65.46%, demonstrating a better balance between prediction accuracy and positive sample capture capability, exhibiting excellent performance in capturing positive samples. Furthermore, DACS-Net demonstrates superior mIoU capability, reaching 69.23%, reflecting its excellent performance in terms of overlap between true and predicted results, resulting in fewer false alarms. Simultaneously, the network achieves a detection metric of 91% in overall accuracy (OA), emphasizing its high accuracy and stability in overall prediction. Overall, DACS-Net performs better in landslide recognition. The Siamese network, which adopts a late-stage fusion approach, makes full use of the feature information extracted by the network at different stages, significantly increasing the diversity and expressive power of features. In addition, the network's feature of densely connecting and fusion differential and aggregated features in the decoder also brings better global semantic information learning ability and detailed feature extraction ability.

[0040] In summary, this embodiment proposes a deep learning-based DACS-Net network to address the complex and challenging problem of landslide identification from SAR images. This network achieves automatic and high-precision identification of landslides from SAR images. Using SAR images from both pre- and post-earthquake phases as input to the network model, a twin-structure backbone network is employed as the encoder to extract landslide features from different time phases. An MDAG module is built in the decoder to perform differential extraction and feature aggregation of features captured by the encoder at four scales across different time phases. Furthermore, a designed MFCM module enables cross-level connections to fully fuse features from different scales, finally restoring the original image size to obtain the landslide identification result. Experimental results show that DACS-Net has a significant advantage in landslide identification results compared to the other five networks. This embodiment combines SAR image-based landslide identification with deep learning algorithms and twin networks, and optimizes for differential features and multi-scale issues, providing a new approach for other researchers to conduct landslide detection tasks. Although the method in this embodiment may miss some smaller landslides, when using higher resolution SAR image data and with the help of methods such as transfer learning, the DACS-Net network proposed in this embodiment can obtain better landslide identification results, playing an important role in providing first-hand information support for emergency rescue after an earthquake.

[0041] Furthermore, this embodiment also provides an earthquake landslide identification system based on a multi-feature cross-fusion twin network, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the earthquake landslide identification method based on the multi-feature cross-fusion twin network. Additionally, this embodiment also provides a computer-readable storage medium storing a computer program for being programmed or configured by the microprocessor to execute the earthquake landslide identification method based on the multi-feature cross-fusion twin network.

[0042] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0043] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying earthquake landslides based on multi-feature cross-fusion twin networks, characterized in that, This includes using a multi-feature cross-fusion twin network to identify earthquake landslides based on SAR images before and after an earthquake in the same scene. The multi-feature cross-fusion twin network includes an encoder and a decoder. The encoder includes two feature extractors with twin structures and shared parameters. The decoder includes a multi-scale differential feature and aggregate feature generation module (MDAG), a multi-feature cross-fusion module (MFCM), and a convolutional upsampling module. The multi-scale differential feature and aggregate feature generation module (MDAG) includes multiple differential feature and aggregate feature generation modules (DAG) that correspond one-to-one with the scale of the high-dimensional features. The earthquake-landslide identification process includes: using two feature extractors to extract high-dimensional features at multiple scales from SAR images of the same scene before and after an earthquake; and using a differential feature and aggregation feature generation module (DAG) to extract high-dimensional features from SAR images of the same scene before and after an earthquake at corresponding scales. and The differential and aggregated features of earthquake-induced landslides are extracted, where the differential features refer to the high-dimensional features of SAR images before and after the earthquake at this scale. and The differences between them are aggregated features, which are high-dimensional features of SAR images before and after the earthquake at this scale. and The features of earthquakes and landslides are aggregated and then fused into a fused feature map using the Multi-Feature Cross-Fusion Module (MFCM). The fused feature map is then sequentially convolutionally upsampled using a convolutional upsampling module to obtain the landslide identification result. The calculation function expression for the aggregation feature is: ， In the above formula, As an aggregation feature, For channel stacking, For Hadama product, For attention feature maps, and These are the high-dimensional features of the SAR images before and after the earthquake at this scale, and the function expression for extracting the attention feature map is: ， In the above formula, For the Sigmoid function, Indicates similarity Adding dimensions, similarity High-dimensional features of SAR images before and after the earthquake at this scale and The similarity between them; When using the Multi-Feature Cross-Fusion Module (MFCM) to fuse the differential and aggregated features of earthquakes and landslides into a fused feature map, for each scale, fusing the differential and aggregated features into a fused feature map at that scale includes: firstly, extracting the tensor matrix for each dimension according to the following formula: ， In the above formula, For the first i The first dimension j Tensor matrices, For the first Aggregated features in multiple dimensions For the first i The system identifies the differences in features across multiple dimensions; then, it stacks the tensor matrices with the largest scale and the aggregated feature A1 through channels to obtain a fused feature map.

2. The earthquake landslide identification method based on multi-feature cross-fusion twin network according to claim 1, characterized in that, The calculation function expression for the differential feature is as follows: ， In the above formula, To highlight the differences, a and b represent SAR images of the same scene before and after an earthquake. For matrix subtraction, and These are the high-dimensional features of SAR images before and after the earthquake at this scale.

3. The earthquake landslide identification method based on multi-feature cross-fusion twin network according to claim 1, characterized in that, The high-dimensional features of SAR images before and after the earthquake at this scale and similarity between The expression for the computation function is: ， In the above formula, and respectively and The corresponding element in the i-th dimension, and These are the high-dimensional features of the SAR images before and after the earthquake at this scale. for and Dimensions.

4. The earthquake landslide identification method based on multi-feature cross-fusion twin network according to claim 1, characterized in that, The process of obtaining landslide recognition results by sequentially performing convolutional upsampling on the fused feature map using the convolutional upsampling module includes: performing pixel binary classification and 4x upsampling on the fused feature map using a 3×3 convolution module and bilinear interpolation to obtain landslide recognition results with the same size as the original image input to the network.

5. The earthquake landslide identification method based on multi-feature cross-fusion twin network according to claim 1, characterized in that, The feature extractor is a ResNet50 network.

6. An earthquake landslide identification system based on a multi-feature cross-fusion twin network, comprising an interconnected microprocessor and a memory, characterized in that, The microprocessor is programmed or configured to execute the earthquake landslide identification method based on multi-feature cross-fusion twin network as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, The computer program is used to be programmed or configured by a microprocessor to execute the earthquake landslide identification method based on multi-feature cross-fusion twin network as described in any one of claims 1 to 5.

Images