A landslide detection method based on UNet semantic segmentation dual-branch improvement

By adopting the UNet semantic segmentation dual-branch improvement method in landslide detection, combining boundary adaptive features and multi-scale feature extraction module, the problem of insufficient landslide detection accuracy in the existing technology is solved, and higher detection accuracy and recall rate are achieved , suitable for landslide warning and railway traffic safety guarantee.

CN118608791BActive Publication Date: 2024-11-26CHINA RAILWAY DESIGN GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411067700.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-26
Estimated Expiration
2044-08-06

AI Technical Summary

Technical Problem

The existing landslide detection algorithm based on deep learning failed to effectively optimize the model design, resulting in insufficient accuracy when detecting features with different scales, diverse shapes and complex edges, and cannot meet the requirements of complex edge extraction and information extraction at different scales.

Method used

Using the improved landslide detection method based on UNet semantic segmentation, the boundary adaptive feature branches and multi-scale feature branches are designed, and the feature map of the input image is obtained through the boundary adaptive feature extraction module and the multi-scale feature extraction module, and the feature map of the input image is obtained through the UNet structure. The deconvolution group is decoded, and finally the landslide detection recognition results are obtained through the LogSoftmax function.

Benefits of technology

The ability to distinguish the characteristics of landslides with diverse morphology and complex edges is improved, the detection accuracy and robustness of the model for landslides of different scales is enhanced, and the accuracy, recall and F1 value of the model are significantly improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118608791B_ABST
    Figure CN118608791B_ABST
Patent Text Reader

Abstract

The invention discloses a landslide detection method based on UNet semantic segmentation double branch improvement, comprising: S1, obtaining high-resolution satellite remote sensing images of the study area; S2, preprocessing the high-resolution satellite remote sensing images to obtain satellite remote sensing images after data enhancement; S3, making a semantic segmentation sample set through visual interpretation; S4, designing a landslide detection model based on UNet semantic segmentation double branch improvement, the landslide detection model is composed of a boundary adaptive feature branch, a multi-scale feature branch and a conventional deconvolution group of UNet; S5, training, testing and verifying the model; S6, applying the landslide detection model obtained in S55 to landslide potential area detection to obtain landslide detection results. The method improves the ability to distinguish the difference between the target and the background, expands the receptive field, better meets the information extraction requirements of targets at different scales, and has higher accuracy and recall rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a landslide detection method based on UNet semantic segmentation double-branch improvement. Background Art

[0002] Landslide is a common and frequent geological disaster that may cause damage to railway lines, obstruction of train operation and even major accidents, resulting in serious casualties and property losses. Therefore, landslide detection is of vital importance to railway transportation and track maintenance. Field investigation can effectively obtain landslide information, but it consumes a lot of manpower, material and financial resources and has poor timeliness.

[0003] By using advanced technical means, such as remote sensing monitoring and geological radar, large-scale landslides can be monitored and warned, and the location of landslides and the affected area can be detected in a timely and accurate manner. This can effectively prevent and respond to potential safety risks, ensure the safety and smooth flow of railway traffic, improve emergency response speed, minimize disaster losses, and ensure the stable operation of railway traffic and tracks.

[0004] As deep learning technology gradually matures, some scholars have introduced it into the field of landslide detection to achieve automated detection. However, the existing landslide detection algorithms based on deep learning have not optimized the model design based on the characteristics of landslides with different scales, diverse shapes, and complex edges, and there is room for improvement in detection accuracy. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a landslide detection method based on UNet semantic segmentation dual branches with strong distinguishing ability and higher accuracy.

[0006] To this end, the present invention adopts the following technical solutions:

[0007] A landslide detection method based on UNet semantic segmentation dual-branch improvement includes the following steps:

[0008] S1, obtain high-resolution satellite remote sensing images of the study area;

[0009] S2, preprocessing the high-resolution satellite remote sensing image to obtain a satellite remote sensing image after data enhancement;

[0010] S3, making semantic segmentation sample sets through visual interpretation: based on expert knowledge, screen the areas containing landslide phenomena in the images preprocessed by S2, visually identify the landslide areas and background on the images, and use annotation tools to annotate the landslide areas pixel by pixel; then randomly assign the obtained image samples to the training set, test set, and validation set;

[0011] S4, designing an improved landslide detection model based on UNet semantic segmentation dual branches: the landslide detection model consists of a boundary adaptive feature branch, a multi-scale feature branch and a conventional deconvolution group of UNet;

[0012] In the encoding part of the model, a feature map of the landslide in the input image is obtained through multiple serially connected boundary adaptive feature extraction modules BAF-1~n and multiple serially connected multi-scale feature extraction modules MSF-1~n; the feature map is divided into n different levels according to its scale, and the length and width of the feature map of each level are half of the length and width of the feature map of the previous level, and the number of channels is twice that of the previous level; the features output by BAF-1 are used as the input of BAF-2, and the features output by BAF-2 are used as the input of BAF-3, and so on; the output features of BAF-1 are accumulated with the calculation results of MSF-1, and the accumulated results are used as the output of MSF-1, and the output of MSF-1 is used as the input of MSF-2; the output features of BAF-2 are accumulated with the calculation results of MSF-2, and the accumulated results are used as the output of MSF-2, and the output of MSF-2 is used as the input of MSF-3, and so on;

[0013] In the decoding part, DeConv-n in the UNet structure deconvolution group is used to restore the output of MSF-n to the size of the feature map of the previous level, and the output features of MSF-(n-1) are accumulated. The accumulated result is used as the input of DeConv-(n-1). This process is iterated until the output feature map of DeConv-1 in the UNet structure deconvolution group is consistent with the size of the input image. Finally, the landslide detection and recognition result is obtained through the LogSoftmax function.

[0014] S5, training the landslide detection model based on the image samples to obtain the model weight, includes the following steps:

[0015] S51, inputting the samples of the training set into the landslide detection model designed in S4, extracting the boundary adaptive features and multi-scale features thereof respectively, and performing information interaction between the encoding process and the decoding process to obtain a landslide prediction map;

[0016] S52, based on the landslide prediction map obtained in step S51 and the pixel-by-pixel labeled image in S3, the loss function is calculated, and then the model parameters are optimized by back propagation gradient calculation to gradually reduce the loss function. During the training process, the validation set obtained in S3 is used to evaluate the performance of the model, to help select the best hyperparameters and model structure, and to prevent the model from overfitting on the training set; the weight of the model with the best performance is saved as the weight of the training output;

[0017] S53, using the test set obtained in S3 to evaluate the generalization ability of the model optimized in S52 on unseen data, and obtaining indicators such as the model's precision and recall rate, and comparing the performance with other existing methods to determine the reliability of the landslide detection model;

[0018] S6, applying the landslide detection model obtained in S53 to landslide potential area detection to obtain landslide detection results.

[0019] In the above step S4, for each boundary adaptive feature extraction module BAF-n, first perform 1×1 convolution on its input features to reduce the number of channels to 1 / 2 of the input features; then perform deformable convolution DFConv to extract features; for each position of the input feature map, first predict the extent to which each position of the feature map should be offset, generate an offset map, and then adjust the sampling position of the convolution kernel according to the predicted offset, add the sampling position of the convolution kernel to the offset, and obtain the actual sampling position of the convolution kernel at the position to generate a sampling grid; then perform weighted summation on the weights of the input feature map and the convolution kernel at the actual sampling position to obtain the corresponding position of the output feature map; the interval of the convolution operation is regularized by BatchNorm to convert the feature distribution interval to the range of [0, 1], and add the GELU function for normalization;

[0020] Repeat the above deformable convolution DFConv, BatchNorm regularization and GELU function normalization process; then restore the channel through 1×1 convolution to be consistent with the input features;

[0021] The output features at this time are accumulated with the input features of the boundary adaptive feature extraction module, and the accumulated results are subjected to a conventional convolution group operation; finally, the features are output to the boundary adaptive feature extraction module BAF-(n+1).

[0022] In the above step S4, for each multi-scale feature extraction module MSF-n, a 3×3 convolution with a step size of 2 is first performed on its input feature to complete downsampling, and then a conventional convolution group operation is performed to obtain a feature map F1 ; Perform dilated convolution on the results of the conventional convolution group with step size s={1, 2, 4, 7} respectively, and add the convolution results together to output and obtain the feature map F2 ;

[0023] Then, the multi-head attention mechanism is used to calculate the learnable weights,, and the feature map F1 , F2 To perform the product, the formula is as follows:

[0024] ;

[0025] in, for Dimensions;

[0026] Finally, the calculation results of the multi-head attention mechanism are accumulated with the output of BAF-n as the output of MSF-n.

[0027] In the DeConv-n of step S4, two conventional convolution group operations are first performed to obtain the feature map, and then transposed convolution, batch normalization, and GELU activation function operations are performed. The scale of the feature map is expanded to twice the input through transposed convolution and the number of channels is reduced by half.

[0028] The above conventional convolution group operations include conventional convolution operations, BatchNorm regularization, and GELU activation function operations.

[0029] The pre-processing described in step S1 includes:

[0030] Cropping is used to extract or adjust the area of ​​interest in the remote sensing satellite images obtained by S1 to reduce the amount of data;

[0031] Radiometric calibration is used to convert pixel values ​​in the cropped digital image into surface reflectance or radiant brightness to accurately represent the characteristics of the surface of the object;

[0032] Geometric correction, which accurately matches the image pixels in the radiometrically calibrated digital image to the actual position on the earth's surface to eliminate geometric distortion;

[0033] The steps of radiation calibration are: firstly, atmospheric correction is performed on the cropped digital image to eliminate the influence; then, the surface reflectance or radiation brightness of each pixel is calculated according to the radiation calibration parameters and the image data after atmospheric correction; finally, the result is compared with the ground field data to verify and adjust the result;

[0034] The steps of geometric correction are: first, select some objects with known geographical coordinates as control points on the digital image, then use the geographical coordinates of the control points and the pixel coordinate information of the corresponding positions on the image to perform image registration, and finally perform projection transformation on the image and project it into the required map projection system.

[0035] Preferably, in step S3, data enhancement operation is further performed on the labeled samples to generate more samples, and the samples are randomly allocated to the training set, the test set and the validation set in a ratio of 6:1:1.

[0036] Preferably, in S4, the n is 5; the channels are 32, 64, 128, 256 and 512 respectively;

[0037] Preferably, in S51, the interval of the convolution operation is regularized by BatchNorm, so that the feature distribution interval is converted to the range of [0, 1], which speeds up the process of gradient descent fitting the optimal model, and adds the GELU function for normalization to enhance the nonlinear characteristics of the model; then the channel is restored through 1×1 convolution to be consistent with the input feature, and the output feature is accumulated with the input feature.

[0038] Preferably, in S52, the output scale of MSF-1 is 160×160×32; for other multi-scale feature extraction modules, the input features are first downsampled, the feature maps are output and a conventional convolution group operation is performed, and then the dilated convolutions are performed with the dilated parameters d={1, 2, 4, 7} respectively, and the convolution results are accumulated and output together; then the extracted results are bilinearly interpolated, , , is a learnable weight, which is multiplied by the feature and then input into the multi-head attention mechanism for calculation; finally, the output of the attention mechanism is accumulated with the output of the corresponding boundary adaptive feature extraction module.

[0039] Preferably, in S53, the encoding features and the deep decoding features are first accumulated to achieve information fusion of features at different levels; then, the feature map is obtained through two conventional convolution group processes, and then transposed convolution and batch normalization plus GELU activation function operations are performed.

[0040] The present invention optimizes the network model by designing a substructure based on the features of landslides of different scales, diverse shapes, and complex edges, and finally forms the improved landslide detection method (MAS-UNet) based on UNet semantic segmentation double branches of the present invention. The method uses satellite remote sensing as the data source, uses artificial intelligence visual algorithms to intelligently identify landslides, builds a landslide detection model and performs pre-training, inputs images into the model, and outputs pixel-by-pixel landslide detection results. The method is used for landslide early warning to prevent railway lines from being blocked by landslides, and can ensure the safety and smoothness of railway traffic.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The present invention designs a boundary adaptive feature branch in view of the diverse morphology and complex edges of landslide areas. The sampling position is adaptively adjusted according to the shape and structure of the target in the image, thereby improving the ability to distinguish the difference between the target and the background, thereby better meeting the extraction of complex edges.

[0043] (2) The present invention designs a multi-scale feature branch in view of the different scales of landslide areas, expands the receptive field, and obtains the feature values ​​of spatial neighborhood points in a larger range; the present invention can adjust the analysis receptive field through scale changes, better meet the information extraction requirements of targets at different scales, and make the model more robust and accurate when dealing with landslides of different scales;

[0044] (3) The present invention is based on the UNet network structure and incorporates a designed dual-branch feature extractor. Compared with existing methods, this method has higher precision, recall rate and F1 value. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of the landslide detection method based on UNet semantic segmentation double branch improvement of the present invention;

[0046] Figure 2 It is the structure diagram of the algorithm model MAS-UNet in the present invention;

[0047] Figure 3 yes Figure 2 Detailed structure diagram of the middle boundary adaptive feature extraction module;

[0048] Figure 4 yes Figure 2 Detailed structure diagram of the multi-scale feature extraction module;

[0049] Figure 5 yes Figure 2 Detailed structural diagram of the deconvolution group in the figure;

[0050] Figure 6 It is the result of detecting small landslides using the existing method and the method of the present invention;

[0051] Figure 7 It is the result of detecting large landslides using the existing method and the method of the present invention;

[0052] Figure 8 It is the result of detecting a large landslide with vegetation interference using the existing method and the method of the present invention. DETAILED DESCRIPTION

[0053] The landslide detection method of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0054] like Figure 1 As shown, the improved landslide detection method based on UNet semantic segmentation dual branches (MAS-UNet) of the present invention includes the following steps:

[0055] S1, obtain high-resolution satellite remote sensing images of the study area.

[0056] Determine the geographic scope of the specific study area for which remote sensing images are required, which can be defined using latitude and longitude coordinates or maps. Then, download or order high spatial resolution optical satellite remote sensing images of the study area, which are two-dimensional visible light band color images or digital images.

[0057] S2, preprocessing the high-resolution satellite remote sensing image, including cropping, radiometric calibration, geometric correction, etc. The details are as follows:

[0058] Cropping is to extract or adjust the area of ​​interest in the original satellite image and remove unnecessary parts for subsequent processing. Cropping can reduce the amount of data and improve processing efficiency.

[0059] Radiometric calibration is to convert the pixel values ​​in the cropped digital image into surface reflectance or radiant brightness in order to accurately represent the surface characteristics of the object. First, atmospheric correction is performed to eliminate the influence; then the surface reflectance or radiant brightness of each pixel is calculated based on the radiometric calibration parameters and the image data after atmospheric correction; finally, the results are compared with the ground field data to verify and adjust the results.

[0060] Geometric correction is to accurately correspond the image pixels after radiometric calibration to the actual position on the earth's surface to eliminate geometric distortion caused by factors such as the earth's surface and sensor imaging geometric characteristics. First, it is necessary to select some objects with known geographical coordinates on the image as control points, and then use the geographical coordinates of the control points and the pixel coordinate information of the corresponding position on the image to perform image registration, and finally perform projection transformation on the image and project it into the required map projection system.

[0061] S3, produces semantic segmentation sample sets through visual interpretation.

[0062] Select ArcGIS, QGIS, LabelImg and other tools suitable for visual interpretation and annotation. According to expert knowledge, screen the areas containing landslide phenomena in the S2 preprocessed images, visually identify the landslide areas and background on the images, and use annotation tools to annotate the landslide areas pixel by pixel. In order to increase the diversity and quantity of training samples, data enhancement operations such as rotation, flipping, scaling, etc. can be performed on the annotated samples to generate more samples. The samples are randomly assigned to the training set, test set and validation set in a ratio of 6:1:1.

[0063] S4, design an improved landslide detection model based on UNet semantic segmentation dual branches (MAS-UNet).

[0064] The model consists of a boundary adaptive feature branch (BAFB), a multi-scale feature branch (MSFB) and a conventional upsampling convolution group (deconvolution group) of UNet. The boundary adaptive feature extraction module can analyze the positional relationship and spatial structure between pixels, provide information such as landslide shape, boundary contour, geometric constraints, etc., to help better segment and detect images. The output features of this module are called boundary adaptive features.

[0065] In the encoding part of the model, a feature map of the landslide in the input image is obtained through multiple serially connected boundary adaptive feature extraction modules BAF-1~n and multiple serially connected multi-scale feature extraction modules MSF-1~n. According to the scale of the feature map, it is divided into n different levels, the size of the output feature map of each level is marked on the map, and the appropriate channel is determined. The length and width of the feature map of each level are half of the length and width of the feature map of the previous level; the images in the training set are respectively input into the first boundary adaptive feature extraction module BAF-1 and the first multi-scale feature extraction module MSF-1, and the output features of BAF-1 and the output features of MSF-1 are accumulated as the input feature map of MSF-2, and so on.

[0066] In the decoding part, DeConv-n in the UNet structure deconvolution group is used to restore the output of MSF-n to the size of the feature map of the previous level, and accumulate it with the output features of MSF-(n-1). The accumulated result is used as the input of DeConv-(n-1). This process is iterated until the output feature map of DeConv-1 in the UNet structure deconvolution group is consistent with the size of the input image, and finally the landslide detection and recognition result is obtained through the LogSoftmax function.

[0067] In one embodiment of the present invention, the overall network structure is as follows: Figure 2 As shown in the figure, according to the scale of the feature map, it is divided into 5 different levels, and the size of the output feature map of each level is marked on the figure, and the channels are 32, 64, 128, 256 and 512 respectively.

[0068] Figure 3 for Figure 2 As shown in the figure, taking BAF-2 as an example, we first perform a 1×1 convolution on its input feature (i.e., the output of BAF-1, with a scale of 160×160×32) to reduce the number of channels to 1 / 2 of the input feature. At this time, the input feature becomes 160×160×16.

[0069] Then, 3×3 deformable convolution (DFConv) is performed to extract features. For each position of the input feature map, the offset of each position of the feature map is calculated to generate an offset map, and then the sampling position of the convolution kernel is adjusted according to the calculated offset. The sampling position of the convolution kernel is added to the offset to obtain the actual sampling position of the convolution kernel at this position, and the sampling grid is generated; then, the weights of the input feature map and the convolution kernel at the actual sampling position are weighted summed to obtain the corresponding position of the output feature map. The introduction of position variability through deformable convolution (DFConv) can better adapt to the edge deformation of the landslide object, thereby improving the model's processing performance for the complex structure and irregular morphological characteristics of the landslide. The interval of the convolution operation is regularized by BatchNorm (BN) to convert the feature distribution interval to the range of [0, 1], accelerate the process of gradient descent fitting the optimal model, and add the GELU function for normalization to enhance the nonlinear characteristics of the model.

[0070] Repeat the above process of deformable convolution, BatchNorm regularization and GELU function normalization. Then restore the channel to be consistent with the input features through 1×1 convolution.

[0071] The output features at this time are accumulated with the input features of the module, and the accumulated results are input into the regular convolution group, that is, 3×3 regular convolution with a step size of 2, BatchNorm regularization and GELU function normalization.

[0072] Finally, after the above conventional convolution group operation, the features (80×80×64) are output.

[0073] The feature extraction operation process of other boundary adaptive feature extraction modules is the same as that of BAF-2.

[0074] This not only ensures that the network has sufficient information for feature extraction, but also reduces the overall number of network parameters. The feature accumulation mechanism reduces the residual between the learned feature value and the true value during module training, alleviates gradient explosion, reduces gradient vanishing, and improves feature extraction capabilities.

[0075] Figure 4 Shown Figure 2 Taking MSF-2 as an example, the input feature (that is, the output of MSF-1, the scale is also 160×160×32) is first convolved with a step size of 2 and a 3×3 convolution to complete the downsampling. The output feature map scale is 80×80×64, and a regular convolution group operation (including regular convolution operation, BatchNorm regularization and GELU activation function) is performed to obtain the feature map F1 ; Perform dilated convolution on the results of the conventional convolution group with step size s={1, 2, 4, 7} respectively, and add the convolution results together to output and obtain the feature mapF2 , thus effectively increasing the receptive field.

[0076] Then, the learnable weights are calculated through the multi-head attention mechanism , , Multiply it with the above output features, the formula is as follows:

[0077] ;

[0078] in, for Dimension.

[0079] Finally, the calculation results of the multi-head attention mechanism are accumulated with the output of BAF-2. The output features at this time include both rich spatial context information and detailed boundary details. The output features of this module are multi-scale features.

[0080] The feature extraction operation process of other multi-scale feature extraction modules is the same as that of MSF-2.

[0081] Figure 5 Shown Figure 2 The detailed structure of the deconvolution group DeConv in the figure. As shown in the figure, taking DeConv-2 as an example, the accumulation of encoding features (output of MSF-2) and deep decoding features (output of DeConv-3) is first implemented to achieve information fusion of features at different levels, which helps the network structure learn richer feature representations and improves the model's ability to understand input data. Subsequently, it is processed by two conventional convolution groups (including conventional convolution operations, BatchNorm regularization, and GELU activation functions) to obtain feature maps, followed by transposed convolution and batch normalization plus GELU activation functions. The function of transposed convolution is to expand the scale of the feature map to twice the input and reduce the number of channels by half. The output result is passed to the previous deconvolution group for further calculation. The deconvolution group process realizes cross-level information transmission and feature reconstruction, so that the model can gradually upsample and output a landslide prediction map with the same size as the input image.

[0082] The operation process of other deconvolution groups is the same as DeConv-2.

[0083] S5, model training, testing and verification. Details are as follows:

[0084] S51, inputting the samples of the training set into the landslide detection model designed in S4, extracting the boundary adaptive features and multi-scale features thereof respectively, and performing information interaction between the encoding process and the decoding process to obtain a landslide prediction map;

[0085] S52, based on the landslide prediction map obtained in step S51 and the pixel-by-pixel labeled image in S3, the loss function is calculated, and then the model parameters are optimized by back propagation gradient calculation to gradually reduce the loss function. During the training process, the validation set obtained in S3 is used to evaluate the performance of the model, to help select the best hyperparameters and model structure, and to prevent the model from overfitting on the training set; the weight of the model with the best performance is saved as the weight of the training output;

[0086] S53, using the test set obtained in S3 to evaluate the generalization ability of the model optimized in S52 on unseen data, and obtaining indicators such as the model's precision and recall rate, and comparing the performance with other existing methods to determine the reliability of the landslide detection model;

[0087] S6, applying the landslide detection model obtained in S53 to the landslide potential area detection to obtain the landslide detection result:

[0088] Prepare the remote sensing image data to be tested, load the trained model architecture and the best performing model weight file into the computer memory, input the image to be tested into the model for image semantic segmentation, and you can get the prediction result of the landslide area. Example

[0089] In order to facilitate the comparison with the accuracy of existing detection methods, this embodiment uses the landslide data set published by Wuhan University. The data set is TripleSat satellite data, the image is three bands of visible light, the spatial resolution is 0.8 meters, the data set has a total of 2773 image pairs, and the landslide size ranges from 30×30 pixels to 1000×1000 pixels. The image pairs are randomly assigned to the training set, test set, and validation set at a ratio of 6:1:1. During the image input model process, the image is scaled to 320×320 size, and conventional data enhancement is performed randomly by cropping, mirroring, etc.

[0090] The training networks of the present invention are all implemented on the Python-based deep learning framework Pytorch1.14, the device is NVIDIA A40, and all network model hyperparameters are uniformly set to: training batch (epoch) is 100, learning rate (learning rate) is 0.01, the number of single training samples (batch size) is 8, and the loss function is the cross entropy loss function.

[0091] The present invention adopts the commonly used semantic segmentation DeepLabV3plus, DDRNet, DANet, PSPNet and UNet as comparison methods, and especially uses UNet as the basic structure of the detection method of the present invention to highlight the effectiveness of the improvement.

[0092] In order to evaluate the model detection performance, the present invention selects F1 value, precision, recall rate and overall accuracy as evaluation indicators, and the calculation formulas of each indicator are as follows:

[0093] ;

[0094] ;

[0095] ;

[0096] .

[0097] The pixel is taken as the minimum evaluation unit, where: TP (True Positive), the label is landslide, and the detection result is also the number of pixels of landslide; TN (True Negative), the label is landslide, and the detection result is the number of pixels of background; FP (False Positive), the label is background, and the detection result is the number of pixels of landslide; FN (False Negative), the label is background, and the detection result is also the number of pixels of background.

[0098] The experimental results are quantified and shown in Table 1. From the perspective of the comprehensive indicator F1 value, the best performance among the five comparison methods is DeepLabv3plus (F1=91.72), and its recall rate is only 87.76%, which means that more than 12% of landslides are not detected; the UNet model has the highest recall rate of 89.07% among the comparison methods, and there are still many missed detections.

[0099] The F1 value and recall rate of the MAS-UNet model proposed in this invention are 96.02% and 93.99% respectively. The overall performance is improved by more than 4.3% compared with the comparison method, and the recall rate is improved by 4.92% compared with the original UNet model, which greatly reduces missed detections and ensures the effectiveness of the method. At the same time, its accuracy and total precision are as high as 99.21% and 98.13% respectively, indicating that there are very few false positives in the detection results, thus greatly reducing the workload of staff in verifying the authenticity of landslide detection results.

[0100] Table 1 Comparison of landslide detection accuracy (%)

[0101]

[0102] Figure 6-8 This is an example of a typical landslide detection result, comparing the landslide detection capabilities of each model at different sizes.

[0103] Figure 6It is a small landslide with an image size of 109×109 pixels. The landslide is located close to the road. All methods can detect the landslide, but there are differences in morphological restoration. The interpretation results of other methods are not accurate enough. The method of the present invention can capture the shape and structure of the landslide target, which is most consistent with the actual situation.

[0104] Figure 7 It is a large landslide with an image size of 512×512. There is a small amount of vegetation coverage at the site of the landslide. There are scattered broken patches in the large landslide extraction results of DDRNet and DeepLabv3plus. These locations are not marked as landslides, indicating that there is a false detection phenomenon, which requires additional manpower or resources to correct or process; DANet and PSPNet are quite different from the real morphology of landslide distribution and cannot correctly distinguish between the target and the background; the extraction results of the UNet method are close to the label, but there are distortions on the edges. The method of the present invention performs better in capturing the edge information of the landslide, can provide clearer and more accurate edge information, and is more accurate and comprehensive in morphological description. In summary, the method of the present invention can achieve good extraction effects at different scales.

[0105] Figure 8 In the area where large landslides occur, vegetation coverage interferes and easily leads to missed detection, such as DDRNet and DeepLabv3plus. However, the method of the present invention can combine global information in such scenes to obtain better detection results.

[0106] In general, the method of the present invention can completely extract landslides in the above three areas, and the shape and edge are consistent with the manually interpreted labels, which is consistent with the conclusion that the recall rate and precision of the quantitative comparison method are higher than those of the comparison method. The method of the present invention is superior to the existing landslide detection method.

Claims

1. A landslide detection method based on UNet semantic segmentation dual-branch improvement, characterized by The following steps are involved: S1, obtain high-resolution satellite remote sensing images of the study area; S2, preprocessing the high-resolution satellite remote sensing image to obtain a satellite remote sensing image after data enhancement; S3, making semantic segmentation sample sets through visual interpretation: based on expert knowledge, screen the areas containing landslide phenomena in the images preprocessed by S2, visually identify the landslide areas and background on the images, and use annotation tools to annotate the landslide areas pixel by pixel; then randomly assign the obtained image samples to the training set, test set, and validation set; S4, designing an improved landslide detection model based on UNet semantic segmentation dual branches: the landslide detection model consists of a boundary adaptive feature branch, a multi-scale feature branch and a conventional deconvolution group of UNet; In the encoding part of the model, a feature map of the landslide in the input image is obtained through multiple serially connected boundary adaptive feature extraction modules BAF-1~n and multiple serially connected multi-scale feature extraction modules MSF-1~n; the feature map is divided into n different levels according to its scale, and the length and width of the feature map of each level are half of the length and width of the feature map of the previous level, and the number of channels is twice that of the previous level; the features output by BAF-1 are used as the input of BAF-2, the features output by BAF-2 are used as the input of BAF-3, and so on; the output features of BAF-1 are accumulated with the calculation results of MSF-1, the accumulated result is used as the output of MSF-1, and the output of MSF-1 is used as the input of MSF-2; The output features of BAF-2 are accumulated with the calculation results of MSF-2, and the accumulated results are used as the output of MSF-2. The output of MSF-2 is used as the input of MSF-3, and so on; In the decoding part, DeConv-n in the UNet structure deconvolution group is used to restore the output of MSF-n to the size of the feature map of the previous level, and the output features of MSF-(n-1) are accumulated. The accumulated result is used as the input of DeConv-(n-1). This process is iterated until the output feature map of DeConv-1 in the UNet structure deconvolution group is consistent with the size of the input image. Finally, the landslide detection and recognition result is obtained through the LogSoftmax function. S5, training the landslide detection model based on the image samples to obtain the model weight, includes the following steps: S51, inputting the samples of the training set into the landslide detection model designed in S4, extracting the boundary adaptive features and multi-scale features thereof respectively, and performing information interaction between the encoding process and the decoding process to obtain a landslide prediction map; S52, based on the landslide prediction map obtained in step S51 and the pixel-by-pixel labeled image in S3, the loss function is calculated, and then the model parameters are optimized by back propagation gradient calculation to gradually reduce the loss function. During the training process, the validation set obtained in S3 is used to evaluate the performance of the model to help select the best hyperparameters and model structure to prevent the model from overfitting on the training set; the model weight with the best performance is saved as the training output weight; S53, using the test set obtained in S3 to evaluate the generalization ability of the model optimized in S52 on unseen data, and obtaining the precision and recall rate indicators of the model, and comparing the performance with other existing methods to determine the reliability of the landslide detection model; S6, applying the landslide detection model obtained in S53 to landslide potential area detection to obtain landslide detection results.

2. The landslide detection method according to claim 1, characterized in that: In step S4, for each boundary adaptive feature extraction module BAF-n, first perform 1×1 convolution on its input features to reduce the number of channels to 1 / 2 of the input features; then perform deformable convolution DFConv to extract features; for each position of the input feature map, first predict the extent to which each position of the feature map should be offset, generate an offset map, and then adjust the sampling position of the convolution kernel according to the predicted offset, add the sampling position of the convolution kernel to the offset, and obtain the actual sampling position of the convolution kernel at the position to generate a sampling grid; then perform weighted summation on the weights of the input feature map and the convolution kernel at the actual sampling position to obtain the corresponding position of the output feature map; the interval of the convolution operation is regularized by BatchNorm to convert the feature distribution interval to the range of [0, 1], and add the GELU function for normalization; Repeat the above process of deformable convolution DFConv, BatchNorm regularization and GELU function normalization; Then restore the channel through 1×1 convolution to be consistent with the input features; The output features at this time are accumulated with the input features of the boundary adaptive feature extraction module, and the accumulated results are subjected to a conventional convolution group operation; finally, the features are output to the boundary adaptive feature extraction module BAF-(n+1).

3. The landslide detection method according to claim 1, characterized in that: In step S4, for each multi-scale feature extraction module MSF-n, a 3×3 convolution with a step size of 2 is first performed on its input feature to complete downsampling, and then a conventional convolution group operation is performed to obtain a feature map F1 ; Perform dilated convolution on the results of the conventional convolution group with step size s={1, 2, 4, 7} respectively, and add the convolution results together to output and obtain the feature map F2 ; Then the learnable weights are calculated through the multi-head attention mechanism , , With feature map F1 , F2 To perform the product, the formula is as follows: , in, for Dimensions; Finally, the calculation results of the multi-head attention mechanism are accumulated with the output of BAF-n as the output of MSF-n.

4. The landslide detection method according to claim 1, characterized in that: In the DeConv-n of step S4, two conventional convolution group operations are first performed to obtain the feature map, and then transposed convolution, batch normalization, and GELU activation function operations are performed. The scale of the feature map is expanded to twice the input through transposed convolution and the number of channels is reduced by half.

5. The landslide detection method according to any one of claims 2 to 4, characterized in that: The conventional convolution group operation includes conventional convolution operation, BatchNorm regularization and GELU activation function operation.

6. The landslide detection method according to claim 1, characterized in that: The preprocessing described in S1 included: Cropping is used to extract or adjust the area of ​​interest in the remote sensing satellite images obtained by S1 to reduce the amount of data; Radiometric calibration is used to convert pixel values ​​in the cropped digital image into surface reflectance or radiant brightness to accurately represent the characteristics of the surface of the object; Geometric correction, which accurately matches the image pixels in the radiometrically calibrated digital image to the actual position on the earth's surface to eliminate geometric distortion; The steps of radiation calibration are: firstly, atmospheric correction is performed on the cropped digital image to eliminate the influence; then, the surface reflectance or radiation brightness of each pixel is calculated according to the radiation calibration parameters and the image data after atmospheric correction; finally, the result is compared with the ground field data to verify and adjust the result; The steps of geometric correction are: first, select some objects with known geographical coordinates as control points on the digital image, then use the geographical coordinates of the control points and the pixel coordinate information of the corresponding positions on the image to perform image registration, and finally perform projection transformation on the image and project it into the required map projection system.

7. The landslide detection method according to claim 1, characterized in that: In step S3, data enhancement operation is performed on the labeled samples to generate more samples, and the samples are randomly allocated to the training set, test set and validation set in a ratio of 6:1:1; in step S4, n is 5; the channels are 32, 64, 128, 256 and 512 respectively.

8. The landslide detection method according to claim 1, characterized in that: In S51, the interval of convolution operation is regularized by BatchNorm, so that the feature distribution interval is converted to the range of [0, 1], which speeds up the process of gradient descent fitting the optimal model, and adds GELU function for normalization to enhance the nonlinear characteristics of the model; then the channel is restored to be consistent with the input feature through 1×1 convolution. The output features are accumulated with the input features.

9. The landslide detection method according to claim 8, characterized in that: In S52, the output scale of MSF-1 is 160×160×32. For other multi-scale feature extraction modules, the input features are first downsampled, the feature maps are output and a conventional convolution group operation is performed, and then the dilated convolutions are performed with dilated parameters d={1, 2, 4, 7} respectively. The convolution results are accumulated and output together. Then the extracted results are bilinearly interpolated. , , is a learnable weight, which is multiplied by the feature and then input into the multi-head attention mechanism for calculation; finally, the output of the attention mechanism is accumulated with the output of the corresponding boundary adaptive feature extraction module.

10. The landslide detection method according to claim 9, characterized in that: In S53, the encoding features and the deep decoding features are first accumulated to achieve information fusion of features at different levels; then, the feature map is obtained through two conventional convolution group processes, and then transposed convolution and batch normalization plus GELU activation function operations are performed.