Unmanned aerial vehicle landslide emergency identification system considering multiple transfer learning and semantic segmentation

Abstract

The invention relates to an unmanned aerial vehicle landslide emergency identification system considering multiple transfer learning and semantic segmentation, and the system comprises a data obtaining unit which is used for obtaining an ImageNet data set, a VOC2012 data set, a Crack500 data set, and a Landslide segmentation data set; the semantic segmentation model is used for predicting a landslide boundary and comprises a backbone network, an ASPP module and a decoder; the transfer learning module is used for executing multiple transfer learning on the semantic segmentation model; the embedded AI computing platform is used for integrating the semantic segmentation model obtained after processing of the transfer learning module and communicating with the unmanned aerial vehicle; and the unmanned aerial vehicle is used for collecting videos or photos of the landslide site. According to the method, the prediction precision and speed of model segmentation can be remarkably improved through multiple transfer learning by using a specific public data set, and important technical support is provided for post-disaster emergency response.

Description

Technical Field

[0001] This invention relates to the field of geological disaster prevention and control technology, specifically to an unmanned aerial vehicle (UAV) landslide emergency identification system that takes into account multiple transfer learning and semantic segmentation. Background Technology

[0002] Landslides are one of the most common geological disasters in my country, causing serious casualties and property losses every year, while also impacting the ecological environment and infrastructure. Heavy rain and strong earthquakes are the most important factors triggering large-scale landslides, and the ability to quickly identify and detect these landslides is fundamental to post-disaster emergency response and rescue work.

[0003] Currently, the detection of large-scale, clustered landslides mainly relies on manual on-site investigations, which is time-consuming and labor-intensive. Furthermore, after heavy rains and strong earthquakes, the environment and transportation are often severely damaged, making it difficult for personnel to enter the disaster area and conduct timely investigations and detections. While drone inspections can remotely acquire disaster images, their flight positions are limited by signal constraints, typically only reaching 1-2 km from the operator. Some studies have used high-resolution remote sensing imagery for landslide interpretation and identification, but the results are highly subjective because the indicators used in the interpretation process are usually qualitative rather than quantitative, heavily dependent on the operator's standards. For example, Guo et al. (Guo et al., A novel hybrid model integrating high resolution remote sensing and stacking ensemble techniques for landslide susceptibility mapping: Application to event-based landslide inventory) used 2m resolution remote sensing imagery to interpret landslides in the Mentougou district of Beijing induced by typhoons and heavy rains. All of the hundreds of landslides identified were obtained through visual interpretation by operators, resulting in significant uncertainty. Deep learning models have also been applied to landslide interpretation. The workflow of this method involves selecting typical landslide and non-landslide samples within the study area to train the model, and then using the trained model to identify all landslides within the area. This method is more accurate than manual interpretation. Liu et al. (A proposed method for landslide detection based on transfer learning and graph neural network) built a model based on an existing landslide dataset using transfer learning and graph neural networks to identify recently occurring cluster landslides. However, this method requires remote sensing imagery, which cannot meet the needs of immediate emergency rescue operations at disaster sites. Due to the scarcity of landslide datasets and the complexity of landslide characteristics, how to quickly and effectively identify cluster landslides and achieve dynamic identification remains a challenge. For disaster relief work, it is crucial to obtain information about the disaster site situation as soon as possible through drone images or related video photos. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention aims to provide a UAV-based landslide emergency identification system that incorporates multiple transfer learning and semantic segmentation. This system utilizes a specific public dataset to train a semantic segmentation model through multiple transfer learning iterations, significantly improving the model's prediction accuracy and speed. Furthermore, this model can be integrated into an embedded AI computing platform and mounted on a UAV. The UAV automatically flies along a preset route at the scene, acquiring high-resolution imagery and video data. The embedded AI computing platform can identify landslides in the video images in real time, dynamically display their extent, and save the accurate landslide boundaries and locations. This significantly improves the efficiency and accuracy of investigating cluster landslide disasters, providing crucial technical support for post-disaster emergency response.

[0005] The technical solution adopted by the present invention to solve the aforementioned technical problem is as follows: In a first aspect, the present invention provides a method for emergency identification of landslides by unmanned aerial vehicles (UAVs) that considers multiple transfer learning and semantic segmentation, the method comprising the following: Obtain the ImageNet dataset, VOC2012 dataset, Crack500 dataset, and Landslide_segmentation dataset respectively; The semantic segmentation model was used for the first transfer learning using the ImageNet and VOC2012 datasets; A second transfer learning was performed on the model after the first transfer learning using the VOC2012 dataset and the Crack500 dataset; A third transfer learning was performed on the model after the second transfer learning using the Crack500 dataset and the Landslide_segmentation dataset. The transfer learning consists of two stages. The first stage is to freeze the backbone network and train only the ASPP module and decoder of the semantic segmentation model. This process uses the Adam optimizer and the Dice Loss loss function. The second stage is to unfreeze the backbone network, reduce the learning rate for fine-tuning, and further optimize the model until the model loss function reaches the convergence condition. The semantic segmentation model after three transfer learning iterations was used for rapid landslide emergency identification.

[0006] Furthermore, the semantic segmentation model is the DeepLab series of semantic segmentation networks, including the DeepLabV3Plus model or an improved DeepLabV3Plus model, etc.

[0007] Furthermore, the improved DeepLabV3Plus model includes two modules: an encoder and a decoder. The encoder consists of a backbone network and a dilated spatial pyramid pooling (ASPP) module. The backbone network replaces the last layer of the Xception structure with a dilated convolutional neural network. The concatenated features of different scales output by the ASPP module are directly fed into the interpolation and upsampling operation in the decoder module without being processed by 1*1 convolutional blocks. The feature map obtained by interpolation and upsampling is concatenated with the low-level feature map of the encoder backbone network output features after channel dimensionality reduction. After being processed by two sets of 3*3 convolutional blocks and then superimposed with a set of 1*1 convolutional blocks, the semantic segmentation mask image is obtained by four-fold upsampling.

[0008] Secondly, the present invention provides a UAV landslide emergency identification system that considers multiple transfer learning and semantic segmentation, the system comprising: The data acquisition unit is used to acquire data from four types of datasets, including the ImageNet dataset, the VOC2012 dataset, the Crack500 dataset, and the Landslide_segmentation dataset. A semantic segmentation model for predicting landslide boundaries, comprising a backbone network, an ASPP module, and a decoder; The transfer learning module is used to perform multiple transfer learning operations on the semantic segmentation model. In the first transfer learning, the ImageNet dataset is input into the semantic segmentation model. First, the backbone network of the model is frozen, and only the ASPP module and decoder are trained. This process uses the Adam optimizer and the Dice Loss function. Then, the backbone network is unfrozen, the VOC2012 dataset is input into the model, the learning rate is reduced for fine-tuning, and convergence conditions are set. When the model's loss function is less than this value, the model stops running, and the hyperparameters and weights of the model at this point are saved. In the second transfer learning, the VOC2012 dataset is input into the semantic segmentation model after the first transfer learning. First, the backbone network of the model is frozen, and only the ASPP module and decoder are trained. This process uses the Adam optimizer and the Dice Loss function. The first step is to apply the loss function. Then, the backbone network is unfrozen, the Crack500 dataset is input into the model, the learning rate is reduced for fine-tuning, and a convergence condition is set. When the model's loss function is less than this value, the model stops running, and the model's hyperparameters and weights are saved at this point. In the third transfer learning, the Crack500 dataset is input into the semantic segmentation model from the second transfer learning. First, the backbone network of the model is frozen, and only the ASPP module and decoder are trained. This process uses the Adam optimizer and the Dice Loss function. Then, the backbone network is unfrozen, the Crack500 dataset is input into the model, the learning rate is reduced for fine-tuning, and a convergence condition is set. When the model's loss function is less than this value, the model stops running, and the model's hyperparameters and weights are saved at this point. An embedded AI computing platform is used to integrate the semantic segmentation model obtained after processing by the transfer learning module and to communicate with the drone. Drones are used to collect videos or photos of landslide sites.

[0009] Furthermore, the UAV acquires continuous frame video of the landslide to be predicted, denoted as the original image. The semantic segmentation model obtained after processing by the transfer learning module is used to process the original image, outputting a result image. The boundaries between the landslide and non-landslide areas in the result image are extracted, and other parts of the image are deleted. The result after deletion is saved as a JPEG image, denoted as the [image name missing]. i The frame predicts the landslide boundary; the saved JPEG image only shows the landslide boundary line, and the rest of the image is transparent; The first i The predicted landslide boundary of frame 1 is combined with the original image to obtain the recognition result with landslide boundary, denoted as the 1st frame. i The dynamic recognition results of the frames are returned to the drone's control handle via the main control panel at set time intervals and displayed to the user.

[0010] Furthermore, the system includes an ArcGIS platform that wirelessly communicates with the UAV. Based on the dynamic recognition results, it automatically calculates the number of closed landslide boundary lines in each image, which is the number of landslides in the area displayed by that image, denoted as . M Calculate the ratio of the total landslide area to the total area of ​​the image in each region; this is the landslide proportion of the region displayed in the image, denoted as . P ;set up M and P threshold Mw and Pw When the dynamic recognition result of a certain frame M value or P Value exceeds Mw or Pw Then at this time M Value and P The values ​​are displayed on the screen of the control handle, and the coordinates of the landslide boundary line at this moment are saved in ASCII format; and transmitted to the emergency command center through the wireless network module installed in the drone, and a landslide hazard distribution map is created in the ArcGIS platform to assist the emergency command center in disaster relief and decision-making.

[0011] Furthermore, M The threshold is 5. P The threshold is 10%.

[0012] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: (1) The system and method of the present invention use multiple transfer learning, select a specific combination of public datasets, start from the most common and largest dataset (ImageNet dataset), and transfer learning three times to a dataset that only contains landslides (Landslide_segmentation dataset). These large datasets have sufficient data volume (ImageNet contains 14 million images, VOC2012 contains 1.2 million images), which can make the model training more sufficient, thus producing good results for landslide recognition.

[0013] (2) Existing technologies typically interpret landslides based on high-resolution remote sensing images. However, remote sensing image interpretation is usually done manually, which is subjective and inefficient. While there are automatic interpretation and recognition models capable of automatically identifying landslide images, these models require manually interpreted images as samples for training. For example, if 1000 landslides occur in a certain area, to automatically identify all 1000 landslides using a model, a portion of the landslides (e.g., 100-200) must first be manually interpreted. Only after training the model with these landslides can the remaining landslides be identified. The system of this invention can automatically and quickly interpret and identify landslides on-site using images and videos captured by drones, avoiding manual interpretation of high-resolution remote sensing images. It also eliminates the need for extensive on-site surveys and investigations, greatly improving the efficiency and accuracy of post-disaster emergency response.

[0014] (3) Landslides have complex and diverse characteristics, and deep learning models require a large number of samples for training. Therefore, existing technologies are based on static landslide images for identification. Data related to the landslide area to be detected (remote sensing images, UAV aerial surveys, etc.) needs to be acquired before model training can be performed, and then landslide detection and identification can be carried out. Usually, researchers are based on a specific landslide event, so they can acquire data, but the process of acquiring data takes time (generating remote sensing images takes time, and going to the landslide site for UAV aerial surveys also takes time). However, if there is little data on the target area or if results are to be obtained as soon as possible, it is difficult to carry out the work without sufficient relevant landslide data. This invention trains a model capable of semantic segmentation of landslides using large datasets publicly available online and transfer learning. The transfer learning combination method of the publicly available datasets specially selected in this invention can learn the complex features of landslides and has a strong correlation with landslides, which significantly improves the accuracy and real-time performance of landslide boundary prediction, making the model performance independent of the images acquired on-site. Preliminary experiments show that the combination of other types of publicly available datasets cannot achieve the effect described in this application. The transfer learning method proposed in this application does not require prior acquisition of landslide-related data and information for the research target area. It can be applied to the identification of cluster landslide disasters induced by earthquakes or heavy rainfall in different regions, and shows high accuracy for any landslide, effectively solving the problem of insufficient engineering data.

[0015] (4) This invention imports the transfer learning model into an embedded AI computing platform, and then loads the trained model onto a drone through the embedded AI computing platform. Because the model has been pre-trained and imported into the embedded AI computing platform, it can start working directly in the field, dynamically identifying the landslide range and saving the vector file of the landslide boundary for subsequent emergency management and decision-making. Therefore, the system workflow of this invention is flexible, and users can adjust and optimize the model parameters and improve the model structure at any time. In the future, other datasets can be added on this basis to further enhance the model's learning ability, thereby ensuring that the model can obtain good generalization performance in areas with different geological conditions.

[0016] (5) This invention replaces the existing technology of semantic segmentation of static images based on deep learning models with dynamic recognition, realizing dynamic landslide recognition based on transfer learning, which can be used directly at disaster sites. The original DeepLabV3Plus model is improved by adopting multi-stage upsampling and a deeper semantic fusion structure in the decoder part, which significantly improves the spatial detail recovery capability and small target segmentation accuracy, and is particularly suitable for scenarios with high detail requirements such as crack recognition, remote sensing segmentation and geological disaster recognition. Attached Figure Description

[0017] Figure 1This is a structural comparison diagram between the improved DeepLabV3Plus model proposed in this invention and the original DeepLabV3Plus model.

[0018] Figure 2 This is a flowchart illustrating one embodiment of the UAV landslide emergency identification system that incorporates multiple transfer learning and semantic segmentation, as described in this invention. Detailed Implementation

[0019] The present invention will be further explained below with reference to the embodiments and accompanying drawings, but this is not intended to limit the scope of protection of this application.

[0020] Example 1: This embodiment considers a drone-based landslide emergency identification method based on multiple transfer learning and semantic segmentation, including the following steps: Step 1: Obtain the ImageNet dataset and the VOC2012 dataset respectively, and perform the first transfer learning using an improved DeepLabV3Plus model. Use the model to learn the annotation capabilities of the ImageNet dataset, then transfer these capabilities to the VOC2012 dataset, and evaluate the results; Both the ImageNet dataset and the VOC2012 dataset can be downloaded from public websites; the former contains 14 million images, and the latter contains 1.2 million images. The original DeepLabV3Plus model is an enhanced version of the proposed DeepLab semantic segmentation network series. Its core idea is to enhance the ability to recover spatial details while maintaining the ability to extract semantic information. It mainly consists of two modules: an encoder and a decoder. The encoder includes a backbone network (DeepConvolutional Neural Networks) and an ASPP (Atrous Spatial Pyramid Pooling) module.

[0021] The original DeepLabV3Plus model uses the Xception architecture as its backbone network, with the last layer being a regular convolutional neural network. This invention replaces this regular convolution with a dilated convolution.

[0022] The ASPP module in the original DeepLabV3Plus model concatenates feature information of different scales using concat, and then sends it to the decoder module through a 1*1 convolutional block (including convolution, BN, activation, and dropout layers). However, this 1*1 convolutional block is removed in this invention.

[0023] In the original DeepLabV3Plus model, the Decoder module performs channel dimensionality reduction on the low-level feature maps using 1x1 convolutions, then performs linear interpolation upsampling on the feature maps from the ASPP module. Finally, the channel-dimension-reduced low-level feature maps and the linearly interpolated upsampling feature maps are concatenated and fed into a set of 3x3 convolutional blocks for processing. This embodiment modifies this set of 3x3 convolutional blocks into two sets of 3x3 convolutional blocks superimposed with a set of 1x1 convolutional blocks. This model structure achieves deeper semantic fusion without significantly increasing model complexity, balancing semantics and computational efficiency.

[0024] The above operations are used to adjust the original DeepLabV3Plus model to obtain the improved DeepLabV3Plus model. Compared with the previous model, the improved model adopts multi-stage upsampling and a deeper semantic fusion structure in the decoder part, which significantly improves the spatial detail recovery capability and small target segmentation accuracy. It is particularly suitable for scenarios with high detail requirements, such as crack recognition, remote sensing segmentation and geological disaster recognition.

[0025] The transfer learning consists of two phases. The first phase involves freezing the backbone network (Xception) and training only the ASPP module and decoder, using the Adam optimizer and Dice Loss function. The second phase involves unfreezing the backbone network, reducing the learning rate for fine-tuning, and further optimizing the model until the model's loss function reaches the convergence condition.

[0026] Step Two: Obtain the Crack500 dataset and perform a second transfer learning using the improved DeepLabV3Plus model described above. Transfer the capabilities learned by the model in Step One to the Crack500 dataset and evaluate the results. The Crack500 dataset can be downloaded from a public website and includes 1896 images; Step 3: Obtain the Landslide_segmentation dataset and perform a third transfer learning using the improved DeepLabV3Plus model described above. Transfer the capabilities learned by the model in Step 2 to the Landslide_segmentation dataset and evaluate the results. The Landslide_segmentation dataset can be downloaded from a public website and includes 1962 images.

[0027] Example 2: In this embodiment, the model trained through three rounds of transfer learning is integrated into a graphics card to form an embedded AI computing platform. This platform is then connected to the main control panel of a typical small drone, allowing them to function as a single unit. As the drone operates in the air, the captured video stream data is directly transmitted to the embedded AI computing platform. The video is then processed according to... T A frame is captured at second intervals, and this frame is saved as the original image. The model trained through three transfer learning iterations is called, using this image as input for model computation, and the output is the recognized result image. The boundary between the landslide area and non-landslide area in the result image is extracted, and other parts of the image are deleted. The result after deletion is saved as a JPEG image, denoted as the first image. i Predicted landslide boundaries for frames.

[0028] Drone flight N seconds, a total of images were extracted from the video. N / T frame.

[0029] The resulting image is a masked image, where identified landslide areas are marked with 255 (displayed in white), and non-landslide areas are marked with 0 (displayed in black). The saved JPEG image only shows the boundary lines of the landslides; the rest of the image is transparent.

[0030] The first i (1≤ i ≤ N / T The predicted landslide boundary of frame 1 is combined with the original image to obtain the recognition result with landslide boundary, denoted as the 1st frame. i The dynamic recognition results of each frame. The dynamic recognition results are then divided into... T The message is displayed to the user via the drone's main control panel on the drone's control handle at intervals of one second.

[0031] For the obtained dynamic recognition results, the number of closed landslide boundary lines in each image is automatically calculated, which is the number of landslides in the area displayed by that image, denoted as . M Calculate the ratio of the total landslide area to the total area of ​​the image in each region; this is the landslide proportion of the region displayed in the image, denoted as . P .set up M and P threshold Mw and Pw When the dynamic recognition result of a certain frame M value or P Value exceeds Mw or Pw Then at this time M Value and PThe values ​​are displayed on the screen, and the coordinates of the landslide boundary line at this moment are saved in ASCII format. The results are transmitted to the emergency command center via a 5G cellular network module installed on the drone, and a landslide hazard distribution map is created in the ArcGIS platform to assist the emergency command center in disaster relief and decision-making.

[0032] This invention utilizes a deep learning model for landslide identification, requiring landslide-related datasets for model training. However, existing publicly available landslide datasets are limited in size; for example, the Landslide_segmentation dataset we used only contains 1962 images. Directly using such a dataset for model training results in insufficient training data, leading to a significant drop in accuracy. This invention, however, employs a specially combined set of publicly available datasets for multiple transfer learning iterations, significantly improving the model's learning and generalization abilities.

[0033] Example 3: This embodiment uses Zixing City, Hunan Province as the research area. In July 2024, affected by Typhoon Fengmei, the region experienced a large-scale, clustered landslide disaster within a few days. Traditional manual on-site investigations and visual interpretation are insufficient to effectively and quickly identify all landslides. Therefore, the method of this invention is used to conduct emergency identification and investigation of landslide disasters, thereby verifying the effectiveness of the proposed method. The main steps are as follows: S1: An improved DeepLabV3Plus model was built based on Python and the PyCharm platform. The Xception architecture was used as the backbone network, with a dilated convolutional neural network as the last layer instead of the standard convolutional neural network used in the original DeepLabV3Plus. Furthermore, the 1x1 convolutional blocks in the ASPP module of the original DeepLabV3Plus model were removed. In the decoder section, the multi-scale feature maps from the ASPP output were linearly interpolated and upsampled. Then, the low-level feature maps obtained from channel dimensionality reduction and the linearly interpolated and upsampled feature maps were concatenated using concat, and then fed into two sets of 3x3 convolutional blocks and one set of 1x1 convolutional blocks for processing. This model structure achieves deeper semantic fusion without significantly increasing model complexity, balancing semantics and computational efficiency.

[0034] The ImageNet and VOC2012 datasets were publicly available. The ImageNet dataset was input into the improved DeepLabV3Plus model. First, the backbone network of the model was frozen, and only the ASPP module and decoder were trained using the Adam optimizer and Dice Loss function. Then, the backbone network was unfrozen, the VOC2012 dataset was input into the model, the learning rate was reduced for fine-tuning, and the convergence condition was set to 10. -5 When the model loss function is less than this value, the model stops running and saves the model hyperparameters and weights at this time.

[0035] S2: Obtain the Crack500 dataset and perform a second transfer learning using the improved DeepLabV3Plus model described above. Load the model hyperparameters and weights saved in S1, input the VOC2012 dataset into the improved DeepLabV3Plus model, first freeze the backbone network, and train only the ASPP module and decoder using the Adam optimizer and Dice Loss function; then unfreeze the backbone network, input the Crack500 dataset into the model, reduce the learning rate for fine-tuning, and set the convergence condition to 10. -5 When the model loss function is less than this value, the model stops running and saves the model hyperparameters and weights at this time.

[0036] S3: Obtain the Landslide_segmentation dataset and perform a third transfer learning using the improved DeepLabV3Plus model described above. Load the model hyperparameters and weights saved in S2, input the Crack500 dataset into the improved DeepLabV3Plus model, first freeze the backbone network, and train only the ASPP module and decoder using the Adam optimizer and Dice Loss function; then unfreeze the backbone network, input the Landslide_segmentation dataset into the model, reduce the learning rate for fine-tuning, and set the convergence condition to 10. -5 When the model loss function is less than this value, the model stops running and saves the model hyperparameters and weights at this time.

[0037] This invention creatively selects specific public datasets from a large number of existing datasets, and uses multiple specific public datasets in a certain order to obtain an effective transfer learning path through multiple transfer learning processes, thereby significantly improving the model's recognition accuracy.

[0038] S4: Integrates the model code trained in S3 into the graphics card to form an embedded AI computing platform. The graphics card has 157 TOPS, featuring a 1024-core NVIDIA Ampere architecture with 32 Tensor Cores and a maximum frequency of 1172MHz. It also includes an 8-core Cortex-A78AE v8.2 64-bit CPU with 2MB of L2 cache and 4MB of L3 cache. This platform can be connected to the main control panel of a typical small drone, allowing them to operate as a single unit.

[0039] The drone operates in the air, and the video stream it captures is directly transmitted to the embedded AI computing platform. The captured video is 600 seconds long, and one frame is extracted every second, resulting in 600 frames, which are recorded as the original images. The pre-trained model in S3, along with its saved hyperparameters and weights, is used to process the 600 frames as input, outputting the recognized result images. These result images are masked images, where identified landslide areas are marked as 255 (displayed in white), and non-landslide areas are marked as 0 (displayed in black).

[0040] The boundaries between landslide and non-landslide areas were extracted from the resulting images, and other parts of the images were deleted. The results were saved as JPEG images, resulting in 600 predicted landslide boundary images. The saved JPEG images only show the landslide boundary lines, with the rest of the images being transparent.

[0041] S5: Combine the 600 predicted landslide boundaries with their corresponding original images at each time point to obtain 600 dynamic recognition results with landslide boundaries. These dynamic recognition results are then sent back to the drone's control handle via the drone's main control panel at 1-second intervals and displayed to the user. The user observes 600 results, each lasting 1 second.

[0042] S6: For the 600 dynamic recognition results obtained in S5, automatically calculate the number of closed landslide boundary lines in each image, which is the number of landslides in the area displayed by that image, denoted as . M Calculate the ratio of the total landslide area to the total area of ​​the image in each region; this is the landslide proportion of the region displayed in the image, denoted as . P .set up M The threshold is set to 5. P The threshold is 10%, and there are 20 dynamic recognition results. M Values ​​greater than 5 were found in 18 dynamic recognition results. PFor values ​​greater than 10%, the corresponding landslide boundary coordinates in these results are saved in ASCII format and transmitted to the emergency command center via a 5G cellular network module installed on the drone. A landslide hazard distribution map is then created in the ArcGIS platform to assist the emergency command center in disaster relief and decision-making.

[0043] S7: The landslide identification results obtained above were compared with those obtained by some other techniques, including four types: ① Landslide identification using the original DeepLabV3Plus model and the multiple transfer learning mentioned in this technique; ② Landslide identification using the improved DeepLabV3Plus model mentioned in other techniques (Yu Junyang et al., an improved urban street scene image segmentation method based on deep learning) and the multiple transfer learning mentioned in this technique; ③ Landslide identification using only the Landslide_segmentation dataset to train the improved DeepLabV3Plus model in S1; ④ Landslide identification using 100 landslides visually interpreted from Planet satellite remote sensing images (2m spatial resolution) as samples to train the improved DeepLabV3Plus model in S1. Meanwhile, to verify the generalization performance of the models, the above four models were trained in the Zixing study area of ​​Hunan Province and then directly used to identify typhoon-induced landslides in the Mentougou district of Beijing in 2023.

[0044] The application results are shown in Table 1 below. It can be seen that this invention achieved the highest landslide identification accuracy in both study areas. The difference between techniques ① and ② and this technology lies in the model structure used; both employ three-stage transfer learning. A comparison of the three reveals that the three-stage transfer learning of this invention significantly improves the accuracy of landslide emergency identification and reduces training time. Furthermore, the improved DeepLabV3Plus model described in this application significantly improves identification accuracy and computational efficiency, while maintaining high identification accuracy across different study areas. This is because the model structure developed in this technology enhances spatial detail recovery capabilities and small target segmentation accuracy, making it particularly suitable for scenarios with high detail requirements, such as crack identification, remote sensing segmentation, and geological hazard identification.

[0045] Technique ③ has a short model training time because it uses only one dataset and does not perform multiple transfer learning iterations, but its accuracy is significantly lower than techniques that involve multiple transfer learning iterations. Furthermore, the Landslide_segmentation dataset has a small sample size, so when using only this dataset for model training, the recognition accuracy in the Mentougou district of Beijing is very poor because the model lacks sufficient samples for learning, resulting in poor generalization ability. Technique ④ does not use transfer learning, but only uses visually interpreted landslides as samples, so it achieves acceptable results in Zixing, Hunan. However, its accuracy will significantly decrease when applied to other research areas. In addition, the visual interpretation process of remote sensing imagery is relatively lengthy, which greatly affects the efficiency of recognition.

[0046] Table 1 Comparison of landslide identification results using different technologies in different study areas

[0047] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A method for emergency identification of landslides using unmanned aerial vehicles (UAVs) considering multiple transfer learning and semantic segmentation, characterized in that, The method includes the following: Obtain the ImageNet dataset, VOC2012 dataset, Crack500 dataset, and Landslide_segmentation dataset respectively; The semantic segmentation model was used for the first transfer learning using the ImageNet and VOC2012 datasets; A second transfer learning was performed on the model after the first transfer learning using the VOC2012 dataset and the Crack500 dataset; A third transfer learning was performed on the model after the second transfer learning using the Crack500 dataset and the Landslide_segmentation dataset. The transfer learning consists of two stages. The first stage is to freeze the backbone network and train only the ASPP module and decoder of the semantic segmentation model. This process uses the Adam optimizer and the Dice Loss loss function. The second stage is to unfreeze the backbone network, reduce the learning rate for fine-tuning, and further optimize the model until the model loss function reaches the convergence condition. The semantic segmentation model after three transfer learning iterations was used for rapid landslide emergency identification.

2. The method according to claim 1, characterized in that, The semantic segmentation model is the DeepLab series of semantic segmentation networks, including the DeepLabV3Plus model or an improved DeepLabV3Plus model.

3. The method according to claim 1, characterized in that, The improved DeepLabV3Plus model includes two modules: an encoder and a decoder. The encoder consists of a backbone network and a dilated spatial pyramid pooling (ASPP) module. The backbone network replaces the last layer of the Xception structure with a dilated convolutional neural network. The concatenated features of different scales output by the ASPP module are directly fed into the interpolation and upsampling operation in the decoder module without being processed by 1*1 convolutional blocks. The feature map obtained by interpolation and upsampling is concatenated with the low-level feature map of the encoder backbone network output features after channel dimensionality reduction. After being processed by two sets of 3*3 convolutional blocks and then superimposed with a set of 1*1 convolutional blocks, the semantic segmentation mask image is obtained by four-fold upsampling.

4. A UAV landslide emergency identification system considering multiple transfer learning and semantic segmentation, characterized in that, The system includes: The data acquisition unit is used to acquire data from four types of datasets, including the ImageNet dataset, the VOC2012 dataset, the Crack500 dataset, and the Landslide_segmentation dataset. A semantic segmentation model for predicting landslide boundaries, comprising a backbone network, an ASPP module, and a decoder; The transfer learning module is used to perform multiple transfer learning operations on the semantic segmentation model. In the first transfer learning, the ImageNet dataset is input into the semantic segmentation model. First, the backbone network of the model is frozen, and only the ASPP module and decoder are trained. This process uses the Adam optimizer and the Dice Loss function. Then, the backbone network is unfrozen, the VOC2012 dataset is input into the model, the learning rate is reduced for fine-tuning, and convergence conditions are set. When the model's loss function is less than this value, the model stops running, and the hyperparameters and weights of the model at this point are saved. In the second transfer learning, the VOC2012 dataset is input into the semantic segmentation model after the first transfer learning. First, the backbone network of the model is frozen, and only the ASPP module and decoder are trained. This process uses the Adam optimizer and the Dice Loss function. The first step is to apply the loss function. Then, the backbone network is unfrozen, the Crack500 dataset is input into the model, the learning rate is reduced for fine-tuning, and a convergence condition is set. When the model's loss function is less than this value, the model stops running, and the model's hyperparameters and weights are saved at this point. In the third transfer learning, the Crack500 dataset is input into the semantic segmentation model from the second transfer learning. First, the backbone network of the model is frozen, and only the ASPP module and decoder are trained. This process uses the Adam optimizer and the Dice Loss function. Then, the backbone network is unfrozen, the Crack500 dataset is input into the model, the learning rate is reduced for fine-tuning, and a convergence condition is set. When the model's loss function is less than this value, the model stops running, and the model's hyperparameters and weights are saved at this point. An embedded AI computing platform is used to integrate the semantic segmentation model obtained after processing by the transfer learning module and to communicate with the drone. Drones are used to collect videos or photos of landslide sites.

5. The system according to claim 4, characterized in that, A drone acquires consecutive video frames of the landslide to be predicted, denoted as the original image. The original image is then processed using a semantic segmentation model obtained after transfer learning, outputting a result image. The boundaries between the landslide and non-landslide areas in the result image are extracted, and other parts of the image are deleted. The result after deletion is saved as a JPEG image, denoted as the [image name missing]. i The frame predicts the landslide boundary; the saved JPEG image only shows the landslide boundary line, and the rest of the image is transparent; The first i The predicted landslide boundary of frame 1 is combined with the original image to obtain the recognition result with landslide boundary, denoted as the 1st frame. i The dynamic recognition results of the frames are returned to the drone's control handle via the main control panel at set time intervals and displayed to the user.

6. The system according to claim 5, characterized in that, The system includes an ArcGIS platform that wirelessly communicates with the UAV. Based on dynamic recognition results, it automatically calculates the number of closed landslide boundary lines in each image, which represents the number of landslides in the area displayed in that image, denoted as . M Calculate the ratio of the total landslide area to the total area of ​​the image in each region; this is the landslide proportion of the region displayed in the image, denoted as . P ; set up M and P threshold Mw and Pw When the dynamic recognition result of a certain frame M value or P Value exceeds Mw or Pw Then at this time M Value and P The values ​​are displayed on the screen of the control handle, and the coordinates of the landslide boundary line at this moment are saved in ASCII format; and transmitted to the emergency command center through the wireless network module installed in the drone, and a landslide hazard distribution map is created in the ArcGIS platform to assist the emergency command center in disaster relief and decision-making.

7. The system according to claim 6, characterized in that, M The threshold is 5. P The threshold is 10%.

Images