A rapid and intelligent detection method for damaged houses using remote sensing images from UAVs after an earthquake

Abstract

The present invention discloses a method for rapid intelligent detection of damaged houses in post-earthquake UAV remote sensing images, comprising the following steps: A. Collecting UAV remote sensing image samples and establishing a damaged house sample set; B. Based on the YOLOv5 deep learning model, firstly, introducing the Vision Transformer structure to optimize the Backbone network and improve the detection accuracy; secondly, replacing the PANet network with BiFPN to enhance the multi-scale information fusion capability in view of the multi-scale characteristics of the target; and then constructing the YOLOv5s-ViT-BiFPN model; C. Crop the input UAV remote sensing image and record its geographic location coding information; D. Based on the model, perform damaged house detection on the cropped image to obtain a target detection frame; E. Restore and vectorize the geographic location information of the target detection frame to obtain a damaged house detection frame vector map. The present invention realizes rapid and high-precision intelligent detection of damaged houses in post-earthquake UAV remote sensing images.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing and analysis, and more specifically relates to a method for rapid intelligent detection of damaged houses using post-earthquake UAV remote sensing images, which is suitable for large-format UAV orthophoto remote sensing images with a resolution of 0.01-0.1m. Background Art

[0002] my country is one of the countries with the most serious earthquake disasters in the world. More than 50% of cities and more than 70% of large and medium-sized cities are located in the intensity zone of 7 degrees or above, and are threatened by moderate and strong earthquakes. my country has experienced many destructive earthquakes with heavy losses, such as the "5.12" Wenchuan Ms8.0 earthquake in 2008, the "4.14" Yushu Ms7.1 earthquake in Qinghai in 2010, the "8.3" Ludian Ms6.5 earthquake in Yunnan in 2014, and the "5.21" Yangbi Ms6.4 earthquake in Dali Prefecture, Yunnan in 2021, which caused a large number of houses to be damaged. Post-earthquake emergency rescue and post-disaster assessment decision support urgently need to quickly and accurately obtain information on damaged houses, but this has always been a challenging problem and difficult to meet the needs of emergency response. UAV remote sensing has the advantages of flexible deployment at low altitude, high spatial resolution and no interference from clouds. It has become an indispensable technical means for post-disaster emergency response and loss assessment in addition to satellite remote sensing in the past 10 years.

[0003] Regarding the extraction of damaged houses, UAV remote sensing mainly extracts damaged houses from the perspectives of roof damage, damage classification, and identification of different types of damage, and has made considerable progress. Representative algorithms include modified CRF (Chinese Restaurant Franchise) unsupervised classification, object-oriented classification, and deep learning methods. In addition, some methods integrate 3D features of LiDAR point clouds. The modified CRF unsupervised classification model proposed by Li S. et al. uses color and shape features to segment the "roof holes" of slightly damaged rural houses with an accuracy of 98.62%, which is about 2% higher than the accuracy of K-means and ISODATA unsupervised classification methods. Li S. et al. classified damaged houses based on the elevation information of post-earthquake UAV images and laser point cloud data, combined with image spectral and morphological features, and included four categories: basically intact, slightly damaged, partially collapsed, and completely collapsed houses. et al. used an object-oriented classification method to extract newly built houses (as damaged houses) using two drone images before and after the simulated earthquake. Vetrivel et al. used multiple kernel learning (MKL) and integrated CNN (Convolutional Neural Network) features with 3D point cloud features to detect damaged houses with an accuracy of 94%. Although these studies have their own advantages, the establishment of the model takes a lot of time and it is difficult to balance its accuracy and efficiency at the same time. With the development of model methods in the field of deep learning, deep learning has shown the advantages of efficient detection and positioning in target detection models and has been applied to the extraction of damaged houses. This type of model is based on candidate regions or regression. Among them, region-based target detection (such as the R-CNN series of models) has high accuracy but slow speed. Regression-based target detection is fast, and the most typical one is the YOLO series of algorithms.

[0004] However, the YOLO series models only use CNN layers and have a relatively simple feature fusion network structure. The extraction of global features needs to be improved, and they cannot solve the problem of complex background objects and susceptibility to many interference factors in remote sensing image target detection, which in turn affects the detection accuracy.

[0005] In summary, the application of large-scale damaged house target detection for post-earthquake emergency rescue is still rare, and the detection accuracy, automation and intelligence of existing methods need to be improved. Damaged house detection based on UAV remote sensing images plays an increasingly important role in disaster emergency management, but the existing remote sensing image analysis technology has not yet met the needs of practical applications. In particular, with the improvement of image resolution and the sharp increase in data volume, the complex environmental background of damaged houses after major earthquakes, the multi-scale characteristics of UAV remote sensing images, and the accuracy and timeliness of disaster information acquisition in post-earthquake emergency response have posed severe challenges to the efficiency and accuracy of automated and intelligent processing of remote sensing data. Summary of the invention

[0006] In view of the defects of the above-mentioned prior art, the complex environmental background of damaged houses, the multi-scale characteristics of drone remote sensing images, and the accuracy and timeliness of disaster information acquisition required by post-earthquake emergency response, the purpose of the present invention is to provide a method for rapid intelligent detection of damaged houses using drone remote sensing images after an earthquake. The focus of the present invention is to detect damaged houses using drone high-resolution remote sensing images. The purpose of damaged house detection is to identify damaged house targets from drone remote sensing images and locate them. The present invention migrates YOlOv5 with fast detection speed to the most urgent damaged house detection need in earthquake emergency response, and aims at the complex background of damaged house targets, such as interference from undamaged houses and piled building materials, trees blocking houses, etc., the inconsistent initial resolution of drone remote sensing images, and the strong demand for timeliness of post-earthquake damage information acquisition in emergency rescue. Based on YOLOv5, the present invention selects YOLOv5s with the smallest volume, introduces the use of VisionTransformer to optimize the Backbone feature extraction network, and uses BiFPN to replace the PANet feature fusion network in the original YOLOv5, so that the model has stronger global feature learning ability and multi-scale information fusion ability, constructs the YOLOv5s-ViT-BiFPN model, and effectively solves the above problems. The method of the present invention is easy to implement and convenient to operate. By acquiring the rapid intelligent recognition and detection of damaged house targets from the remote sensing images of unmanned aerial vehicles in the post-earthquake disaster area, the target can be quickly locked, and the automation and intelligent processing efficiency and accuracy of damaged house target detection can be improved. This helps to improve the efficiency of emergency rescue command decision-making and enhance the decision-making support capabilities of disaster emergency management after a major disaster occurs, while providing objective disaster loss data for post-disaster reconstruction.

[0007] To further achieve the above object, the present invention adopts the following technical solutions:

[0008] The concept of the present invention is: first, establish a high-resolution damaged house sample set; second, train and test the damaged house samples to obtain a damaged house detection model; then, crop the UAV remote sensing image to meet the input requirements of the deep learning model; then, perform target detection on the processed UAV remote sensing image and obtain the target prediction frame; finally, restore and vectorize the location information of the target detection result, and finally obtain a damaged house detection vector map.

[0009] A method for rapid intelligent detection of damaged houses using remote sensing images of unmanned aerial vehicles after an earthquake, the steps of which are:

[0010] A. Collect UAV remote sensing image samples and establish a sample set of damaged houses;

[0011] B. Based on the YOLOv5 deep learning model, the Vision Transformer structure is introduced to optimize the Backbone network and improve the detection accuracy. Secondly, the PANet network is replaced with BiFPN to enhance the multi-scale information fusion capability based on the multi-scale characteristics of the target. Then the YOLOv5s-ViT-BiFPN model is constructed.

[0012] C. Crop the input UAV remote sensing image and record its geographic location coding information;

[0013] D. Detect damaged houses in the cropped image based on the model to obtain the target detection frame;

[0014] E. Restore and vectorize the geographic location information of the target detection frame to obtain the vector map of the damaged house detection frame.

[0015] Optionally, in step A, high-resolution orthophoto remote sensing images taken by drones are first collected, and then the damaged houses are labeled using the deep learning target detection and labeling tool LabelImg, and the external rectangular boxes and categories of the damaged houses are saved in corresponding XML files, respectively, to construct a sample set of damaged houses.

[0016] Optionally, in step B, first, introduce the Vision Transformer structure to modify the backbone network of YOLOv5s to optimize the network structure and focus on the target features, thereby improving the detection accuracy. Secondly, according to the scale diversity of damaged houses, optimize the feature fusion network, that is, replace the Path Aggregation Network (PANet) of the YOLOv5 model with the Bi-Directional Feature Pyramid Network (BiFPN), establish the YOLOv5s-ViT-BiFPN model, train and test the model, and finally obtain the damaged house target detection model YOLOv5s-ViT-BiFPN;

[0017] Furthermore, a hybrid structure of CNN network and Vision Transformer is adopted to make up for the defect that the convolution layer cannot learn the global features of the feature map well while retaining the strong feature extraction ability of CNN. Vision Transformer is added to the last layer of Backbone. Vision Transformer consists of Patch Embedding and Transformer Encoder. First, Path Embedding is used to reduce the dimension of the input feature map. Secondly, the position encoding operation is added to solve the problem of position information loss caused by dimensionality reduction. Finally, the vector is input into Transformer Encoder. The original Transformer Encoder consists of MSHA (Multi-Head Self-Attention) and MLP (Multilayer Perception). In order to minimize the number of parameters, the MLP layer is replaced by two fully connected layers (FC) layers, and the number of heads in MSHA is set to 4.

[0018] Furthermore, after the Backbone module extracts features, multi-scale feature fusion is required in the Neck module to detect targets of different resolutions. PANet has the problem of too many model parameters and large amount of calculation in multi-scale fusion through upsampling and downsampling, while BiFPN adopts the strategy of cross-scale connection, replaces PANet with BiFPN, removes the nodes in PANet that contribute less to feature fusion, and adds additional connections between input and output nodes at the same level, which can fuse more features while improving model efficiency.

[0019] Optionally, in step C, the input image size of the deep learning target detection model is generally hundreds or thousands of pixels. For example, the size of the input image based on the YOLOv5s-ViT-BiFPN network used in the present invention is 416×416 pixels, which is very different from the size of large-format remote sensing images (Large Remote Sensing Images). Therefore, before the remote sensing image is input into the network, it is necessary to perform a cropping operation to obtain a standard image block and record its geographic location encoding information relative to the large-format remote sensing image.

[0020] Optionally, in step D, the cropped standard image blocks are sequentially input into the trained damaged house detection model of YOLOv5s-ViT-BiFPN, damaged house target detection is performed for each standard image block, and a damaged house target detection frame is obtained.

[0021] Optionally, in step E, the geographic location information of each damaged house target detection frame is restored based on the geographic location coding information recorded in the standard image block where the damaged house target detection frame is located and the spatial coordinate system of the original remote sensing image, and then all target detection frames are vectorized to finally generate a damaged house detection vector map.

[0022] From the above, through the technical measures of the above five steps, rapid intelligent detection of damaged houses in post-earthquake UAV remote sensing images can be achieved. Since the Backbone in the original YOLOv5 model is only composed of convolutional neural networks, it cannot learn the global features of the feature map very well, and since the resolution of the UAV image cannot be kept consistent due to the influence of flight altitude and posture changes, so in the key step B, Vision Transformer is used to optimize the Backbone feature extraction network, and BiFPN is used to replace the PANet feature fusion network in the original YOLOv5. This innovation successfully solves the problem of insufficient global feature learning ability and insufficient multi-scale feature fusion ability of the original YOLOv5 target detection model.

[0023] Destructive earthquakes often cause a large number of houses to be damaged. How to accurately and automatically obtain this information in the shortest time to provide decision-making support for emergency rescue and post-disaster assessment has always been a challenging problem in disaster emergency management. However, there are many problems in extracting information about damaged houses after an earthquake, including complex backgrounds of damaged houses, such as interference from undamaged houses and piled building materials, and trees blocking houses; scale diversity problems caused by changes in flight altitude and posture of damaged houses in drone remote sensing images; and the current problem of low timeliness in extracting information about damaged houses after an earthquake. In response to the above problems, the present invention collects images taken by drones after an earthquake and establishes a sample set of damaged houses. The real-time target detection model YOLOv5 network is selected as the target detection network, and the network is improved in response to the above problems. The Backbone feature extraction network is optimized using VisionTransformer, and the PANet feature fusion network in the original YOLOv5 is replaced with BiFPN. Finally, the YOLOv5s-ViT-BiFPN damaged house detection model is established. This enables the damaged house detection model based on deep learning to have stronger global feature learning capabilities and multi-scale information fusion capabilities.

[0024] In natural images, the target of interest usually occupies a large proportion of the entire image. However, a remote sensing image has a huge number of pixels and a large geographical range, and the target of interest is usually very small relative to the background. A large-format remote sensing image cannot be directly input into a deep learning target detection network. In response to the above problems, before a large-format remote sensing image is input into a deep learning target detection network, the present invention crops it into standard image blocks so that it can be normally input into a deep learning target detection and recognition network, and records the geographic location encoding information of each standard image block, and uses this information to restore and vectorize the geographic location information of the damaged house target detection frame after detection, and finally obtains the vector result of the damaged house information extraction of the large-format remote sensing image. It effectively solves the problem that the deep learning network cannot directly perform target detection on large-format remote sensing images.

[0025] Due to the needs of emergency management, it is necessary to quickly grasp the status of damaged houses at the first time, so the present invention realizes rapid, accurate and intelligent detection of damaged houses in drone remote sensing images.

[0026] Compared with the prior art, the present invention has at least the following advantages and effects:

[0027] (1) The method for rapid intelligent detection of damaged houses from post-earthquake UAV remote sensing images established by the present invention can automatically extract damaged house targets from large-format UAV remote sensing images with high speed and accuracy.

[0028] (2) The present invention improves the YOLOv5 model, proposes and constructs the YOLOv5s-ViT-BiFPN model, and enhances the model's ability to learn global features of images and fuse multi-scale information during the detection process.

[0029] (3) The present invention crops and restores geographic information of large-format remote sensing images, so that damaged house targets can be directly identified and detected based on large-format remote sensing images, and a vector map of damaged houses can be output, making it easier to conduct statistical analysis of damaged house information. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 A flow chart of the rapid intelligent detection method of damaged houses using remote sensing images from drones after an earthquake;

[0032] Figure 2 The network structure of the YOLOv5s-ViT-BiFPN target detection model;

[0033] Figure 3 The Vision Transformer structure diagram added for this article;

[0034] Figure 4 A comparison chart of the Bi-Directional Feature Pyramid Network (BiFPN) and the Path Aggregation Network (PANet);

[0035] Figure 5 The change curves of the loss value and average precision (AP) of the verification data during the training process of the three models YOLOv5s, YOLOv5s-ViT, and YOLOv5s-ViT-BiFPN; (a) loss value change curve; (b) average precision (AP) change curve;

[0036] Figure 6 The drone remote sensing image of the damaged house to be inspected in the embodiment;

[0037] Figure 7 It is the damaged house detection vector map and the UAV remote sensing image overlay map in the embodiment. DETAILED DESCRIPTION

[0038] To make the purpose, technical solutions and advantages of the present invention more clear, the following will be combined with the attached Figure 1-7 The embodiments of the present invention are described in further detail.

[0039] like Figures 1 to 7As shown, in order to solve the problems of large workload and low timeliness of traditional remote sensing disaster information extraction, the present invention uses a convolutional neural network (CNN) to automatically select the optimal features and establish a damaged house target detection model. However, due to the huge number of pixels in remote sensing images, a large-scale remote sensing image cannot be directly input into a deep learning target detection network. Therefore, the present invention proposes a method for rapid intelligent detection of damaged houses in post-earthquake drone remote sensing images, selects the real-time target detection model YOLOv5 as the basic framework, introduces the Backbone feature extraction network optimized by Vision Transformer, and replaces the PANet feature fusion network in the original YOLOv5 with BiFPN, which solves the complex background of damaged house detection and increases the robustness of the model's multi-scale detection. In order to solve the problem of small object detection in large-scale remote sensing images, the two steps of drone remote sensing image cropping and damaged house target detection frame geographic location information restoration solve the problem that deep learning models cannot directly detect large-scale remote sensing images. The method for rapid intelligent detection of damaged houses in post-earthquake UAV remote sensing images proposed by the present invention can realize rapid and high-precision intelligent extraction of damaged house targets in post-earthquake UAV remote sensing images.

[0040] The process of the method for rapid intelligent detection of damaged houses by remote sensing images of unmanned aerial vehicles after an earthquake is as follows: Figure 1 As shown in the figure, it includes 7 units, mainly including the establishment of a damaged house target detection model based on deep learning and the damaged house target detection in large-format remote sensing images. The process of establishing a damaged house target detection model based on deep learning is as follows: first, collect and preprocess the post-earthquake UAV remote sensing images, then establish a damaged house sample set, and then establish a damaged house target detection model based on YOLOv5s-ViT-BiFPN. In order to obtain the damaged house target detection results of large-format remote sensing images, first crop the large-format remote sensing images, then perform damaged house target detection on the cropped standard image blocks, obtain the damaged house target detection frame, restore the geographical location and vectorize it, and finally obtain the damaged house detection vector map of the UAV remote sensing image.

[0041] The present invention proposes a rapid intelligent detection method for damaged houses in post-earthquake UAV remote sensing images, and establishes YOLOv5s-ViT-BiFPN. The network structure is as follows Figure 2 As shown in Figure 2, the YOLOv5s-ViT-BiFPN model consists of input, Backbone, BiFPN, and YOLOHead.

[0042] The input is a fixed-size 416×416 pixel image. Backbone is a feature extraction network. The Vision Transformer structure is added to the last layer of Backbone. The Vision Transformer consists of Patch Embedding and Transformer Encoder. First, Path Embedding is used to reduce the dimension of the input feature map. Secondly, position encoding operations are added to solve the problem of position information loss caused by dimensionality reduction. Finally, the vector is input into Transformer Encoder. The original Transformer Encoder consists of MSHA (Multi-Head Self-Attention) and MLP (Multilayer Perception). In order to minimize the number of parameters, the MLP layer is replaced by two fully connected layers (FC), and the number of heads in MSHA is set to 4. BiFPN is used to fuse multi-scale features, and the present invention uses it to replace PANet of YOLOv5. PANet has the problem of too many model parameters and large amount of calculation in multi-scale fusion through upsampling and downsampling, while the Bi-Directional Feature Pyramid Network (BiFPN) adopts a cross-scale connection strategy, replaces the Path Aggregation Network (PANet) with BiFPN, deletes the nodes in PANet that contribute less to feature fusion, and adds additional connections between input and output nodes at the same level, which can integrate more features while improving model efficiency. YOLO Head outputs the category probability and bounding box of the target at three scales respectively. Finally, the bounding box of the damaged house target detection in the input image is output.

[0043] Figure 3 The improved Vision Transformer structure used in the present invention.

[0044] Figure 4 The present invention utilizes a bidirectional feature pyramid network (Bi-Directional Feature Pyramid Network, BiFPN) to replace a path aggregation network (Path Aggregation Network, PANet).

[0045] Figure 5(a) and (b) are the change curves of the loss value and average precision (AP) of the validation data during the training process of the three models YOLOv5s, YOLOv5s-ViT, and YOLOv5s-ViT-BiFPN. It can be seen from the figure that the loss function curves and average precision (AP) curves of the three models gradually remain stable with the increase in the number of training times, but there are obvious differences in their change characteristics. The jitter amplitude of the loss function curve of the YOLOv5s-ViT-BiFPN model remains stable first and the loss value is the smallest. The loss function curves of the YOLOv5s and YOLOv5s-ViT models have larger jitter amplitudes, slower convergence speeds, and larger loss values ​​than the YOLOv5s-ViT-BiFPN model. The jitter amplitude of the average precision AP of the YOLOv5s-ViT-BiFPN model is the first to remain stable and has the largest value. The AP curves of the YOLOv5s and YOLOv5s-ViT models have larger jitter amplitudes and smaller AP values ​​than those of the YOLOv5s-ViT-BiFPN model.

[0046] On this basis, Figure 6 The post-earthquake drone remote sensing image of Baimu Village in Yangbi Yi Autonomous County, Yunnan Province, obtained after the "5.21" Ms6.4 earthquake, shows the results of the inspection of damaged houses. Figure 7 As shown, the dark rectangular blocks are correctly detected damaged house targets, and the light rectangular blocks are missed damaged house targets.

[0047] The present invention establishes a fast and intelligent detection method for damaged houses in post-earthquake UAV remote sensing images, uses an improved YOLOv5 target detection network to build a YOLOv5s-ViT-BiFPN model, extracts deeper features for damaged house target detection, and makes the detection results have higher accuracy, stronger robustness and higher timeliness. In view of the complex background of damaged house targets, such as interference from undamaged houses and piled building materials, trees blocking houses, scale diversity of damaged houses in UAV remote sensing images due to flight altitude and posture changes, and the current post-earthquake damaged house information extraction timeliness is not strong, the present invention is based on YOLOv5, introduces the use of Vision Transformer to optimize the Backbone feature extraction network, and uses BiFPN to replace the PANet feature fusion network in the original YOLOv5, so that the model has stronger global feature learning ability and multi-scale information fusion ability, effectively solving the above problems. Detecting small objects from large-scale images is one of the main problems in remote sensing image analysis. Therefore, in the process of large-format remote sensing image segmentation, standard image blocks are generated by cropping before detection and their geographic location coding information is recorded. After detection, the geographic location information is restored and vectorized according to the geographic location coding information of the standard image block where the damaged house target detection frame is located and the spatial coordinate system of the remote sensing image. This solves the problem of damaged house detection in large-format remote sensing images. Taking the application of emergency rescue in key earthquake disasters as an example, in order to quickly lock on the damaged house targets after the earthquake, it helps rescue personnel to quickly grasp the situation of the damaged houses and win time for emergency rescue. Through the test of a large-format remote sensing image after the "5.21" Ms6.4 earthquake in Yangbi, Yunnan in 2021 (Example 1), the method of the present invention can effectively extract damaged house targets in drone remote sensing images, and the average accuracy (AP) reaches 90.41%, which verifies the effectiveness of the present invention.

[0048] Embodiment 1:

[0049] A method for rapid intelligent detection of damaged houses using remote sensing images of unmanned aerial vehicles after an earthquake, the steps of which are:

[0050] 100. Collection and preprocessing of UAV remote sensing images: Taking the 2021 "5.21" Ms6.4 earthquake in Yangbi, Yunnan as an example, drone images of the earthquake-stricken areas were collected, remote sensing data sources suitable for target detection of damaged houses were screened out, and the remote sensing images were preprocessed, including the generation of drone orthophotos.

[0051] 101. Establishment of damaged house sample set: Collect and annotate damaged house remote sensing image samples to establish a damaged house sample set. In this example, 400 images with a size of 416×416 pixels were obtained, each of which contained at least one damaged house target, and there were 860 damaged house targets in total. Then, the deep learning target detection and labeling tool LabelImg was used to annotate the damaged houses, and the external rectangular box and category of the damaged houses were saved in an XML file to construct a damaged house sample set.

[0052] 102. Establishment of YOLOv5s-ViT-BiFPN damaged house detection model: Based on the YOLOv5 framework, VisionTransformer is used to optimize the Backbone feature extraction network, and BiFPN is used to replace the PANet feature fusion network in the original YOLOv5. The model is trained and verified, and the average precision (AP) of the model reaches 90.94%. Finally, the YOLOv5s-ViT-BiFPN damaged house detection model is established.

[0053] 103. Cropping of UAV remote sensing images: The input image is a remote sensing image of the Ms6.4 earthquake disaster area in Yangbi, Yunnan on May 21, 2021, with a resolution of 0.03m. Cropping the input remote sensing image to obtain standard image blocks and record their geographic location coding information.

[0054] 104. Damaged house target detection: Based on the YOLOv5s-ViT-BiFPN damaged house detection model, each standard image block is input into the YOLOv5s-ViT-BiFPN target detection model to obtain the damaged house target detection box in each standard image block.

[0055] 105. Restoration of geographic location information of damaged house target detection frames: Based on the geographic location coding information of the standard image block to which each target detection frame belongs and the spatial coordinate system of the original remote sensing image, the geographic location information of the target detection frame is restored, and all target detection frames are merged to generate a vector.

[0056] 106. Detection vector diagram of damaged houses in UAV remote sensing images: After the above steps, the final detection vector diagram of damaged houses in UAV remote sensing images is output.

[0057] The embodiments of the present invention are implemented on a PC platform, and are verified by experiments to obtain relatively ideal results.

[0058] The present invention proposes a method for rapid intelligent detection of damaged houses in post-earthquake UAV remote sensing images, which solves the problems of low accuracy and timeliness in target detection of traditional methods and existing machine learning methods, and few large-format remote sensing images. Through the experimental verification of damaged house target detection in large-format remote sensing images of the Ms6.4 earthquake disaster area in Yangbi, Yunnan on May 21, 2021, it was found that this method can effectively improve the accuracy and efficiency of damaged house target detection. Figure 6 Among the 73 damaged houses, 66 were correctly detected, with an accuracy of 90.41% and a detection time of 38.02s. The damaged house detection vector diagram and the UAV remote sensing image overlay are shown in the figure below: Figure 7 shown.

[0059] The above embodiments are only preferred embodiments of the present invention and are not intended to limit the embodiments of the present invention. Other different forms of changes or modifications can be made based on the above embodiments, and it is impossible to list all the embodiments here. All obvious changes and modifications derived from the technical solution of the present invention still fall within the scope of protection of the present invention.

Claims

1. A method for rapid intelligent detection of damaged houses using remote sensing images from unmanned aerial vehicles after an earthquake, characterized in that: Here are the steps: Step A: Collect UAV remote sensing image samples and establish a damaged house sample set; Step B: Based on the YOLOv5 deep learning model, the Vision Transformer structure is introduced to optimize the Backbone network; the bidirectional feature pyramid network is used to replace the path aggregation network for the target, and then the YOLOv5s-ViT-BiFPN model is constructed; Step C, cropping the inputted UAV remote sensing image and recording its geographic location coding information; Step D: Based on the YOLOv5s-ViT-BiFPN model, damaged houses are detected on the cropped image to obtain a target detection frame; Step E: restore and vectorize the geographic location information of the target detection frame to obtain a vector map of the damaged house detection frame; In the step B, the YOLOv5s-ViT-BiFPN model includes input, Backbone, BiFPN and YOLO Head; The input is an image with a fixed size of 416×416 pixels. Backbone is a feature extraction network, and the Vision Transformer structure is added to the last layer of Backbone. The Vision Transformer consists of Patch Embedding and Transformer Encoder. First, Path Embedding is used to reduce the dimension of the input feature map. Secondly, a position encoding operation is added to solve the problem of position information loss caused by dimensionality reduction. Finally, the vector is input into the Transformer Encoder. The original Transformer Encoder includes MSHA and MLP. The MLP layer is replaced by two fully connected layers, and the number of heads in MSHA is set to 4. BiFPN is used to fuse multi-scale features, and it is used to replace the PANet of YOLOv5. The YOLO Head outputs the category probability and bounding box of the target at three scales respectively. Finally, the bounding box of the damaged house target detection in the input image is output.

2. According to claim 1, a method for rapid intelligent detection of damaged houses using remote sensing images of unmanned aerial vehicles after an earthquake, characterized in that: In the step A, firstly, a sample set of damaged houses is established using remote sensing images taken by drones, and then on this basis, the deep learning target detection and labeling tool LabelImg is used to label the targets, and the bounding rectangles and categories of the damaged houses are saved in corresponding XML files respectively.

3. According to claim 1, a method for rapid intelligent detection of damaged houses using remote sensing images of unmanned aerial vehicles after an earthquake, characterized in that: In the step C, the size of the YOLOv5s-ViT-BiFPN network input image is 416×416 pixels. Before the remote sensing image is input into the network, a cropping operation is performed to obtain a standard image block of 416×416 pixels, and its geographic location encoding information is recorded so that the geographic location information of the damaged house target detection frame obtained by the model detection can be restored.

4. According to claim 1, a method for rapid intelligent detection of damaged houses using remote sensing images of unmanned aerial vehicles after an earthquake, characterized in that: In the step D, the segmented standard image blocks of 416×416 pixels are respectively input into the trained damaged house detection model YOLOv5s-ViT-BiFPN to obtain a damaged house target detection frame in the standard image block including the damaged house; the damaged target detection frame includes four parameters, the horizontal coordinate and the vertical coordinate of the upper left corner, and the horizontal coordinate and the vertical coordinate of the lower right corner.

5. The method for rapid intelligent detection of damaged houses using remote sensing images of UAVs after an earthquake according to claim 1 is characterized in that: In the step E, the geographic location information of the target detection frame is restored according to the position of the target detection frame relative to the standard image block and the geographic location encoding information, and then all the target detection frames are vectorized to generate a damaged house detection vector map.

Images