Unmanned aerial vehicle image-based residential area tiny ground feature identification and risk grading method

Abstract

The invention discloses a residential area tiny ground feature recognition and risk grading method based on unmanned aerial vehicle images, which is characterized by comprising the steps of constructing a tiny ground feature data set, training and recognizing a tiny ground feature recognition model, marking an area with the highest tiny ground feature density, carrying out risk grading and the like. The tiny ground feature data set adopts DJ Terra to carry out orthographic TIFF image construction; the areas with the highest tiny ground feature density are identified and marked, risk grading is carried out, and kernel density analysis is carried out by adopting a spatial interpolation method. Compared with the prior art, the method has the advantages that tiny ground objects in a site can be efficiently and accurately recognized and counted, the risk of missing detection and false detection is effectively reduced, solid technical support is provided for safe execution of flight operation of the unmanned aerial vehicle, the problem of positioning and recognition of the tiny ground objects in the flight process of the unmanned aerial vehicle is solved, and the unmanned aerial vehicle flight safety is improved. The method can meet the recognition requirements of tiny ground objects in different scenes, is higher in adaptability and robustness, and is good in application prospect.

Description

Technical Field

[0001] The present invention relates to the field of safe take-off and landing of unmanned aerial vehicles and flight technology, and more specifically, to a method for A method for identifying small features and grading risks in residential areas based on learning technology. Background Art

[0002] With the rapid development of drone technology, the safety of drone take-off and landing has become one of the key factors restricting its widespread application. As a core component of the drone operation process, the safety of drone take-off and landing is directly related to the success or failure of drone mission execution and the safety of drone equipment. In serious cases, it may threaten the safety of life and property of operators. However, in actual operations, the presence of tiny objects on the ground poses a potential risk to drone take-off and landing that cannot be ignored. Although they are small in size, they are very likely to cause physical damage to key components such as landing gear and propellers during drone take-off and landing. In serious cases, it may even cause safety accidents, resulting in damage to the drone or casualties.

[0003] Given the significant impact of tiny ground objects on the safety of drone takeoff and landing, it is particularly important to accurately and efficiently identify and locate them. However, traditional manual detection methods are not only time-consuming and labor-intensive, but also inefficient. In the face of complex and changing ground environments, it is often difficult to ensure the accuracy and comprehensiveness of detection, thereby increasing the safety hazards during drone takeoff and landing. Moreover, with the increasing popularity of drone technology, more and more drone flying enthusiasts have mastered the technology of using drones to perform flight missions. The landing sites are diverse and unsupervised, and often cannot be inspected in detail. Especially for the situation where humans and machines land in different places, it is often difficult to obtain the ground conditions of the landing site, and the potential risks of rash landings should not be underestimated.

[0004] In recent years, with the rapid progress of deep learning technology, especially the continuous innovation of image recognition and object detection technology, new solutions have been provided for the automatic identification of tiny objects on the ground. With its powerful data processing and pattern recognition capabilities, deep learning models can accurately identify tiny objects in complex backgrounds, providing new technical guarantees for the safety of drone takeoff and landing. It can significantly improve the recognition accuracy and speed of tiny objects, reduce the risk of missed detection and false detection, and further improve the safety and stability of drone takeoff and landing, providing more solid technical support for the widespread application of drones.

[0005] In summary, the flatness of the ground is very important during the take-off and landing of drones. Scattered tiny debris may pose a safety hazard during the take-off and landing phases. During the flight of drones, existing technologies cannot locate and identify tiny objects on the ground, which seriously affects the safe take-off and landing and flight of drones. Summary of the invention

[0006] The purpose of the present invention is to provide a method for identifying and grading the risks of micro-land objects in residential areas based on drone images in response to the deficiencies of the prior art. The method uses a micro-land object dataset to train a micro-land object recognition model based on YOLOv7 to identify and mark the areas with the highest density of micro-land objects for risk grading. The method combines morphological image processing technology with deep learning algorithms to enhance the safety assurance capabilities of drone landing and take-off sites, and to achieve efficient and accurate identification and counting of micro-land objects in the site, providing solid technical support for the safe execution of drone flight operations, and better It solves the problem of positioning and identifying tiny objects on the ground during the flight of UAV aircraft, and has good application prospects and commercial development value.

[0007] The specific technical scheme for achieving the purpose of the present invention is: a method for identifying and grading the risks of micro-features in residential areas based on drone images, which is characterized in that the method includes: constructing a micro-feature data set, training a micro-feature recognition model based on YOLOv7, identifying and marking the areas with the highest density of micro-features and performing risk grading, wherein the construction of the micro-feature data set is to use a drone to collect images of the study area, perform orthophoto construction, cropping, and manual calibration to make labels, and construct a data set; the training of the micro-feature recognition model based on YOLOv7 is to use the constructed micro-feature data set to train the micro-feature recognition model; the identification and marking of the areas with the highest density of micro-features and performing risk grading is to evaluate the overall performance and spatial performance of the model, perform statistical comparisons on the number and type of each type of real and predicted micro-features at the community scale and the sub-region scale, identify and mark the areas with the highest density of micro-features in the experience area by calculating the kernel density value, and perform risk grading on each area.

[0008] The construction of the micro-feature dataset is to use drone photography to obtain high-definition images of areas 1 to 6, convert the original image data into TIFF format, crop and convert the format to generate JPEG format, and use LabelImg software to draw real polygon annotations of nine types of micro-features in each image to form a dataset with label information.

[0009] The training of the micro-object recognition model based on YOLOv7, that is, using the pre-trained weights "yolov7_training.pt" provided by the YOLOv7 official website to initialize the network architecture, the data set is randomly divided into a training set, a validation set, and a test set. By inputting the images in the test set into the model and comparing the predicted results with the true labels, the target recognition accuracy of the model is obtained. The results show that the F1 score and average precision of the YOLO v7 model on the validation set and the test set are both over 0.99.

[0010] The method of identifying and marking the area with the highest density of micro-land objects and performing risk classification is to use spatial interpolation technology to perform kernel density analysis to form hot spots with the highest core density values, that is, a significant clustering effect.

[0011] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress: 1) It can locate and identify tiny obstacles such as stones, rubble, and branches more quickly and accurately, thereby effectively reducing the risk of missed detection and false detection, significantly improving the recognition accuracy and efficiency of tiny ground objects, and providing more reliable protection for the safety of UAV take-off and landing.

[0012] 2) When faced with complex and changing ground environments, the deep learning model can automatically learn and extract the characteristic information of tiny objects, thereby effectively identifying tiny objects of different shapes, sizes, and materials. This feature makes the method of the present invention more widely applicable in practical applications, can meet the needs of tiny object identification in different scenarios, and has stronger adaptability and robustness.

[0013] 3) By training and optimizing the deep learning model, it can realize real-time online monitoring of tiny ground objects, reduce the workload of operators, improve detection efficiency, provide more timely and accurate information support for the safe take-off and landing of drones, and have a high degree of automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example

[0016] See also Figure 1 , the present invention comprises the following specific steps: Step 1: Build a small object dataset High-definition images of areas 1 to 6 of a new village were obtained using drone photography, covering a total area of ​​approximately 0.436 square kilometers. The flight route was planned with the help of the DJI Phantom 4 Pro V2 drone and its accompanying Pixel 4d Capture application, ensuring that the image overlap was maintained within the ideal range of 70% to 80%. At the same time, the flight altitude was set to 30 meters, thus achieving a ground sampling distance of 0.82 cm / pixel. A total of 6,594 high-quality images were obtained, and two-dimensional reconstruction was performed using DJI Terra 3.4.4 software. The original image data was converted into TIFF format images, which were cropped and converted to generate a JPEG format image set. On this basis, LabelImg software was used to draw real polygon annotations for nine types of tiny features in each image, namely: aquatic plants, trash cans, bowls, boxes, buckets, earthenware jars, lids, flower pots and sinks, forming a data set with labeled information.

[0017] Step 2: Training of small object recognition model based on YOLOv7 YOLOv7 was selected as the deep learning recognition model, and a transfer learning strategy was adopted. Specifically, the pre-trained weights "yolov7_training.pt" provided by the YOLOv7 official website were used to initialize the network architecture, and training was started on this basis to improve the recognition accuracy and convergence speed of the model. For the training and verification of the model, the dataset was randomly divided into a training set (60%), a validation set (20%), and a test set (20%). The training dataset has 9808 images and 20713 labels, the validation dataset has 3270 images and 6969 labels, and the test dataset has 3270 images and 7013 labels. Among them, the training set and validation set are used for model training and optimization, while the test set is used to evaluate the final accuracy of the model.

[0018] This example uses a personal computer equipped with a 2.9GHz i7-10700 processor, 16GB memory and NVIDIA GeForceRTX 2070 SUPER GPU, and the experimental framework is Pytorch. During the training process, the batch size was set to 8, and 100 training cycles were performed on the GPU. The object recognition accuracy of the model was obtained by inputting images in the test set into the model and comparing the predicted results with the true labels.

[0019] This embodiment uses indicators such as precision, recall, F1 score and average precision to comprehensively evaluate the model performance. The experimental results show that the F1 score and average precision of the YOLOv7 model on the validation set and the test set are both over 0.99, which fully demonstrates the robustness of the model. At the community scale, the spatial distribution comparison shows an accuracy rate of up to 97%. In addition, at a more detailed sub-regional scale, the distribution of different public micro-feature categories is also exactly the same, and the average difference between the actual number and the predicted number is only 1.4.

[0020] Step 3: Identify and mark areas with the highest density of micro-objects and classify their risks After successfully obtaining the coordinate information of the tiny features, we used the QGIS software platform and spatial interpolation technology to conduct an in-depth kernel density analysis. As an effective spatial analysis method, kernel density estimation can transform point data into a trend that changes continuously in space, thus providing an intuitive and clear perspective to observe and analyze the spatial distribution pattern of tiny features.

[0021] Specifically, kernel density is the result of weighted summation of all point data within a certain range (i.e., radius or bandwidth) around a central point (xi, yi). In this process, the closer the point is, the greater the weight is given to ensure the accuracy and rationality of the analysis. The measurement unit of kernel density is set to per square meter for more accurate comparison and analysis. Through kernel density analysis, the high-density areas are analyzed to form obvious hotspots on the map, providing a strong guide for clearing tiny features. Among them, the hotspot with the highest core density value is located in the southwest of Sub-district 4 and extends to the northwest of Sub-district 5, forming a significant clustering effect. In addition, other hotspot areas worthy of attention are found in the north, northeast and southeast of Sub-district 4. At the same time, two elongated clusters are presented in the middle of Sub-district 1, while four major concentrated areas are distributed in Sub-district 5, which are mainly located on the west side. In contrast, the core density values ​​of Sub-districts 2 and 6 are generally lower than the average level, showing relatively sparse distribution characteristics. It is worth noting that Sub-district 3 shows unique characteristics in kernel density analysis. This area has the lowest kernel density and the least number of tiny features, showing a uniform distribution. Based on the maximum value of the kernel density and the regional average value, the risk is classified into three levels: I, II, and III.

[0022] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A method for identifying and grading small objects in residential areas based on drone images, characterized in that: The constructed tiny feature dataset is used to train a tiny feature recognition model based on YOLOv7, and the areas with the highest density of tiny features are identified and marked for risk grading. The tiny feature dataset is constructed by collecting orthophotos of the study area using drones, and is randomly divided into training set, validation set and test set in a ratio of 6:2:2; the method of identifying and marking the areas with the highest density of tiny features for risk grading uses a statistical comparison of the number and type of each type of real and predicted tiny features at the community scale and sub-region scale, and calculates the kernel density value to identify and mark the areas with the highest density of tiny features in the empirical area, as well as the sub-regions that most need to reduce the number, and performs risk grading based on the maximum value of the kernel density and the regional average value.

2. The method for identifying and grading small objects in residential areas based on drone images according to claim 1 is characterized in that: The micro-object dataset is constructed by using DJ Terra to construct orthophoto TIFF images, which are cropped and converted into JPEG images with 960×960 pixels. LabelImg is used to manually calibrate and create labels for nine types of micro-objects to construct the micro-object dataset.

3. The method for identifying and grading small objects in residential areas based on drone images according to claim 1 is characterized in that: The training of the micro-object recognition model adopts the transfer learning method, uses the pre-trained weights "yolov7_training.pt" provided by the YOLOv7 official website to initialize the network, and starts training the model from the initial state.

4. The method for identifying and grading small objects in residential areas based on drone images according to claim 1 is characterized in that: The method of identifying and marking the areas with the highest density of micro-features and grading the risks draws a distribution map of micro-features based on the model recognition F1 score and average accuracy higher than 0.99, creates a kernel density map to identify the areas with the highest density of micro-features, and divides them into three risk levels: I, II and III according to the maximum value of the kernel density and the regional average value.

5. The method for identifying and grading small objects in residential areas based on drone images according to claim 1 or 4, characterized in that: The identification and marking of the area with the highest density of micro-land objects and risk classification are performed by using a spatial interpolation method to perform kernel density analysis.

Images