A method for identifying and classifying risks of micro-objects in residential areas based on images of unmanned aerial vehicles

Abstract

This invention discloses a method for identifying and risk-classifying minute ground features in residential areas based on UAV imagery. The method includes steps such as constructing a minute ground feature dataset, training a minute ground feature identification model, identifying and marking the areas with the highest density of minute ground features, and risk-classifying them. The minute ground feature dataset is constructed using orthorectified TIFF imagery from DJ Terra. The identification and marking of the areas with the highest density of minute ground features and the risk-classification employ kernel density analysis using spatial interpolation. Compared with existing technologies, this invention offers efficient and accurate identification and counting of minute ground features within a site, effectively reducing the risk of missed and false detections. It provides solid technical support for the safe execution of UAV flight operations, solves the problem of locating and identifying minute objects on the ground during UAV flight, can meet the needs of minute ground feature identification in different scenarios, has stronger adaptability and robustness, and shows promising application prospects.

Description

Technical Field

[0001] This invention relates to the field of safe take-off and landing and flight technology for unmanned aerial vehicles (UAVs), specifically a method based on UAV imagery and depth...

[0002] A learning-based method for identifying and risk-classifying small features in residential areas. Background Technology

[0003] With the continuous and rapid development of drone technology, the safety of drone takeoff and landing has become one of the key factors restricting its widespread application. As a core component of drone operation, the safety of takeoff and landing directly affects the success or failure of drone missions and the safety of drone equipment, potentially threatening the lives and property of operators. However, in actual operation, the presence of even small ground objects poses a significant potential risk to drone takeoff and landing. Despite their small size, these objects can easily cause physical damage to critical components such as landing gear and propellers during takeoff and landing, potentially leading to accidents, drone destruction, or personal injury.

[0004] Given the significant impact of minute ground features on the safety of drone takeoff and landing, accurate and efficient identification and location are crucial. However, traditional manual inspection methods are not only time-consuming, labor-intensive, and inefficient, but also often fail to guarantee accuracy and comprehensiveness in the face of complex and changing ground environments, thus increasing safety hazards during drone takeoff and landing. Furthermore, with the increasing prevalence of drone technology, more and more drone enthusiasts are mastering the skills to perform flight missions, resulting in diverse and unsupervised landing sites that often cannot be thoroughly inspected. Especially in cases where humans and drones land in different locations, it is often difficult to obtain information about the ground conditions of the landing site, and the potential risks of landing rashly should not be underestimated.

[0005] In recent years, the rapid advancement of deep learning technology, particularly the continuous innovation in image recognition and object detection, has provided new solutions for the automatic identification of minute ground features. Deep learning models, with their powerful data processing and pattern recognition capabilities, can accurately identify minute ground features in complex backgrounds, providing new technical safeguards for the safety of UAV takeoff and landing. This can significantly improve the accuracy and speed of minute feature identification, reduce the risk of missed and false detections, and further enhance the safety and stability of UAV takeoff and landing, providing stronger technical support for the widespread application of UAVs.

[0006] In conclusion, the flatness of the ground is crucial during drone takeoff and landing, as scattered small debris can pose safety hazards during these phases. Current technology cannot locate or identify small objects on the ground during drone flight, severely impacting the safe takeoff, landing, and operation of drones. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for identifying and risk-classifying minute features in residential areas based on UAV imagery. This method employs a dataset of minute features to train a YOLOv7-based model for identifying and marking areas with the highest density of minute features for risk classification. This approach integrates morphological image processing techniques with deep learning algorithms, aiming to enhance the safety assurance capabilities of UAV take-off and landing sites and achieve efficient and accurate identification and counting of minute features within the site. This provides solid technical support for the safe execution of UAV flight operations and effectively...

[0008] It solves the problem of locating and identifying tiny objects on the ground during the flight of unmanned aerial vehicles (UAVs), and has good application prospects and commercial development value.

[0009] The specific technical solution to achieve the purpose of this invention is: a method for identifying and risk-classifying small features in residential areas based on UAV imagery. The method includes: constructing a small feature dataset; training a small feature recognition model based on YOLOv7; identifying and marking the areas with the highest density of small features and classifying their risks. Constructing the small feature dataset involves using a UAV to collect images of the study area, constructing orthophotos, cropping, and manually labeling them to create the dataset. Training the small feature recognition model based on YOLOv7 involves using the constructed small feature dataset to train the model. Identifying and marking the areas with the highest density of small features and classifying their risks involves evaluating the overall and spatial performance of the model, statistically comparing the number and type of each type of real and predicted small features at the community and sub-regional scales, calculating kernel density values, identifying and marking the areas with the highest density of small features in the empirical area, and classifying the risks of each area.

[0010] The construction of the micro-feature dataset involves acquiring high-resolution images of zones 1 to 6 using drone photography, converting the original image data to TIFF format, cropping and converting the format to generate JPEG format, and using LabelImg software to draw real polygon labels for nine types of micro-features in each image, forming a dataset with labeled information.

[0011] The training of the YOLOv7-based miniature object recognition model involved initializing the network architecture using the pre-trained weights "yolov7_training.pt" provided on the official YOLOv7 website. The dataset was randomly divided into training, validation, and test sets. The model's target recognition accuracy was determined by inputting images from the test set into the model and comparing the predictions with the ground truth labels. Results showed that the YOLOv7 model achieved an F1 score and mean accuracy exceeding 0.99 on both the validation and test sets.

[0012] The process involves identifying and marking the areas with the highest density of small ground features and classifying them for risk. This involves using spatial interpolation techniques to perform kernel density analysis, forming hotspots with the highest core density values, which represent a significant clustering effect.

[0013] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress:

[0014] 1) It can locate and identify small obstacles such as stones, rubble, and branches more quickly and accurately, thereby effectively reducing the risk of missed detection and false detection, significantly improving the accuracy and efficiency of identifying small ground objects, and providing a more reliable guarantee for the safety of UAV take-off and landing.

[0015] 2) When faced with complex and ever-changing ground environments, deep learning models can automatically learn and extract feature information of minute ground features, thereby achieving effective identification of minute ground features of different shapes, sizes, and materials. This feature makes the method of this invention more widely applicable in practical applications, able to meet the needs of minute ground feature identification in different scenarios, and has stronger adaptability and robustness.

[0016] 3) By training and optimizing deep learning models, real-time online monitoring of small ground objects can be achieved, reducing the workload of operators, improving detection efficiency, and providing more timely and accurate information support for the safe take-off and landing of UAVs, with a high degree of automation. Attached Figure Description

[0017] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0019] See Figure 1 The present invention includes the following specific steps:

[0020] Step 1: Construct a small ground feature dataset

[0021] High-resolution images of zones 1 to 6 of a new village were acquired using drone photography, covering a total area of ​​approximately 0.436 square kilometers. The flight path was planned using a DJI Phantom 4 Pro V2 drone and its accompanying Pixel 4d Capture application, ensuring that the image overlap remained within the ideal range of 70% to 80%. A flight altitude of 30 meters was set, achieving a ground sampling distance of 0.82 cm / pixel. A total of 6594 high-quality images were obtained. Two-dimensional reconstruction was performed using DJI Terra 3.4.4 software, converting the original image data to TIFF format. After cropping and format conversion, a JPEG image set was generated. Based on this, LabelImg software was used to create realistic polygon labels for nine types of small ground features on each image: aquatic plants, trash cans, bowls, boxes, buckets, earthenware pots, lids, flower pots, and troughs, forming a labeled dataset.

[0022] Step 2: Training a YOLOv7-based micro-feature recognition model

[0023] YOLOv7 was selected as the deep learning recognition model, and a transfer learning strategy was adopted. Specifically, the pre-trained weights "yolov7_training.pt" provided on the official YOLOv7 website were used to initialize the network architecture, and training began on this basis to improve the model's recognition accuracy and convergence speed. For model training and validation, the dataset was randomly divided into a training set (60%), a validation set (20%), and a test set (20%). The training dataset has 9808 images and 20713 labels, the validation dataset has 3270 images and 6969 labels, and the test dataset has 3270 images and 7013 labels. The training and validation sets were used for model training and optimization, while the test set was used to evaluate the model's final accuracy.

[0024] This embodiment uses a personal computer equipped with a 2.9GHz i7-10700 processor, 16GB of memory, and an NVIDIA GeForce RTX 2070 SUPER GPU, with PyTorch as the experimental framework. During training, a batch size of 8 was set, and 100 training epochs were performed on the GPU. The target recognition accuracy of the model was determined by inputting images from the test set into the model and comparing the prediction results with the true labels.

[0025] This embodiment uses metrics such as precision, recall, F1 score, and mean precision to comprehensively evaluate model performance. Experimental results show that the YOLOv7 model achieves an average F1 score and mean precision exceeding 0.99 on both the validation and test sets, fully demonstrating the model's robustness. At the community scale, spatial distribution comparisons show an accuracy as high as 97%. Furthermore, at more granular sub-regional scales, the distribution of different public micro-feature categories is completely consistent, with an average difference of only 1.4 between the actual and predicted quantities.

[0026] Step 3: Identify and mark the areas with the highest density of small ground features and classify them for risk.

[0027] After successfully acquiring the coordinate information of minute features, an in-depth kernel density analysis was conducted using spatial interpolation technology on the QGIS software platform. Kernel density estimation, as an effective spatial analysis method, can transform point data into continuously changing trends in space, thus providing an intuitive and clear perspective for observing and analyzing the spatial distribution patterns of minute features.

[0028] Specifically, kernel density is the weighted sum of all point data within a certain range (i.e., radius or bandwidth) around a central point (xi, yi). In this process, points closer to each other are assigned greater weight, ensuring the accuracy and rationality of the analysis. The unit of measurement for kernel density is set to per square meter for more precise comparison and analysis. Kernel density analysis reveals that high-density areas form obvious hotspots on the map, providing strong guidance for clearing small features. The hotspot with the highest core density value is located in the southwest of zone 4 and extends to the northwest of zone 5, forming a significant clustering effect. In addition, other noteworthy hotspot areas were found in the north, northeast, and southeast of zone 4. Meanwhile, two elongated clusters were observed in the central part of zone 1, while four main concentrated areas were distributed within zone 5, primarily located on the west side. In contrast, the core density values ​​of zones 2 and 6 are generally below average, showing a relatively sparse distribution. Notably, zone 3 exhibited unique characteristics in the kernel density analysis. This region exhibits a uniform distribution with the lowest kernel density and the fewest micro-features. Therefore, based on the maximum kernel density and the regional average, the risk is classified into three levels: I, II, and III.

[0029] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of the inventive concept are included in this invention and are protected by the appended claims.

Claims

1. A method for identifying and risk-classifying minute features in residential areas based on UAV imagery, characterized in that, A YOLOv7-based micro-feature recognition model was trained using a constructed micro-feature dataset. This model identifies and marks the areas with the highest micro-feature density for risk classification. The method identifies and counts micro-features within UAV take-off and landing sites, providing information support for safe UAV take-off and landing. During flight, the UAV locates and identifies micro-objects on the ground. The micro-feature dataset is constructed using orthophotos of the study area collected by the UAV, and randomly divided into training, validation, and test sets in a 6:2:2 ratio. The risk classification of areas with the highest micro-feature density involves statistically comparing the number and type of each type of real and predicted micro-features at both the community and sub-regional scales. Kernel density values ​​are calculated to identify and mark the areas with the highest micro-feature density in the empirical area, as well as the sub-regions where the number of micro-features needs to be reduced the most. Risk classification is then performed based on the maximum kernel density and the regional average. The micro-feature dataset was constructed by acquiring high-resolution images using drone photography, constructing orthorectified TIFF images using DJ Terra, cropping and converting them into 960×960 pixel JPEG images, and manually creating labels for nine micro-features using LabelImg. Specifically, for each image, real polygon labels for nine micro-features were drawn, namely: aquatic plants, trash cans, bowls, boxes, buckets, earthenware pots, lids, flower pots, and troughs, forming a dataset with labeled information. The process involves identifying and marking the areas with the highest density of micro-features for risk classification. Based on the model identification results, a distribution map of micro-features is drawn, and a kernel density map is created to identify the areas with the highest density of micro-features. Based on the maximum kernel density and the regional average, the areas are classified into three risk levels: I, II, and III. The identification and marking of areas with the highest density of micro-features for risk classification employs spatial interpolation for kernel density analysis. Specifically, after obtaining the coordinate information of the micro-features, kernel density analysis is performed using spatial interpolation technology on the QGIS software platform. Kernel density is the result of weighted summation of all point data within a certain radius or bandwidth around a center point. In this process, points that are closer are assigned greater weight, and the unit of measurement for kernel density is set to per square meter. Through kernel density analysis, high-density areas form hotspots on the map, providing guidance for the removal of micro-features. 2.The residential area micro-objects identification and risk classification method based on UAV images according to claim 1, wherein, The training of the micro-feature recognition model adopts the transfer learning method. The network is initialized using the pre-trained weights "yolov7_training.pt" provided by the YOLOv7 official website, and the model is trained from the initial state.

Images