Small target detection method and system based on unmanned aerial vehicle image, and storage medium
By introducing spatial depth convolution and adaptive feature fusion modules into the YOLOv8 framework, the problem of small object detection accuracy and low efficiency in drone images is solved, and efficient and accurate small object detection is achieved, which is suitable for real-time applications.
Patent Information
- Application Number
- CN202411108094.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-15
AI Technical Summary
The multi-scale, high proportionality and overlap of small targets in drone images lead to low detection accuracy and efficiency, and existing algorithms are difficult to meet the real-time processing needs.
Based on the YOLOv8 object detection framework, spatial depth convolution and adaptive feature fusion strategies are introduced to build an efficient object detection feature pyramid model, and by adjusting the detection head, cropping feature map output, using spatial depth convolution and adaptive feature fusion modules , enhance the detection ability of small targets.
It significantly improves the detection accuracy and robustness of small targets in the drone image, reduces the amount of parameters, realizes the model lightweight, is suitable for real-time small target detection tasks, improves detection accuracy and the parameter quantity is only 39 of the original YOLOv8 %.
Smart Images

Figure CN118968362A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection, and in particular to a small target detection method, system and storage medium based on unmanned aerial vehicle images. Background Art
[0002] In today's daily life, surveillance is everywhere, especially in crowded places such as squares, stations, residential areas, and traffic roads, where a large number of cameras are distributed for real-time monitoring. The combination of the Internet of Things and drones can be used for real-time video surveillance. Due to the development of advanced sensor technology and the Internet of Things, the types and number of monitoring equipment are increasing, the images are becoming clearer, and the types, quantity, and quality of the collected related image data are constantly increasing and improving. It is impractical to manually process pictures or videos with these huge amounts of image data, and manual processing methods of pictures, videos, etc. will also result in omissions and errors. With the improvement of deep learning algorithms based on convolutional neural networks (CNNs) in detection rate and recognition accuracy, image processing is becoming more and more efficient. Applying deep learning to target detection in drone aerial images and extracting image features can effectively improve detection rate and detection accuracy.
[0003] Existing deep learning-based object detection algorithms are mainly divided into two categories: two-stage detection algorithms based on candidate regions and single-stage detection methods based on end-to-end training. Two-stage detection algorithms, such as R-CNN and its variants Fast-RCNN and Faster-RCNN, generate candidate regions through a region proposal network (RPN), and then classify and regress each candidate region into a bounding box. Although these algorithms have achieved good results in detection accuracy, their computational efficiency is low and it is difficult to meet the requirements of real-time or near-real-time processing of drones. Single-stage detection algorithms, such as SSD and YOLO series, can complete target detection through a single forward propagation and have a high detection speed. In particular, the YOLO series of algorithms achieves fast and relatively accurate target detection by converting the target detection task into a regression problem. However, target detection in drone images faces a series of unique challenges, which mainly stem from the particularity of drone shooting conditions. First, the targets in drone images are usually small in scale. This is because drones are often shot at high altitudes, resulting in targets on the ground appearing smaller in size in the image. Secondly, due to the large changes in the shooting angle and height of the drone, the scale of the target in the image is also more complex, which increases the difficulty of the detection algorithm. In addition, there may be a high degree of overlap between the targets in the drone image, and the occlusion is serious, which further reduces the accuracy of the detection. Summary of the invention
[0004] In view of the challenges of multi-scale, high-proportion, overlapping and poor quality of some images of small targets in drone images, the present invention proposes a small target detection method, system and storage medium based on drone images, and makes innovative improvements on the existing YOLOv8 target detection framework. By introducing advanced spatial deep convolution technology and adaptive feature fusion strategy, an efficient target detection feature pyramid model is constructed. These improvements make the method of the present invention very suitable for application in the field of drone image target detection, which can meet real-time requirements and provide accurate detection results. While maintaining a high detection speed, the method of the present invention significantly improves the detection accuracy and robustness of small targets.
[0005] The technical solution provided by the present invention is:
[0006] On the one hand, the present invention provides a small target detection method based on drone images, comprising:
[0007] Acquire a drone image dataset, and preprocess the drone images in the dataset;
[0008] Set the prior box based on the size of the target box in the dataset;
[0009] A small target detection model based on an improved YOLOv8 model is constructed, and the small target detection model is trained using the preprocessed data set; the improved YOLOv8 model includes: deleting the C5 downsampling layer, adding a shallower feature map as a P2 detection layer, adding convolution and upsampling operations after the first downsampling to further expand the feature map; and then merging the obtained feature map with the feature map extracted by the network backbone to obtain a larger feature map for small target detection;
[0010] The trained small target detection model is used to detect small targets in UAV images, and the non-maximum suppression algorithm is used to screen the final prediction box to obtain the small target detection results.
[0011] Furthermore, the drone images in the data set are preprocessed, including:
[0012] Scale, crop, and normalize the image to fit the model’s input requirements;
[0013] Data augmentation through random rotation, scaling, cropping, and color transformation;
[0014] Parse the metadata of the image to extract the target category and location information, and convert the extracted target category and location information into scale coordinates relative to the entire image;
[0015] Shuffle the order of images in the dataset.
[0016] Furthermore, the k-means clustering algorithm is used to divide the object box sizes in the dataset into multiple clusters, each cluster represents a possible bounding box size and aspect ratio, and a set of prior boxes are determined;
[0017] The determined prior boxes are placed at different positions in the feature map so that the model can detect objects of different scales.
[0018] Furthermore, the improved YOLOv8 model also includes: using spatial depth convolution to replace traditional convolution and pooling to retain detail information of low-resolution images and small objects;
[0019] The operation process of the spatial depth convolution includes:
[0020] Input a feature map X of arbitrary size S×S×C and divide it into scale 2 sub-mapping sequence, scale is an adjustable factor;
[0021] For the original feature map X, the sub-map f x,y It consists of all items X(i, j) where i+x and j+y are divisible by scale; it is equivalent to each sub-map being obtained by downsampling by a multiple of scale; then all sub-feature maps are concatenated along the channel dimension (a total of scale 2 ), and obtain a new feature map
[0022] Further, the obtained feature map is merged with the feature map extracted by the network backbone, including: using an adaptive feature fusion module based on a feature pyramid model to adaptively fuse feature maps at different levels;
[0023] The calculation formula of the adaptive fusion module includes:
[0024] y1,y2,y3=split((concat(x1,x2,x3)));
[0025] W1=f (1×1) (y1);
[0026] W2=f (3×3) (y2+W1);
[0027] W3=f (3×3) (y3+W2);
[0028] output = Concat(W1, W2, W3);
[0029] where f (1×1) and f (3×3)They represent 1×1 convolution and 3×3 convolution operations respectively; Split and Concat represent the division and concatenation operations of channels respectively; channel dimension reduction is performed through 1×1 convolution; the shallow feature map is added to the middle-level information after 3×3 convolution. The network can emphasize the detail information in the shallow features and fuse it into the middle-level features. Similarly, the fused middle and high-level features are continued to be added after 3×3 convolution; finally, the fused feature maps are spliced and combined into a global feature map for final output.
[0030] Furthermore, the non-maximum suppression algorithm is used to screen the final prediction box, including:
[0031] Sort all prediction boxes in descending order according to their confidence scores;
[0032] Select the prediction box with the highest score as the first prediction box to be retained;
[0033] For the remaining prediction boxes, calculate their intersection-over-union ratio with the retained prediction boxes;
[0034] Set an IoU threshold. If the IoU of the remaining prediction boxes with the retained prediction boxes is greater than this threshold and their scores are lower than those of the retained prediction boxes, these low-scoring boxes are suppressed.
[0035] Repeatedly select the box with the highest score among the remaining boxes as the next retained prediction box, and suppress the low-scoring box whose intersection-over-union ratio exceeds the threshold until all prediction boxes have been processed.
[0036] Furthermore, it also includes: visualizing the detection results, displaying the bounding box and category label predicted by the model directly on the original image, and intuitively showing the detection results.
[0037] Furthermore, the detection results are visualized, including:
[0038] Read predicted bounding box coordinates, class labels, and confidence scores;
[0039] Draw bounding boxes using different colors on the original image and display the class label and confidence score inside or near the box.
[0040] In another aspect, the present invention further provides a small target detection system based on drone images, comprising:
[0041] A data acquisition module is used to acquire a UAV image data set and preprocess the UAV images in the data set;
[0042] The prior frame setting module sets the prior frame based on the size of the target frame in the dataset;
[0043] A model building module, constructing a small target detection model based on an improved YOLOv8 model, and training the small target detection model using a preprocessed data set; the improved YOLOv8 model includes: deleting a C5 downsampling layer, adding a shallower feature map as a P2 detection layer, adding convolution and upsampling operations after the first downsampling, and further expanding the feature map; and then merging the obtained feature map with the feature map extracted by the network backbone to obtain a larger feature map for small target detection;
[0044] The target detection module uses the trained small target detection model to detect small targets in drone images, and uses the non-maximum suppression algorithm to screen the final prediction box to obtain the small target detection results.
[0045] On the other hand, the present invention also provides a computer-readable storage medium, which stores a computer instruction set. When the computer instruction set is executed by a processor, the above-mentioned small target detection method based on drone images is implemented.
[0046] Compared with the prior art, the present invention has the following beneficial effects:
[0047] The present invention provides a small target detection method based on drone images. On the basis of the original YOLOv8, the detection head is adjusted to improve the detection accuracy of small targets, while reducing the number of parameters. The feature map output of the large target detection layer is cropped, and the spatial depth convolution is used to replace the original convolution step and pooling operation to better retain the detail information of low-resolution images and small objects. An adaptive feature fusion module is designed to adaptively fuse feature maps of different levels to enhance the feature expression capability. A feature pyramid model is constructed, and the shallow features of the backbone and the middle and deep features of the neck are effectively fused through the adaptive feature fusion module, so that the information of different levels is fully utilized to improve the detection accuracy and robustness of small targets. The present invention improves the detection accuracy of small targets in drone images, and the number of parameters is only 39% of the original YOLOv8, which realizes the lightweight model and is more suitable for real-time small target detection tasks. The present invention has broad application prospects in multiple fields such as agricultural monitoring, urban planning, and traffic management, and can significantly improve the intelligent operation level of drones, and has important commercial value and market potential. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0049] Figure 1 This is a flow chart of a small target detection method based on drone images in an embodiment of the present invention;
[0050] Figure 2 A structural diagram of a model detection head in an embodiment of the present invention;
[0051] Figure 3 4 is a structural diagram of a multi-scale fusion method in an embodiment of the present invention. DETAILED DESCRIPTION
[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0054] In response to the challenges of multi-scale, high-ratio, overlapping and poor image quality of small targets in drone images, this paper makes innovative improvements on the existing YOLOv8 target detection framework. By optimizing the detection head, introducing spatial deep convolution and adaptive feature fusion modules, the accuracy and speed of small target detection are significantly improved.
[0055] YOLOv8 is a new generation of single-stage target detection framework further developed and improved by Ultralytics based on YOLOv5 and released in 2023. It inherits the advantages of fast speed and high precision of the YOLO series of algorithms, and has made innovations and improvements in many aspects to adapt to a wider range of application scenarios and higher performance requirements. The backbone network and Neck part refer to the design ideas of YOLOv7ELAN, replacing the C3 structure of YOLOv5 with a C2f structure with richer gradient flow, and adjusting the number of channels for different scale models. It is a careful fine-tuning of the model structure, no longer a mindless set of parameters for all models, which greatly improves the model performance. Compared with YOLOv5, the head part has been greatly changed. It has been replaced with the current mainstream decoupled head structure, separating the classification and detection heads, and also changed from Anchor-Based to Anchor-Free. In terms of loss calculation, the TaskAlignedAssigner positive sample allocation strategy is adopted, and the Distribution Focal Loss is introduced. The data enhancement part of the training introduced the operation of turning off Mosiac enhancement in the last 10 epochs in YOLOX, which can effectively improve the accuracy.
[0056] The main features and structure of YOLOv8 are as follows:
[0057] 1. Model architecture: YOLOv8 inherits and develops the core concept of the YOLO series, providing faster detection speed and higher accuracy. It supports real-time object detection on images of different sizes and resolutions.
[0058] 2. Backbone network: YOLOv8 adopts an improved backbone network design and introduces a new network structure C2f to enhance the ability of feature extraction. The C2f structure optimizes the performance of the model through carefully designed channel and layer configuration.
[0059] 3. Decoupled-Head: The Head part of YOLOv8 has been significantly improved. The decoupled head structure is adopted to separate the classification and detection tasks, which improves the flexibility and accuracy of the model.
[0060] 4. Anchor-Free Mechanism: Unlike YOLOv5, YOLOv8 adopts the Anchor-Free mechanism, which helps to reduce the dependence on the size of the anchor box and improve the detection performance of small objects.
[0061] 5. Loss calculation: YOLOv8 introduces new loss functions, including Task-Aligned Assigner and Distribution Focal Loss (DFL). These improvements help train the model more efficiently and improve detection accuracy.
[0062] The YOLOv8 detection part can be mainly divided into the following modules:
[0063] (1) Backbone. This network is responsible for extracting key features from the input image. It generates feature maps of the image through convolution (conv), activation functions (such as ReLU), and pooling operations. In order to speed up detection, YOLOv8 chose ResNet as its Backbone. ResNet solves the gradient vanishing problem in deep network training through deep residual connections, while keeping the network lightweight, ensuring that high detection accuracy is maintained while reducing computational and memory requirements.
[0064] (2) Head network. The function of the head network is to generate the final prediction results from the features extracted by the backbone network. It processes the different levels of features of the backbone network and finally extracts the category and location information of the target from three feature maps of different resolutions.
[0065] (3) Neck module. The neck module adjusts the size of the feature map through upsampling and downsampling techniques to facilitate the fusion of features at different levels. This process not only obtains multi-scale feature maps, but also promotes the effective combination of shallow geometric features to deep semantic features. This fusion strategy goes beyond simple feature concatenation and can extract richer and more comprehensive feature representations.
[0066] (4) CSP structure. CSPNet improves computational efficiency by splitting feature maps and processing them in parallel, with one part performing convolution operations and the other part concatenating with the convolution results. In classification tasks, CSPNet helps reduce the computational burden, while in object detection tasks, as part of Backbone, CSPNet significantly improves the performance of the model while maintaining a low computational cost.
[0067] like Figure 1 As shown, a small target detection method based on drone images in an embodiment of the present invention specifically includes the following steps:
[0068] A. Image preprocessing and initialization: extract target information, disrupt the order, convert the scale, enhance the data, and normalize the data.
[0069] B. Prior box setting: Use a clustering algorithm to cluster the target boxes in the training set, and determine the size of the anchor box based on the clustering results. Data-based statistical methods help adapt to targets of different sizes and improve detection results.
[0070] C. Detection head adjustment: The YOLOv8 model adjusts the detection head to improve the accuracy of small target detection and reduce the number of parameters. The feature map output of the large target detection layer is cropped and the small target detection head is added to enhance the model's detection ability for small targets.
[0071] D. Spatial depth convolution: Use spatial depth convolution instead of traditional convolution and pooling to retain the detail information of low-resolution images and small objects.
[0072] E. Multi-scale feature fusion: Design an adaptive feature fusion module to adaptively fuse feature maps at different levels, extract features of different scales, enhance feature expression capabilities, enrich feature expression, and improve the model's ability to detect multi-scale targets.
[0073] F. Non-maximum suppression processing: The prediction boxes are screened through the non-maximum suppression algorithm to ensure that only one optimal prediction box is retained for each target, thereby improving the accuracy of detection.
[0074] G. Result visualization: Draw the prediction box on the original image and mark the prediction score to intuitively display the target detection results.
[0075] Seven steps are performed in the image training and target detection process. First, the data set is preprocessed in steps AB. Then, fine feature extraction and fusion are performed in step CE to improve the ability to extract target features in drone images and learn the features of small targets. In step F, redundant bounding boxes are removed, and in step G, the detection results are visualized.
[0076] The steps are as follows:
[0077] Step A: Initialization:
[0078] The VisDrone2019 dataset used in the embodiment of the present invention contains 8629 static images (6471 for training, 548 for verification, and 1610 for testing), where the maximum resolution of the image is 2000×1500.
[0079] The dataset is preprocessed before training. First, the images are scaled, cropped, and normalized to fit the model’s input requirements. Next, to enhance the model’s generalization capabilities, the training data is augmented through data augmentation techniques such as random rotation, scaling, cropping, and color transformation. At the same time, the image’s metadata is parsed to extract the target category and location information, which is then converted to scale coordinates relative to the entire image to facilitate model training. To prevent overfitting, the training data is randomly shuffled. The images are also standardized by subtracting the mean and dividing by the standard deviation to stabilize the distribution of the model input. Finally, the processed data is loaded into memory and processed in batches to provide an efficient input flow for model training. These comprehensive preprocessing steps lay the foundation for model training, allowing the model to learn effective features from the data.
[0080] Step B: Setting of prior frame:
[0081] Anchor Boxes are predefined bounding boxes used to locate and classify objects during model prediction. The size and aspect ratio of these bounding boxes are optimized based on the target statistics in the dataset to ensure that the model can cover objects of different sizes and shapes. The target box sizes in the training set are analyzed using the k-means clustering algorithm to determine a set of anchor boxes. The algorithm divides the target box sizes into multiple clusters, each representing a possible bounding box size and aspect ratio. Through cluster analysis, a set of anchor boxes that cover objects of various sizes and ratios in the training data can be determined. These anchor boxes are then placed at different locations in the feature map so that the model can detect objects of different scales.
[0082] By using these prior boxes, the model can quickly locate and classify objects on the feature map without the need for the model to learn the size and shape of the bounding box itself. This not only improves the detection speed, but also enhances the model's adaptability to objects of different sizes.
[0083] Step C: Adjust the detection head:
[0084] In the original YOLOv8 model, the C5 downsampling layer occupies a large number of parameters, so the present invention deletes this layer structure, which can not only reduce a large number of parameters to maintain lightweight, but also reduce the part of the feature information lost during the downsampling of the feature map. After deleting the original C5 downsampling layer, it is still not possible to improve the accuracy of small object detection by the YOLOv8 algorithm. The reason may be that when the downsampling multiple is too large, there will be problems such as loss of feature expression and disappearance of fine structure. In addition, according to the characteristics of the data set containing a large number of small-sized objects (the area of micro-small targets is less than 8×8 pixels, the area of small targets is between 8×8 and 32×32 pixels, the area of medium-sized targets is between 32×32 and 96×96 pixels, and the area of large targets is greater than 96×96 pixels). Therefore, a shallower feature map is newly added as the P2 detection layer, and operations such as convolution and upsampling are added after the first downsampling to further expand the feature map, such as Figure 2 The obtained feature map is then merged with the feature map extracted by the network backbone to obtain a larger feature map for small target detection, making the network more sensitive to small objects in high-resolution images and enhancing the network's detection ability.
[0085] Step D, spatial depth convolution:
[0086] Spatial deep convolution is a special type of convolution operation, which has been widely used in the field of deep learning. It is mainly used for image processing tasks such as image classification, object detection, pose estimation, motion recognition, etc. It converts the input image data from the spatial dimension to the depth dimension. It is a spatial downsampling method that can effectively reduce the spatial resolution of the image while increasing the number of channels, thereby increasing the nonlinearity and sparsity of the network, and improving the representation ability and computational efficiency of the model.
[0087] Input a feature map X of arbitrary size S×S×C and divide it into scale 2 A sequence of sub-maps (scale is an adjustable factor):
[0088] f 0,0 =X[0:S:scale,0:S:scale],......,f scale-1,0 =X[scale-1:S:scale,0:S:scale];
[0089] f 0,1 =X[0:S:scale,1:S:scale],......,f scale-1,1 =X[scale-1:S:scale,1:S:scale]; ......
[0091] f 0,scale-1 =X[0:S:scale,0:S:scale],......,f scale-1,scale-1 =X[scale-1:S:scale,scale-1:S:scale].
[0092] For the original feature map X, the sub-map f x,y It can be composed of all items X(i,j) where i+x and j+y are divisible by scale. This is equivalent to each sub-map being obtained by downsampling by a multiple of scale. Then all sub-feature maps are concatenated along the channel dimension (a total of scale 2 ), and obtain a new feature map
[0093] Spatial depth convolution divides the input image data into multiple blocks, arranges the pixels in each block in the depth dimension, and then combines these blocks into new image data in a certain order. This operation can reduce the spatial dimension of the input data while increasing the depth dimension, so that the network can better detect objects of different sizes and improve the efficiency and accuracy of object detection.
[0094] Step E: Multi-scale feature fusion:
[0095] In the implementation of the present invention, the shallow feature map obtained by the backbone is fused with the middle and deep feature maps of the neck part, so that multi-level information can be combined at the same time. When these feature maps are fused, an adaptive fusion module (Adaptive Fusion) is designed, which can not only keep the original number of channels unchanged without adding too many parameters, but also perform multi-level feature fusion and make full use of multi-level information. The network structure of the adaptive fusion module is as follows Figure 3 The calculation formula is shown in (1)-(5):
[0096] y1,y2,y3=split((concat(x1,x2,x3))) (1)
[0097] W1=f (1×1) (y1) (2)
[0098] W2=f (3×3) (y2+W1) (3)
[0099] W3=f (3×3) (y3+W2) (4)
[0100] output=Concat(W1,W2,W3) (5)
[0101] where f(×1) and f (×3) Represent 1×1 convolution and 3×3 convolution operations respectively. Split and Concat represent the division and concatenation operations of channels respectively. Concatenating feature maps from different levels (shallow, middle, and deep) enables the network to consider information of multiple scales at the same time, which helps detect features of different sizes and levels, thereby improving the performance of the network in different object sizes and scenes. Channel dimension reduction through 1×1 convolution reduces computational costs, helps the network learn features more effectively and improves computational efficiency. After the shallow feature map is convolved with the middle layer information after 3×3 convolution, the network can emphasize the detail information in the shallow features and fuse it into the middle layer features. Similarly, the fused middle layer and high layer features are continued to be convolved with 3×3 and then added. This hierarchical processing can improve the adaptability of the network. In addition, by fusing information of different scales and levels, the network can better understand the local and global information of the image, thereby improving the performance of object detection and classification. Finally, the fused feature maps are concatenated and combined into a global feature map for final output.
[0102] Step F: Non-maximum suppression processing:
[0103] In the detection of drone aerial photography using YOLOv8, non-maximum suppression (NMS) is a key post-processing step for screening the final prediction box. First, all prediction boxes are sorted in descending order according to the confidence score. The prediction box with the highest score is selected as the first prediction box to be retained. Then, for the remaining prediction boxes, their intersection over union (IOU) with the retained prediction box is calculated. An IOU threshold is set. If the IOU of the remaining prediction box with the retained prediction box is greater than this threshold, it means that they may be different predictions of the same target. For these prediction boxes whose IOU with the retained prediction box exceeds the threshold, if their scores are lower than the retained prediction box, these low-scoring boxes are suppressed. Repeatedly select the highest-scoring box among the remaining boxes as the next retained prediction box, and suppress the low-scoring box whose IOU exceeds the threshold until all prediction boxes have been processed. This processing can remove redundant bounding boxes and ensure that each target has only one accurate prediction box.
[0104] Step G, result visualization: Finally, visualize the final detection results, and display the bounding box and category label predicted by the model directly on the original image, which can intuitively show the effect of the detection. Specifically, it is to read the predicted bounding box coordinates, category label, and confidence score. Then, draw the bounding box with different colors on the original image, and display the category label and confidence score in or near the box. This makes the detection effect clear at a glance and can help users evaluate the overall performance of the model.
[0105] A small target detection method based on drone images in the above embodiment improves the detection accuracy of small targets by adjusting the detection head on the basis of the original YOLOv8, while reducing the amount of parameters. The feature map output of the large target detection layer is cropped, and the spatial depth convolution is used to replace the original convolution step and pooling operation to better retain the detail information of low-resolution images and small objects. An adaptive feature fusion module is designed to adaptively fuse feature maps of different levels to enhance the feature expression capability. A feature pyramid model is constructed, and the shallow features of the backbone and the middle and deep features of the neck are effectively fused through the adaptive feature fusion module, making full use of information at different levels to improve the detection accuracy and robustness of small targets. This method improves the detection accuracy of small targets in drone images, and the parameter amount is only 39% of the original YOLOv8, realizing the lightweight model, which is more suitable for real-time small target detection tasks. The present invention has broad application prospects in multiple fields such as agricultural monitoring, urban planning, and traffic management, and can significantly improve the intelligent operation level of drones, and has important commercial value and market potential.
[0106] Corresponding to the small target detection method based on drone images in the above embodiment, the present invention also provides a small target detection system based on drone images, comprising:
[0107] A data acquisition module is used to acquire a UAV image data set and preprocess the UAV images in the data set;
[0108] The prior frame setting module sets the prior frame based on the size of the target frame in the dataset;
[0109] A model building module, constructing a small target detection model based on an improved YOLOv8 model, and training the small target detection model using a preprocessed data set; the improved YOLOv8 model includes: deleting a C5 downsampling layer, adding a shallower feature map as a P2 detection layer, adding convolution and upsampling operations after the first downsampling, and further expanding the feature map; and then merging the obtained feature map with the feature map extracted by the network backbone to obtain a larger feature map for small target detection;
[0110] The target detection module uses the trained small target detection model to detect small targets in drone images, and uses the non-maximum suppression algorithm to screen the final prediction box to obtain the small target detection results.
[0111] For a small target detection system based on drone images in an embodiment of the present invention, since it corresponds to a small target detection method based on drone images in the above embodiment, the description is relatively simple. For relevant similarities, please refer to the description of a small target detection method based on drone images in the above embodiment, which will not be described in detail here.
[0112] An embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer instruction set. When the computer instruction set is executed by a processor, a small target detection method based on drone images as provided in any of the above embodiments is implemented.
[0113] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0115] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0116] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A small target detection method based on drone images, It is characterized in that include: Acquire a drone image dataset, and preprocess the drone images in the dataset; Set the prior box based on the size of the target box in the dataset; A small target detection model based on an improved YOLOv8 model is constructed, and the small target detection model is trained using the preprocessed data set; the improved YOLOv8 model includes: deleting the C5 downsampling layer, adding a shallower feature map as a P2 detection layer, adding convolution and upsampling operations after the first downsampling, and further expanding the feature map; The obtained feature map is then merged with the feature map extracted by the network backbone to obtain a larger feature map for small object detection; The trained small target detection model is used to detect small targets in UAV images, and the non-maximum suppression algorithm is used to screen the final prediction box to obtain the small target detection results.
2. According to the small target detection method based on drone images described in claim 1, It is characterized in that The drone images in the dataset are preprocessed, including: Scale, crop, and normalize the image to fit the model’s input requirements; Data augmentation through random rotation, scaling, cropping, and color transformation; Parse the metadata of the image to extract the target category and location information, and convert the extracted target category and location information into scale coordinates relative to the entire image; Shuffle the order of images in the dataset.
3. According to the small target detection method based on drone images as described in claim 1, It is characterized in that Use the k-means clustering algorithm to divide the target box sizes in the dataset into multiple clusters, each cluster represents a possible bounding box size and aspect ratio, and determine a set of prior boxes; The determined prior boxes are placed at different positions in the feature map so that the model can detect objects of different scales.
4. According to the small target detection method based on drone images as described in claim 1, It is characterized in that The improved YOLOv8 model also includes: using spatial depth convolution to replace traditional convolution and pooling to retain detail information of low-resolution images and small objects; The operation process of the spatial depth convolution includes: Input a feature map X of arbitrary size S×S×C and divide it into scale 2 sub-mapping sequence, scale is an adjustable factor; For the original feature map X, the sub-map f x,y It consists of all items X(i, j) where i+x and j+y are divisible by scale; it is equivalent to each sub-map being obtained by downsampling by a multiple of scale; then all sub-feature maps are concatenated along the channel dimension (a total of scale 2 ), and obtain a new feature map 5. According to the small target detection method based on drone images as described in claim 1, It is characterized in that The obtained feature map is merged with the feature map extracted by the network backbone, including: using an adaptive feature fusion module based on a feature pyramid model to adaptively fuse feature maps at different levels; The calculation formula of the adaptive fusion module includes: y 1 ,y 2 ,y 3 =split((concat(x 1 ,x 2 ,x 3 ))); W 1 =f (1×1) (y 1 ); W 2 =f (3×3) (y 2 +W 1 ); W 3 =f (3×3) (y 3 +W 2 ); output=Concat(W 1 ,W 2 ,W 3 ); where f (1×1) and f (3×3) They represent 1×1 convolution and 3×3 convolution operations respectively; Split and Concat represent the division and concatenation operations of channels respectively; channel dimension reduction is performed through 1×1 convolution; the shallow feature map is added to the middle-level information after 3×3 convolution. The network can emphasize the detail information in the shallow features and fuse it into the middle-level features. Similarly, the fused middle and high-level features are continued to be added after 3×3 convolution; finally, the fused feature maps are spliced and combined into a global feature map for final output.
6. The small target detection method based on drone images according to claim 1, It is characterized in that The non-maximum suppression algorithm is used to screen the final prediction box, including: Sort all prediction boxes in descending order according to their confidence scores; Select the prediction box with the highest score as the first prediction box to be retained; For the remaining prediction boxes, calculate their intersection-over-union ratio with the retained prediction boxes; Set an IoU threshold. If the IoU of the remaining prediction boxes with the retained prediction boxes is greater than this threshold and their scores are lower than those of the retained prediction boxes, these low-scoring boxes are suppressed. Repeatedly select the box with the highest score among the remaining boxes as the next retained prediction box, and suppress the low-scoring box whose intersection-with-union ratio exceeds the threshold until all prediction boxes have been processed.
7. The small target detection method based on drone images according to claim 1, It is characterized in that Also includes: Visualize the detection results and display the bounding boxes and category labels predicted by the model directly on the original image to intuitively show the detection results.
8. The small target detection method based on drone images according to claim 7, It is characterized in that Visualize the detection results, including: Read predicted bounding box coordinates, class labels, and confidence scores; Draw bounding boxes using different colors on the original image and display the class label and confidence score inside or near the box.
9. A small target detection system based on drone images, It is characterized in that include: A data acquisition module is used to acquire a UAV image data set and preprocess the UAV images in the data set; The prior frame setting module sets the prior frame based on the size of the target frame in the dataset; A model building module, constructing a small target detection model based on an improved YOLOv8 model, and training the small target detection model using a preprocessed data set; the improved YOLOv8 model includes: deleting a C5 downsampling layer, adding a shallower feature map as a P2 detection layer, adding convolution and upsampling operations after the first downsampling, and further expanding the feature map; and then merging the obtained feature map with the feature map extracted by the network backbone to obtain a larger feature map for small target detection; The target detection module uses the trained small target detection model to detect small targets in drone images, and uses the non-maximum suppression algorithm to screen the final prediction box to obtain the small target detection results.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer instruction set, and when the computer instruction set is executed by the processor, a small target detection method based on drone images as described in any one of claims 1 to 8 is implemented.
Citation Information
Cited By
Human-changed gas station monitoring method based on visual algorithm
CN119360312A
Grassland mouse hole identification method, device and equipment and storage medium
CN119942383A
A method, apparatus, device, and storage medium for identifying grassland rodent burrows.
CN119942383B
Unmanned aerial vehicle micro-target feature decoupling detection method, system, device and medium
CN122821418A
Feature decoupling detection methods, systems, equipment, and media for small targets on unmanned aerial vehicles (UAVs).
CN122821418B
