Unmanned aerial vehicle visual angle dense small target detection method and device based on deep learning

Abstract

The invention discloses an unmanned aerial vehicle visual angle dense small target detection method and device based on deep learning, and belongs to the technical field of target detection. Aiming at the problem that a YOLOv7 network is insufficient in dense small target feature extraction capability in the prior art, the method comprises the following steps: acquiring an unmanned aerial vehicle image data set, acquiring a dense small target image from the unmanned aerial vehicle image data set, and preprocessing the dense small target image; constructing a dense small target detection model based on YOLOv7, wherein the dense small target detection model comprises a backbone network, a neck network and a head network; wherein a small target detection head is added in the dense small target detection model, and a dense detection module is introduced in the head network; and detecting the dense small target image through the dense small target detection model to obtain a detection result. Therefore, by improving a YOLOv7 detection head and introducing a dense detection module, secondary reasoning on a dense small target area is increased, and the dense small target detection effect is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and more specifically, to a method and device for detecting dense small targets from the perspective of an unmanned aerial vehicle based on deep learning. Background Art

[0002] In recent years, with the development of deep learning theory and hardware equipment, various tasks in the field of computer vision have shown a trend of transitioning from general scenarios to specific scenarios. UAVs, due to their convenient controllability and flexible maneuverability, can effectively combine UAVs with various technologies in the field of vision after being equipped with cameras, greatly expanding the use scenarios of UAVs, and are gradually being applied to tasks in various specific scenarios.

[0003] As one of the core tasks of computer vision, object detection plays an important role in social production practice. However, the object detection task from the perspective of drones faces many challenges, especially in the detection of small objects and dense areas. Due to the size and load-bearing limitations of drones, the computing power and memory of deployed equipment are often very limited, which means that the algorithm needs to maintain high accuracy while also having a fast inference speed.

[0004] At present, the detection algorithm from the perspective of drones still has shortcomings in the detection effect of small targets and dense areas. On the one hand, due to the small scale of small targets, the number of pixels they occupy in the image is limited, which makes feature extraction difficult, detection accuracy is low, and missed detection is more serious. On the other hand, in dense areas, occlusion and overlap between targets are common, which further increases the difficulty of detection. Traditional target detection algorithms often find it difficult to effectively deal with these problems, resulting in a decrease in detection performance.

[0005] YOLOv7 is a YOLO (You Only Look Once) series of object detection algorithms. It inherits the core idea of ​​the YOLO series of algorithms, which is to predict the position and category of all objects in the image at one time, thereby achieving fast and accurate object detection. Compared with previous versions, YOLOv7 has made multiple optimizations in algorithm structure and performance.

[0006] Although YOLOv7 has made significant progress in the field of target detection, its effect is still somewhat lacking in the detection of small targets and dense areas from the perspective of drones. First, the target detection task from the perspective of drones has its own particularity. Due to the high flying altitude of drones, the targets in the captured images are often small and densely distributed, which poses a great challenge to the detection of small targets. Although YOLOv7 adopts an efficient network structure and optimized data enhancement strategy, when dealing with small targets, due to its less feature information, the model often finds it difficult to accurately capture the features of the target, resulting in a decrease in detection accuracy. Secondly, the detection of dense areas is also a major problem faced by YOLOv7. In dense areas, occlusion and overlap between targets are common, which makes it difficult for the model to distinguish different targets. Although YOLOv7 improves the detection ability of the model by improving the feature fusion method and context information extraction strategy, when dealing with dense areas, the detection performance of the model will still be affected to a certain extent due to the mutual influence between targets.

[0007] In addition, images from the drone's perspective are often affected by environmental factors such as lighting and weather, which further increases the difficulty of target detection. It is still necessary to tune and improve the model for specific scenarios to improve the adaptability and detection accuracy of the model.

[0008] After searching, the Chinese patent application, application number 202211559260.3, published on May 9, 2023, discloses a method for detecting drone aerial images based on YOLOv5. The method includes: taking the YOLOv5 algorithm as the basic model framework, in order to improve the accuracy of small target detection in aerial images over the city, designing a network for extracting multiple context features; in order to improve the network's attention to dense areas, proposing a series cross self-attention algorithm, which is added between the backbone network and the three detection heads respectively, to further enhance the information of dense areas; obtaining the final network model through iterative training and parameter update, and then using multi-scale prediction to improve the small target detection performance, and finally obtaining the final result through the prediction of the three-scale detection heads. This method can effectively alleviate the problem of context information loss, enhance feature extraction capabilities, and capture more diverse feature spaces, but the YOLOv5 version used in this method is too low, and the adaptability of the main model itself is changed too little. Therefore, for dense small target detection, there is still a lot of room for improvement in robustness, convergence and accuracy. Summary of the invention

[0009] 1. Technical problems to be solved

[0010] In view of the problem that the YOLOv7 network in the prior art has insufficient feature extraction capability for dense small targets, the present invention provides a method and device for dense small target detection from a drone perspective based on deep learning. By improving the YOLOv7 detection head and adding secondary reasoning for dense small target areas, the dense small target detection effect is improved, and the generalization capability and accuracy of detection are enhanced.

[0011] 2. Technical solution

[0012] The purpose of the present invention is achieved through the following technical solutions.

[0013] A method for detecting dense small targets from a drone's perspective based on deep learning includes the following steps:

[0014] Collect UAV image datasets, obtain dense small target images from the UAV image datasets, and preprocess the dense small target images;

[0015] A dense small target detection model based on YOLOv7 is constructed. The dense small target detection model includes a backbone network, a neck network, and a head network. A small target detection head is added to the dense small target detection model, and a dense detection module is introduced into the head network.

[0016] The dense small target image is detected by the dense small target detection model to obtain the detection result.

[0017] As a further improvement of the present invention, preprocessing of dense small target images includes correction processing and brightness adjustment.

[0018] As a further improvement of the present invention, feature extraction is performed on dense small target images through a backbone network, a neck network and a head network to obtain common features.

[0019] As a further improvement of the present invention, the size of the dense small target image is increased based on the small target detection head, and the features of the dense small target image after the size increase are extracted through the backbone network, the neck network and the head network to obtain the detail features.

[0020] As a further improvement of the present invention, a dense detection module is used to determine whether there is a dense small target area in the dense small target image. If there is a dense small target area, the dense small target area is cropped and re-input into the backbone network, neck network and head network for feature extraction to obtain dense area features.

[0021] As a further improvement of the present invention, the dense small object image is judged by a dense detection module whether there is a dense small object area, and the steps include:

[0022] Input the dense small target image into the dense small target detection model for detection to obtain several target boxes;

[0023] After scaling several target frames, calculate the IOU values ​​between each target frame, and form an IOU matrix through the IOU values;

[0024] Set the IOU overlap threshold θ, filter out the IOU values ​​greater than or equal to the IOU overlap threshold θ in the IOU matrix and form the IOU matrix C;

[0025] Set the IOU connectivity threshold Φ and calculate the connected area of ​​the IOU matrix C. When the number of targets in the connected area is greater than the IOU connectivity threshold Φ, the connected area is a dense small target area.

[0026] As a further improvement of the present invention, the common features, detail features and dense area features are fused and then the detection result is output.

[0027] A device for detecting small targets from a dense drone perspective based on deep learning, comprising:

[0028] The image acquisition module collects UAV image data sets, obtains dense small target images from the UAV image data sets, and pre-processes the dense small target images;

[0029] Model building module, building a dense small target detection model based on YOLOv7, the dense small target detection model includes a backbone network, a neck network and a head network; wherein, a small target detection head is added to the dense small target detection model, and a dense detection module is introduced into the head network;

[0030] The detection module detects the dense small target image through the dense small target detection model to obtain the detection result.

[0031] A computer device comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements any of the above-mentioned methods when executing the computer program.

[0032] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above methods is executed.

[0033] 3. Beneficial effects

[0034] Compared with the prior art, the advantages of the present invention are:

[0035] (1) The present invention provides a method and device for detecting dense small targets from a drone perspective based on deep learning. By adding a small target detection head, the size of the dense small target image is increased, and features are extracted from the dense small target image after the size increase, so as to obtain more detailed features of the dense small target image. The dense small targets can be identified more accurately through the detailed features.

[0036] (2) The present invention discloses a method and device for detecting dense small targets from a drone perspective based on deep learning. A dense detection module is introduced to determine whether there is a dense small target area in a dense small target image. The overlap between different target frames is evaluated by calculating the IOU matrix. When a certain area is determined to be a dense small target area, the dense small target area is automatically cropped into independent images by the dense detection module, thereby decomposing the complex dense small target area into smaller parts that are easier to handle, so as to more accurately capture the features of each target. Afterwards, the independent images are input into the dense small target detection model again for secondary reasoning operations, so as to deeply explore the dense area features of the targets in the dense small target area, significantly improve the detection accuracy of the targets in the dense small target area, and enhance the robustness of the dense small target detection.

[0037] (3) The present invention provides a method and device for detecting dense small targets from a drone's perspective based on deep learning. By constructing a dense small target detection model, better results can be achieved by using only lightweight detectors for dense small target areas. Using detectors with simpler structures in the process of processing dense small target areas can greatly reduce the resources required for calculation, improve the detection speed of the dense small target detection model, and reduce the memory usage required in the detection process, thus avoiding unnecessary waste and effectively utilizing limited computing resources. This enables the practical application of an efficient dense small target detection model on dense small target images, and has strong practicality and wide applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Schematic diagram of the detection structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0040] Example

[0041] The present embodiment provides a method for detecting dense small targets from a drone perspective based on deep learning, comprising the following steps: collecting a drone image dataset, obtaining dense small target images from the drone image dataset, and preprocessing the dense small target images; constructing a dense small target detection model based on YOLOv7, wherein the dense small target detection model includes a backbone network, a neck network, and a head network; wherein a small target detection head is added to the dense small target detection model, and a dense detection module is introduced into the head network; the dense small target images are detected by the dense small target detection model to obtain detection results.

[0042] Specifically in this embodiment, the video data is shot by a drone, and the video data is frame-processed to obtain a drone image data set, and a dense small target image is obtained from the drone image data set. It should be noted that in this embodiment, during the shooting process of the drone, due to environmental interference, equipment factors, etc., the video data is often accompanied by various noises, which not only affect the visual effect, but may also have an adverse effect on subsequent tasks such as image segmentation and target detection. Therefore, it is necessary to perform denoising on the collected video data. In this embodiment, denoising is a prior art. After obtaining the dense small target image, the dense small target image is preprocessed. In this embodiment, the preprocessing includes correction processing and brightness adjustment. The correction processing includes using the existing DeblurGANv2 network to remove the dynamically blurred image in the dense small target image, thereby reducing the amount of calculation and improving the detection efficiency. Brightness adjustment is used to improve the brightness and clarity of the dense small target image, so that more target features can be extracted.

[0043] like Figure 1 As shown, a dense small target detection model based on YOLOv7 is constructed. The dense small target detection model includes a backbone network, a neck network and a head network. It is worth noting that in this embodiment, a small target detection head is added to the dense small target detection model, and a dense detection module is introduced into the head network.

[0044] Therefore, in this embodiment, when a dense small target image is input into a dense small target detection model, the backbone network, the neck network, and the head network perform feature extraction on the dense small target image to obtain common features. In this embodiment, the common features include shallow features and deep features of the dense small target image. It should be noted that, in this embodiment, feature extraction of dense small target images by the backbone network, the neck network, and the head network is a prior art.

[0045] Furthermore, in this embodiment, a small target detection head is added to the dense small target detection model. The size of the dense small target image is increased by the small target detection head. After the size of the dense small target image is increased, the backbone network, the neck network and the head network are used to extract features of the dense small target image after the size is increased to obtain more detailed features of the dense small target image. The detailed features can be used to more accurately identify the dense small targets.

[0046] Therefore, in this embodiment, the global information features of the dense small object image can be obtained through the common features and detail features.

[0047] Furthermore, in this embodiment, a dense detection module (Dense_Module module) is used to determine whether there is a dense small target area in the dense small target image. If there is a dense small target area, the dense small target area is cropped and then input into the backbone network, neck network and head network again for feature extraction to obtain dense area features.

[0048] Specifically, a dense small target image is input into a dense small target detection model for detection to obtain several target frames. Then, the obtained several target frames are scaled based on their original size and a preset scaling parameter σ. It should be noted that in this embodiment, by scaling the target frame, it is convenient to increase the overlapping area between the target frames when calculating the intersection-over-union (IOU) in subsequent steps, so as to avoid the situation where the target frames are dense but do not intersect. In this embodiment, there is no fixed range for the preset scaling parameter σ, and it needs to be adjusted according to the situation of different reasoning scenarios. After scaling, all target frames are merged, the IOU values ​​between the target frames are calculated, and an IOU matrix is ​​formed by the IOU values. It should be noted that in this embodiment, all target frames are merged into a preset list to facilitate the subsequent traversal of all target frames in the list.

[0049] In this embodiment, the IOU overlap threshold θ is set, and the IOU values ​​greater than or equal to the IOU overlap threshold θ in the IOU matrix are screened out to form the IOU matrix C. Specifically, the state where the IOU value is lower than the IOU overlap threshold θ is set to 0, and the state where the IOU value is greater than or equal to the IOU overlap threshold θ is set to 1, so that the IOU values ​​that meet the IOU overlap threshold θ are screened out and the IOU matrix C is reconstructed. Therefore, in this embodiment, the target frame with a low degree of overlap is eliminated by setting the IOU overlap threshold θ, so as to more accurately identify dense small target areas.

[0050] In this embodiment, the IOU connectivity threshold Φ is set, and the connected area of ​​the IOU matrix C is calculated. When the number of targets in the connected area is greater than the IOU connectivity threshold Φ, the connected area is a dense small target area. For example, if the IOU values ​​of the current targets A, B, and C are all greater than or equal to the IOU connectivity threshold Φ, then it is considered that the three regions of target A, target B, and target C are connected, and the number of connections is 3. If the IOU values ​​of the current targets A, B, and C are all less than the IOU connectivity threshold Φ, then the number of connected areas is 1. In this process, when the number of targets in a connected area exceeds the IOU connectivity threshold Φ, the connected area is confirmed as a dense small target area. In this embodiment, it is necessary to save the dense small target area data and repeat the above operation until there are no longer areas that meet the conditions in the dense small target image.

[0051] In this embodiment, a cropping operation is performed on a dense small target image based on the formed set of dense small target regions, and the cropped dense small target image is input again into the backbone network, neck network and head network for feature extraction to obtain dense region features, and then high-precision target features of the dense small target image can be obtained through the dense region features.

[0052] Therefore, in this embodiment, a dense detection module is introduced to determine whether there is a dense small target area in the dense small target image, and the overlap degree between different target frames is evaluated by calculating the IOU matrix. When a certain area is judged to be a dense small target area, the dense small target area is automatically cropped into independent pictures by the dense detection module, thereby decomposing the complex dense small target area into small parts that are easier to handle, so as to more accurately capture the characteristics of each target. Afterwards, the independent pictures are input into the dense small target detection model again for secondary reasoning operations, and the dense area characteristics of the targets in the dense small target area are deeply explored, which significantly improves the detection accuracy of the targets in the dense small target area and enhances the robustness of the dense small target detection.

[0053] In this embodiment, the dense small target image is detected by the dense small target detection model to obtain the detection result. Specifically, the dense small target image is input into the dense small target detection model for detection, and the common features, detail features and dense area features are obtained respectively, and the common features, detail features and dense area features are fused, that is, the global information features and high-precision target features of the dense small target image are fused, and the detection result is output.

[0054] Therefore, the present embodiment provides a method for detecting dense small targets from a drone's perspective based on deep learning. By constructing a dense small target detection model, better results can be achieved by using only lightweight detectors for dense small target areas. Using detectors with simpler structures in the processing of dense small target areas can greatly reduce the resources required for calculation, improve the detection speed of the dense small target detection model, and reduce the video memory occupancy required during the detection process, thereby avoiding unnecessary waste and effectively utilizing limited computing resources. This allows the practical application of efficient dense small target detection models in dense small target images, and has strong practicality and wide applicability.

[0055] The present embodiment also provides a device for detecting dense small targets from the perspective of a drone based on deep learning, comprising an image acquisition module, a model building module and a detection module. The image acquisition module is used to collect a drone image data set, obtain dense small target images from the drone image data set, and pre-process the dense small target images. The model building module is used to construct a dense small target detection model from the perspective of a drone based on deep learning, and the dense small target detection model includes a backbone network, a neck network and a head network; wherein a small target detection head is added to the dense small target detection model, and a dense detection module is introduced into the head network. The detection module detects the dense small target image through the dense small target detection model to obtain a detection result. The device for detecting dense small targets from the perspective of a drone based on deep learning provided in the present embodiment can implement any of the methods for detecting dense small targets from the perspective of a drone based on deep learning, and the specific working process of the device for detecting dense small targets from the perspective of a drone based on deep learning can refer to the corresponding process in the embodiment of the method for detecting dense small targets from the perspective of a drone based on deep learning. The method and device provided in the present embodiment can be implemented in other ways. For example, the device embodiments described above are only illustrative; for example, the division of a module is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual connection or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0056] This embodiment also provides a computer device. A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for detecting small intensive targets from a drone's perspective based on deep learning when executing the computer program.

[0057] This embodiment also provides a computer-readable storage medium. A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a method for detecting dense small targets from a drone's perspective based on deep learning described in this embodiment is executed. Among them, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or device; the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wires, optical cables, RF, etc., or any suitable combination of the above.

[0058] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited thereto, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all belong to the protection scope of the present invention. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate the name, and do not indicate any specific order.

Claims

1. A method for detecting dense small targets from a drone's perspective based on deep learning, comprising the following steps: Collect UAV image datasets, obtain dense small target images from the UAV image datasets, and preprocess the dense small target images; A dense small target detection model based on YOLOv7 is constructed. The dense small target detection model includes a backbone network, a neck network, and a head network. A small target detection head is added to the dense small target detection model, and a dense detection module is introduced into the head network. The dense small target image is detected by the dense small target detection model to obtain the detection result.

2. According to the deep learning-based method for detecting small targets from a densely populated drone perspective as described in claim 1, it is characterized in that: Preprocessing of dense small target images includes correction processing and brightness adjustment.

3. The method for detecting small targets from a densely populated drone perspective based on deep learning according to claim 1, characterized in that: The backbone network, neck network and head network are used to extract features of dense small target images to obtain common features.

4. The method for detecting small targets from a densely populated drone perspective based on deep learning according to claim 3 is characterized in that: The size of dense small target images is increased based on the small target detection head, and the features of the increased-size dense small target images are extracted through the backbone network, neck network and head network to obtain detail features.

5. The method for detecting small targets from a densely populated drone perspective based on deep learning according to claim 4 is characterized in that: The dense detection module is used to determine whether there is a dense small target area in the dense small target image. If there is a dense small target area, the dense small target area is cropped and then input into the backbone network, neck network and head network again for feature extraction to obtain the dense area features.

6. The method for detecting small targets from a densely populated drone perspective based on deep learning according to claim 5 is characterized in that: The dense detection module is used to determine whether a dense small object image has a dense small object area, and the steps include: Input the dense small target image into the dense small target detection model for detection to obtain several target boxes; After scaling several target frames, calculate the IOU values ​​between each target frame, and form an IOU matrix through the IOU values; Set the IOU overlap threshold θ, filter out the IOU values ​​greater than or equal to the IOU overlap threshold θ in the IOU matrix and form the IOU matrix C; Set the IOU connectivity threshold Φ and calculate the connected area of ​​the IOU matrix C. When the number of targets in the connected area is greater than the IOU connectivity threshold Φ, the connected area is a dense small target area.

7. The method for detecting small targets from a densely populated drone perspective based on deep learning according to claim 6, characterized in that: The detection results are output after fusing common features, detail features and dense area features.

8. A device for detecting small targets from a dense drone perspective based on deep learning, characterized in that: include: The image acquisition module collects UAV image data sets, obtains dense small target images from the UAV image data sets, and pre-processes the dense small target images; Model building module, building a dense small target detection model based on YOLOv7, the dense small target detection model includes a backbone network, a neck network and a head network; wherein, a small target detection head is added to the dense small target detection model, and a dense detection module is introduced into the head network; The detection module detects the dense small target image through the dense small target detection model to obtain the detection result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images