Unmanned aerial vehicle aerial photography personnel detection method and device based on lightweight network

Abstract

The invention provides an unmanned aerial vehicle aerial photography personnel detection method based on a lightweight network, and belongs to the field of unmanned aerial vehicle image personnel detection and identification, and the method comprises the steps: obtaining an unmanned aerial vehicle aerial photography image, marking personnel in the image, and obtaining a data set; a BackBone network in a YOLOv8 model is replaced by an improved HGNetV2 network, a coordinate attention mechanism is added in a Neck network, the improved YOLOv8 model is obtained, the improved HGNetV2 network comprises an HGStem module, a GhostHGBlock module and a DWConv module, the HGStem module is used for preprocessing an input image, the GhostHGBlock module is used for feature extraction, and the DWConv module is used for carrying out channel-by-channel convolution on input features and extracting spatial features; replacing the loss function CIoU with a loss function Inner-SIoU, training an improved YOLOv8 model by using a data set, and obtaining a detection model when the total loss function is minimum; deploying the detection model on the unmanned aerial vehicle, and detecting a target person by using the detection model; the invention further provides a detection device. And the detection precision of unmanned aerial vehicle aerial photographing personnel in complex background and small target detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of drone image personnel detection and recognition, and in particular to a drone aerial photography personnel detection method and device based on a lightweight network. Background Art

[0002] Drone imagery person detection uses computer vision technology to detect and identify people in images captured by drone-mounted cameras. This technology has been widely used in various fields, particularly in security, public safety, disaster relief, and environmental monitoring.

[0003] With the rapid development of drone technology, drones have become an important tool for obtaining high-altitude image data. Drones are equipped with high-resolution cameras and can capture images of wide areas in a low-cost and efficient manner. Compared with traditional monitoring equipment, drones can obtain data in more flexible scenarios, especially in difficult-to-access areas such as disaster areas, battlefields, and mountainous areas, and can quickly obtain key information. Person detection is an important task in the field of computer vision, which automatically identifies people from images or videos. For drone images, person detection becomes more challenging because the images captured by drones may have the following characteristics:

[0004] (1) Perspective diversity: UAV images are often taken from different perspectives, and the scale and position of people in the image also vary. This makes it difficult to apply traditional detection methods based on fixed perspectives.

[0005] (2) Lighting changes and weather interference: When a drone is flying, the image may be affected by environmental factors such as lighting, clouds, shadows, wind and rain, which will increase the difficulty of detection.

[0006] (3) Uneven distribution of target scale: Due to the change in flight altitude, the scale of people in the image may be very small, and even difficult to identify from long-distance images. UAV images often cover a wide area, and human targets may only occupy a small area.

[0007] (4) Complex background: Images captured by drones may contain complex backgrounds, especially in cities, buildings, traffic or natural scenes. People detection algorithms need to be able to distinguish between people and other interfering objects.

[0008] With the rapid development of drone technology, how to detect people efficiently and in real time has become a key issue. Especially when running on resource-constrained edge computing devices, how to improve the operating efficiency and accuracy of the model has become one of the technical challenges. Although existing target detection algorithms have high accuracy, they often have problems such as large computational complexity, large models, and slow inference speed, making it difficult to meet the real-time processing needs of drones in complex environments.

[0009] In the prior art, for example, the Chinese invention patent application CN117593674A, “A Lightweight UAV Aerial Photography Target Real-time Detection Method,” discloses a lightweight UAV aerial photography target real-time detection method, including: obtaining an initial data set captured by a UAV camera; performing data enhancement processing on the initial data set; improving the YOLOv8n model; training the improved YOLOv8n model, and deploying the trained model to a mobile device for real-time target detection. The improvement of the YOLOv8n model specifically includes: replacing the C2f module in the backbone network of the YOLOv8n model with a lightweight block DWC2f module, modifying the PANET neck structure of the YOLOv8n to a TFPN neck structure, replacing the CBS module in the head structure of the YOLOv8n model with the lightweight block DWUnit, and replacing the SIoU loss function of the YOLOv8n with MPDIoU. This detection method is used to detect multiple categories of pedestrians or vehicles. However, since the target person features are smaller than the category features of pedestrians or vehicles, the target person may only occupy a very small area. Therefore, this detection method has certain limitations in detecting target persons. Summary of the Invention

[0010] The technical problem to be solved by the present invention is how to improve the detection accuracy of drone aerial photography personnel detection in complex backgrounds and small target detection.

[0011] The present invention solves the above technical problems through the following technical solutions: a method for detecting people in drone aerial photography based on a lightweight network, the method comprising:

[0012] Obtain drone aerial images and label the people in the images to obtain a dataset;

[0013] The BackBone network in the YOLOv8 model is replaced with the improved HGNetV2 network, and the coordinate attention mechanism is added to the Neck network to obtain the improved YOLOv8 model. The improved HGNetV2 network includes the HGStem module, the GhostHGBlock module, and the DWConv module. The HGStem module is used to preprocess the input image, the GhostHGBlock module is used for feature extraction, and the DWConv module is used to perform channel-by-channel convolution on the input features to extract spatial features.

[0014] The loss function CIoU is replaced by the loss function Inner-SIoU, and the improved YOLOv8 model is trained with the dataset. When the total loss function is minimized, the detection model is obtained.

[0015] The detection model is deployed on the drone and the target person is detected using the detection model.

[0016] Beneficial effects: The present invention proposes an optimized YOLOv8 algorithm by combining the GhostHGNet lightweight backbone, coordinate attention mechanism and Inner-SIoU improvement. Among them, GhostHGNet optimizes the backbone network through efficient lightweight design, reduces the number of parameters and calculations, significantly reduces the calculation amount and memory usage of the model, making the model more advantageous on resource-constrained devices while maintaining a high detection accuracy. By using coordinate attention to improve the neck network, the coordinate attention mechanism can further enhance the feature extraction capability, especially in the detection of complex backgrounds and small targets. In addition, by introducing Inner-SIoU to improve the loss function of YOLOv8, the model has significantly improved the accuracy and convergence speed of bounding box regression, further improving the detection accuracy and generalization ability, which can significantly improve the detection accuracy and generalization.

[0017] Preferably, the improved HGNetV2 network includes an HGStem module, a first GhostHGBlock module, a first DWConv module, a second GhostHGBlock module, a second DWConv module, a third GhostHGBlock module, a fourth GhostHGBlock module, a fifth GhostHGBlock module, a third DWConv module, a sixth GhostHGBlock module, and an SPPF module. The input image passes through the HGStem module, the first GhostHGBlock module, the first DWConv module, and the second GhostHGBlock module in sequence to obtain a feature map T1. The feature map T1 passes through the second DWConv module, the third GhostHGBlock module, the fourth GhostHGBlock module, and the fifth GhostHGBlock module in sequence to obtain a feature map T2. The feature map T2 passes through the third DWConv module, the sixth GhostHGBlock module, and the SPPF module in sequence to obtain a feature map T3. The feature maps T1, T2, and T3 enter the Neck network.

[0018] Beneficial Effects: This paper replaces the backbone network of YOLOv8 with HGNetV2 and uses ghost convolution for optimization. The improved HGNetV2 can significantly reduce the model's computational complexity and parameter count while maintaining high accuracy. HGStem is the network's preprocessing layer, which performs preliminary feature extraction on the input image through convolution operations and then downsamples using maximum pooling operations to capture feature variations at different scales. This process not only enhances feature diversity but also further improves the model's feature expression capabilities through effective fusion with post-convolution feature maps.

[0019] Preferably, the HGStem module includes a first Conv module, a second Conv module, a Maxpool2d module, a Concat module, and a third Conv module. The data of the input image after feature extraction by the first Conv module enters the second Conv module and the Maxpool2d module respectively. The data after downsampling by the Maxpool2d module and the output data of the second Conv module are fused in the Concat module, and the fused data are input into the third Conv module for feature extraction.

[0020] Preferably, the GhostHGBlock module includes multiple GhostConv modules, Concat modules, and Add modules. The input data is subjected to convolution operations and feature generation operations in multiple GhostConv modules to obtain a first feature map. The output data of each GhostConv module is input into the Concat module with the first feature map for fusion. The fused data is sequentially subjected to convolution operations and feature generation operations in two GhostConv modules to obtain a second feature map. The output data of each GhostConv module and the second feature map are input into the Add module to obtain the output result.

[0021] Beneficial Effects: This paper utilizes GhostConv as the convolutional layer in HGBlock. By generating linear combinations of redundant features, the GhostConv module reduces the number of parameters and computational complexity of HGNetv2 while maintaining the richness of feature extraction. Combined with the efficient feature extraction capabilities of HGNet, this combination significantly improves the detection accuracy of small targets while ensuring a lightweight model, adapting to the characteristics of drone aerial images, where there are many small targets and large scale variations.

[0022] Preferably, the DWConv module is a convolutional layer, and its channel-by-channel convolution parameter quantity Conv DWConv =c×k×k, parameter Conv DWConv The number of parameters is reduced by m-1 times compared to the standard convolution, where c is the number of channels of the input feature map, k is the size of the convolution kernel, and m is the number of channels of the output feature map.

[0023] Beneficial Effects: Compared with standard convolution, the DWconv module of the present invention has a parameter count reduced by (m-1) times. Using the DWConv module instead of standard convolution in the entire model can significantly reduce the overall parameter count and computational complexity of the model, effectively reducing the computational complexity of the model. The DWconv module extracts spatial features through channel-by-channel convolution, performing independent convolution operations between each channel. Each convolution kernel only needs to learn the features of a single channel, thereby improving the efficiency of convolution.

[0024] Preferably, the improved HGNetV2 network outputs three feature maps and enters the Neck network. The feature map T3 is input into the first UpSampling module for upsampling to obtain the feature map C1. The feature map C1 and the feature map T2 are input into the first Concat module for fusion. The fused features are sequentially passed through the first C2f module, the first CA module, and the second UpSampling module to obtain the feature map C2. The feature map C2 and the feature map T1 are input into the second Concat module for fusion. The fused features are sequentially passed through the second C2f module and the second CA module to obtain the feature map T1. The feature map T1 is fused with the feature map C2 after the convolution operation of the first Conv module. The fused features are sequentially passed through the third C2f module and the third CA module to output the feature map T2. The feature map T2 is fused with the feature map C1 after the convolution operation of the second Conv module. The fused features are sequentially passed through the fourth C2f module and the fourth CA module to output the feature map T3. The feature map T1, the feature map T2, and the feature map T3 are respectively sent to the Detect module.

[0025] Preferably, the CA module in the Neck network includes a Resdidual module, a first AvgPool module, and a second AvgPool module. The input data processed by the Resdidual module enters the first AvgPool module and the second AvgPool module respectively, and the input data is globally average pooled in the horizontal and vertical directions to obtain feature maps in two directions. The features after the fusion of the feature maps in the two directions are subjected to 1×1 convolution, batch normalization, and nonlinear activation for feature conversion to obtain horizontal attention and vertical attention. The horizontal attention and vertical attention are respectively subjected to 1×1 convolution and normalization processing, and then weightedly merged with the data processed by the Resdidual module and output.

[0026] Beneficial Effects: This invention introduces coordinate attention, a mechanism that embeds position information into channel attention. It aggregates spatial features through two 1D feature encoding processes, preserving precise position information. This improvement enhances the directional perception and position sensitivity of feature maps, particularly in complex backgrounds and small target detection. It effectively improves detection accuracy, allowing the model to focus more on the target area and reduce background noise interference, thereby reducing the false detection rate.

[0027] Preferably, the weighted combination outputs y c (i,j) is:

[0028]

[0029] Among them, x c (i, j) is the data processed by the Resdidual module, is the horizontal attention g h The data after 1×1 convolution and normalization processing, is the horizontal attention g w Data after 1×1 convolution and normalization respectively;

[0030] g h =σ(F h (f h ))

[0031] g w =σ(F w (f w ))

[0032] f=δ(F1([z h ,z w ]))

[0033] Among them, δ is a nonlinear activation function, f∈R C / r×(H+W) It is represented as an intermediate feature map containing horizontal and vertical spatial information, r is the reduction factor, and F1 is the convolution change function;

[0034]

[0035]

[0036] in, Indicates that for the c-th channel of the feature map, at position h in the height direction, the average value of all pixel values at that position in the width direction is calculated. It means that for the c-th channel of the feature map, at position w in the width direction, the average value of all pixel values in the height direction of this position is calculated.

[0037] Preferably, the total loss function Loss improve for:

[0038] Loss improve =Loss Inner-SIoU +Loss CLs +Loss DFL

[0039] in, IoU inner is the loss function Inner-SIoU intersection-over-union ratio, Δ is the distance loss, Ω is the shape loss, Loss CLs 、Loss DFL They are classification loss and bounding box regression loss respectively;

[0040]

[0041] union=(w gt ×h gt )×r 2 +(w×h)×r 2 -inner

[0042] in, They are the left boundary, right boundary, top boundary, and bottom boundary of the real box respectively, b l 、b r 、b t 、b b They are the left boundary, right boundary, top boundary, and bottom boundary of the prediction box, respectively, w gt 、h gt are the height and width of the real box, w and h are the height and width of the anchor box, and r is the scale factor.

[0043] Beneficial effect: The present invention replaces the loss function CIoU with the loss function Inner-SIoU. Inner-SIoU optimizes the bounding box regression process by introducing auxiliary bounding boxes of different scales, and performs particularly well in small target detection. It not only accelerates the convergence speed of high IoU samples, but also enhances the regression effect of low IoU samples, significantly improving the detection accuracy and generalization ability of the model in complex scenarios.

[0044] The present invention also provides a lightweight network-based drone aerial photography personnel detection device, which includes:

[0045] The data processing module is used to obtain drone aerial images and annotate the people in the images to obtain a data set;

[0046] The model construction module is used to replace the BackBone network in the YOLOv8 model with the improved HGNetV2 network and add a coordinate attention mechanism to the Neck network to obtain an improved YOLOv8 model. The improved HGNetV2 network includes the HGStem module, the GhostHGBlock module, and the DWConv module. The HGStem module is used to preprocess the input image, the GhostHGBlock module is used for feature extraction, and the DWConv module is used to perform channel-by-channel convolution on the input features to extract spatial features.

[0047] The model training module is used to replace the CIoU loss function with the Inner-SIoU loss function, and train the improved YOLOv8 model using the dataset. When the total loss function is minimized, the detection model is obtained;

[0048] The detection module is used to deploy the detection model on the drone and use the detection model to detect the target person.

[0049] The present invention also provides the following advantages: the prediction model of the present invention is deployed on a drone, and can effectively improve detection accuracy and enhance small target detection capabilities in drone aerial photography personnel detection tasks. The model has good generalization performance while maintaining a lightweight design, and is suitable for application on resource-constrained drone platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a method for detecting people in drone aerial photography based on a lightweight network provided by an embodiment of the present invention;

[0051] Figure 2 This is an architecture diagram of the improved YOLOv8 model in the lightweight network-based drone aerial photography personnel detection method provided by an embodiment of the present invention;

[0052] Figure 3 This is an architectural diagram of the HGStem module in the improved YOLOv8 model in the lightweight network-based drone aerial photography personnel detection method provided by an embodiment of the present invention;

[0053] Figure 4 (a) is the architecture diagram of the HGBlock module in the existing YOLOv8 model;

[0054] Figure 4 (b) is an architectural diagram of the GhostHGBlock module in the improved YOLOv8 model in the lightweight network-based drone aerial photography personnel detection method provided by an embodiment of the present invention;

[0055] Figure 5 A diagram illustrating the working principle of the DWConv module in the improved YOLOv8 model in the lightweight network-based drone aerial photography personnel detection method provided by an embodiment of the present invention;

[0056] Figure 6 This is an architectural diagram of the CA module in the improved YOLOv8 model in the lightweight network-based drone aerial photography personnel detection method provided by an embodiment of the present invention;

[0057] Figure 7 A schematic diagram of a lightweight network-based drone aerial photography personnel detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the following describes the technical solutions of the present invention clearly and completely with reference to specific embodiments and the accompanying drawings. It is obvious that the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0059] like Figure 1 As shown, this embodiment provides a method for detecting people in drone aerial photography based on a lightweight network, including the following steps:

[0060] Step 1: Obtain drone aerial images and label the people in the images to obtain a dataset.

[0061] Use drones to capture image data in a variety of environments, including city streets, rural areas, forests, and around buildings. The dataset should also cover a variety of weather, lighting, and time of day conditions, such as daytime, nighttime, overcast, sunny, and rainy, to simulate the changing environments likely encountered in real-world applications. Furthermore, ensure that the image data includes different shooting angles and flight altitudes to fully reflect the changes in the drone's perspective under different flight conditions. By capturing data from different scenes, environments, angles, and altitudes, we can ensure the diversity and representativeness of the dataset and establish a high-quality drone aerial photography personnel detection dataset.

[0062] We manually annotated the captured images using tools like LabelImg, accurately locating the people in each image and marking the precise location of each target person with a box. We divided the dataset into training, validation, and test sets. A typical ratio is 70% for model training, 20% for model validation, and 10% for model testing to ensure dataset balance and model generalization.

[0063] Step 2. Replace the BackBone network in the YOLOv8 model with the improved HGNetV2 network, add the coordinate attention mechanism to the Neck network, and obtain the improved YOLOv8 model. The improved HGNetV2 network includes the HGStem module, the GhostHGBlock module, and the DWConv module. The HGStem module is used to preprocess the input image, the GhostHGBlock module is used for feature extraction, and the DWConv module is used to perform channel-by-channel convolution on the input features to extract spatial features.

[0064] See also Figure 2The improved HGNetV2 network includes an HGStem module, a first GhostHGBlock module, a first DWConv module, a second GhostHGBlock module, a second DWConv module, a third GhostHGBlock module, a fourth GhostHGBlock module, a fifth GhostHGBlock module, a third DWConv module, a sixth GhostHGBlock module, and an SPPF module. The input image passes through the HGStem module, the first GhostHGBlock module, the first DWConv module, and the second GhostHGBlock module in sequence to obtain a feature map T1. The feature map T1 passes through the second DWConv module, the third GhostHGBlock module, the fourth GhostHGBlock module, and the fifth GhostHGBlock module in sequence to obtain a feature map T2. The feature map T2 passes through the third DWConv module, the sixth GhostHGBlock module, and the SPPF module in sequence to obtain a feature map T3. The feature maps T1, T2, and T3 enter the Neck network.

[0065] See also Figure 3 The HGStem module includes a first Conv module, a second Conv module, a Maxpool2d module, a Concat module, and a third Conv module. The data of the input image after feature extraction by the first Conv module enters the second Conv module and the Maxpool2d module respectively. The data after downsampling by the Maxpool2d module and the output data of the second Conv module are fused in the Concat module. The fused data is input into the third Conv module for feature extraction.

[0066] In the field of drone target detection algorithms, lightweight and high-precision models are crucial, because drones usually need to complete tasks with limited resources. Although the YOLOv8 backbone network CSPDarkNet-53 has high detection accuracy, its model is large and complex in structure, making it difficult to directly apply to drone platforms.

[0067] To meet the needs of drone aerial photography personnel detection, this paper replaces the YOLOv8 backbone network with HGNetV2 and optimizes it using ghost convolution. The improved HGNetV2 significantly reduces the model's computational complexity and parameter count while maintaining high accuracy. HGStem is the network's preprocessing layer, which performs preliminary feature extraction on the input image through convolution operations and then downsamples it using max pooling to capture feature variations at different scales. This process not only enhances feature diversity but also further improves the model's feature expression capabilities through effective fusion with the post-convolution feature map.

[0068] See also Figure 4 (b) The GhostHGBlock module includes multiple GhostConv modules, Concat modules, and Add modules. The input data undergoes convolution operations and feature generation operations on multiple GhostConv modules to obtain a first feature map. The output data of each GhostConv module is input into the Concat module with the first feature map for fusion. The fused data undergoes convolution operations and feature generation operations on two GhostConv modules in turn to obtain a second feature map. The output data of each GhostConv module and the second feature map are input into the Add module to obtain the output result.

[0069] Figure 4 (a) is a schematic diagram of the HGBlock module, which is used for feature extraction. It captures the diverse features of the input data through multiple convolutional layers of different sizes, enabling the network to better represent features at different levels. The Ghost module is a lightweight convolutional neural network designed to address the trade-off between feature extraction and computational efficiency in traditional convolutional neural networks. With the development of deep learning, especially in mobile devices and embedded systems, computational resource constraints have made lightweight and efficient network architectures a research focus. By generating lightweight feature maps, the Ghost module can significantly reduce the computational complexity of the model while maintaining efficient feature extraction capabilities.

[0070] Traditional convolutional neural networks improve feature representation by increasing the number of convolution kernels in convolutional layers. However, this often leads to a dramatic increase in computational complexity, especially in deep networks, where computational complexity often grows exponentially. To alleviate this problem, many lightweight networks, such as MobileNet and ShuffleNet, have proposed reducing the number of convolution kernels to improve computational efficiency. However, this approach often results in a decrease in the network's representational capabilities.

[0071] The idea of the Ghost module is to enhance the expressive power of the model by generating ghost features, rather than by adding more convolution kernels. Specifically, the Ghost module combines traditional convolution operations with lightweight feature generation operations to generate more feature maps using simple operations. The present invention uses GhostConv as the convolution layer in HGBlock. The GhostConv module reduces the number of parameters and computational complexity of HGNetv2 by generating linear combinations of redundant features, while maintaining the richness of feature extraction. Combined with the efficient feature extraction capability of HGNet, this combination significantly improves the detection accuracy of small targets while ensuring the lightweight of the model, adapting to the characteristics of many small targets and large scale changes in drone aerial images.

[0072] See also Figure 5 , the DWConv module is a convolutional layer. Assuming that the size of the input feature map is n×n×c, the convolution kernel size is k×k×m, the output feature map size is n×n×m, and the number of parameters of the standard convolutional layer Conv norm and the number of parameters of DWConvConv DWConv They are:

[0073] Conv norm =c×m×k×k

[0074] Conv DWConv =c×k×k

[0075] Among them, n and k represent the size of the feature map and convolution kernel respectively, c represents the number of channels of the input feature map, m represents the number of channels of the output feature map, c×k×k represents the number of parameters of channel-by-channel convolution, and the parameter ratio of the DWConv module to the ordinary convolution is:

[0076]

[0077] As can be seen, the DWconv module has a (m-1)-fold reduction in parameters compared to standard convolution. Using the DWConv module instead of standard convolution in the entire model significantly reduces the overall model parameters and computational complexity, effectively lowering the model's computational complexity. The DWconv module extracts spatial features through channel-by-channel convolution, performing independent convolution operations on each channel. Each convolution kernel only needs to learn the features of a single channel, thereby improving convolution efficiency.

[0078] Continue to see Figure 2 The improved HGNetV2 network outputs feature map T1, feature map T2, and feature map T3 and enters the Neck network. Feature map T3 is input into the first UpSampling module for upsampling to obtain feature map C1. Feature map C1 and feature map T2 are jointly input into the first Concat module for fusion. The fused features pass through the first C2f module, the first CA module, and the second UpSampling module to obtain feature map C2. Feature map C2 and feature map T1 are jointly input into the second Concat module for fusion. The fused features pass through the second C2f module and the second CA module to obtain feature map T1. Feature map T1 is fused with feature map C2 after the convolution operation of the first Conv module. The fused features pass through the third C2f module and the third CA module to output feature map T2. Feature map T2 is fused with feature map C1 after the convolution operation of the second Conv module. The fused features pass through the fourth C2f module and the fourth CA module to output feature map T3. Feature map T1, feature map T2, and feature map T3 are respectively sent to the three Detect modules.

[0079] See also Figure 6 The CA module in the Neck network includes a Resdidual module, a first AvgPool module, and a second AvgPool module. The input data processed by the Resdidual module enters the first AvgPool module and the second AvgPool module respectively. The input data is globally average pooled in the horizontal and vertical directions respectively to obtain feature maps in two directions. The features after the fusion of the feature maps in the two directions are transformed by 1×1 convolution, batch normalization, and nonlinear activation to obtain horizontal attention and vertical attention. The horizontal attention and vertical attention are respectively processed by 1×1 convolution and normalization, and then weighted and merged with the data processed by the Resdidual module for output.

[0080] CA attention performs global average pooling on the input drone image in the horizontal and vertical directions respectively, and obtains the feature map in the horizontal direction. Feature map in the vertical direction The calculation formula is:

[0081]

[0082] in, Indicates that for the c-th channel of the feature map, at position h in the height direction, the average value of all pixel values at that position in the width direction is calculated. It means that for the c-th channel of the feature map, at position w in the width direction, the average value of all pixel values in the height direction of this position is calculated.

[0083] Merge the feature maps in the horizontal and vertical directions, and perform feature transformation through 1×1 convolution, batch normalization, and nonlinear activation. The formula is as follows:

[0084] f=δ(F1([z h ,z w ]))

[0085] g h =σ(F h (f h ))

[0086] g w =σ(F w (f w ))

[0087] Among them, δ is a nonlinear activation function, f∈R C / r×(H+W) It is represented as an intermediate feature map containing horizontal and vertical spatial information, r is the reduction factor, and F1 is the convolution change function.

[0088] After the two attentions are normalized by sigmoid, they are weighted and combined and output, and the output result is y c (i,j) is:

[0089]

[0090] Among them, x c (i, j) is the data processed by the Resdidual module, is the horizontal attention g h The data after 1×1 convolution and normalization processing, is the horizontal attention g w The data are processed by 1×1 convolution and normalization respectively.

[0091] In drone aerial imagery, human targets are often small and situated against complex backgrounds. The coordinate attention mechanism enhances feature extraction and localization accuracy for small targets through precise position perception. Furthermore, the coordinate attention mechanism is designed to be lightweight, adding only a small amount of computational overhead. This is particularly important for drone aerial photography tasks, as drones typically require limited computing resources to complete their tasks. In aerial images with complex backgrounds, the coordinate attention mechanism helps the model better distinguish between targets and background. By introducing coordinate attention, the coordinate attention mechanism embeds position information into channel attention, aggregating spatial features through two 1D feature encoding processes to preserve precise position information. This improvement enhances the feature map's directional perception and position sensitivity, particularly when detecting small targets in complex backgrounds. It effectively improves detection accuracy, allowing the model to focus more closely on the target area and reduce interference from background noise, thereby reducing false detection rates.

[0092] Step 3: Replace the loss function CIoU with the loss function Inner-SIoU, and use the dataset to train the improved YOLOv8 model. When the total loss function is minimized, the detection model is obtained.

[0093] Calculating the loss function for bounding box regression is crucial for the performance of object detection. The model calculates the loss by comparing the difference between the predicted bounding box and the true box. In the object detection task, IoU is used to calculate the degree of overlap between the predicted box and the true box. The formula for IoU is defined as:

[0094]

[0095] Among them, B pr 、B ht Represent the center points of the predicted box and the true box respectively.

[0096] However, the IoU loss function only considers the overlapping area of the bounding boxes, without considering their relative positions in space. To address the above issues, YOLOv8 uses the CIoU loss function to calculate the regression loss of the bounding box. CIoU has made improvements based on the IoU loss function. It not only includes IoU's measurement of the degree of overlap between the predicted box and the true box, but also introduces a minimum center point penalty term to measure the distance between the two, thereby enhancing the sensitivity to differences in the position of the bounding boxes. In addition, CIoU also adds an aspect ratio penalty term to further improve the assessment of the consistency of the bounding box shape. These improvements enable the CIoU loss function to comprehensively optimize the bounding box, thereby achieving higher accuracy and robustness in object detection tasks.

[0097] SIoU loss adds consideration of the angle difference between the predicted box and the ground-truth box on the basis of CIoU. By incorporating the angle loss into the bounding box regression loss function, the model's prediction accuracy of the target direction is improved. SIoU includes IoU loss, distance loss Δ, angle loss Λ, and shape loss Ω. The formula is as follows:

[0098]

[0099] Among them, IoU is the intersection over union ratio between the predicted box and the real box.

[0100] The angle loss Λ is:

[0101]

[0102] γ=2-Λ[0,1]

[0103] in, Indicates the center point of the real box, (x c ,y c ) represents the center point of the anchor box.

[0104] The distance loss Δ is:

[0105]

[0106] The shape loss Ω is:

[0107]

[0108] The present invention replaces the loss function CIoU with the loss function Inner-SIoU. When calculating the loss function Inner-SIoU, the real box coordinates are defined as follows:

[0109]

[0110] in, They are the left boundary, right boundary, top boundary and bottom boundary of the real box respectively.

[0111] The prediction box coordinates are defined as follows:

[0112]

[0113] b l 、b r 、b t 、b b They are the left, right, top and bottom boundaries of the prediction box respectively.

[0114] The intersection and union areas of the target box and the anchor box are defined as follows:

[0115]

[0116] union=(w gt ×h gt )×r 2 +(w×h)×r 2 -inner

[0117] Among them, w gt 、h gt are the height and width of the real box, w and h are the height and width of the anchor box, and r represents a variable scale factor with a value between 0.5 and 1.5.

[0118] Loss function Inner-SIoU Intersection over Union (IoU) inner for:

[0119]

[0120] Apply the Inner-SIoU loss to the SIoU bounding box regression loss function to obtain the loss function Loss Inner-SIoU :

[0121]

[0122] Total loss function Loss improve for:

[0123] Loss improve =Loss Inner-SIoU +Loss CLs +Loss DFL

[0124] Among them, Loss CLs 、Loss DFL They are classification loss and bounding box regression loss respectively.

[0125] The present invention replaces the loss function CIoU with the loss function Inner-SIoU. Inner-SIoU optimizes the bounding box regression process by introducing auxiliary bounding boxes of different scales, and performs particularly well in small target detection. It not only accelerates the convergence speed of high IoU samples, but also enhances the regression effect of low IoU samples, significantly improving the detection accuracy and generalization ability of the model in complex scenarios.

[0126] Step 4: Deploy the detection model to the drone and use it to detect the target person.

[0127] The trained detection model weights are deployed on the drone's computing platform to ensure that the model can quickly and accurately detect target personnel during real-time flight, realizing the automation and intelligence of the drone's personnel detection system.

[0128] The present invention proposes an optimized YOLOv8 algorithm by combining the GhostHGNet lightweight backbone, coordinate attention mechanism and Inner-SIoU improvement. Among them, GhostHGNet optimizes the backbone network through efficient lightweight design, reduces the number of parameters and calculations, significantly reduces the calculation amount and memory usage of the model, making the model more advantageous on resource-constrained devices while maintaining high detection accuracy. By using coordinate attention to improve the neck network, the coordinate attention mechanism can further enhance the feature extraction capability, especially in the detection of complex backgrounds and small targets. In addition, by introducing Inner-SIoU to improve the loss function of YOLOv8, the model has been significantly improved in the accuracy and convergence speed of bounding box regression, further improving the detection accuracy and generalization ability, which can significantly improve the detection accuracy and generalization ability.

[0129] The prediction model of the present invention is deployed on a drone. In the drone aerial photography personnel detection task, it can effectively improve the detection accuracy and enhance the small target detection capability. The model has good generalization performance while maintaining a lightweight design, which is suitable for application on resource-constrained drone platforms.

[0130] See also Figure 7 The present invention also provides a lightweight network-based drone aerial photography personnel detection device for executing the method of the present invention. For details not disclosed in the device of the present invention, please refer to the content of the method of the present invention, which will not be repeated here. The device includes:

[0131] The data processing module is used to obtain drone aerial images and annotate the people in the images to obtain a data set;

[0132] The model construction module is used to replace the BackBone network in the YOLOv8 model with the improved HGNetV2 network and add a coordinate attention mechanism to the Neck network to obtain an improved YOLOv8 model. The improved HGNetV2 network includes the HGStem module, the GhostHGBlock module, and the DWConv module. The HGStem module is used to preprocess the input image, the GhostHGBlock module is used for feature extraction, and the DWConv module is used to perform channel-by-channel convolution on the input features to extract spatial features.

[0133] The model training module is used to replace the CIoU loss function with the Inner-SIoU loss function, and train the improved YOLOv8 model using the dataset. When the total loss function is minimized, the detection model is obtained;

[0134] The detection module is used to deploy the detection model on the drone and use the detection model to detect the target person.

[0135] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A lightweight network-based drone aerial photography personnel detection method, characterized by: Methods include: Obtain drone aerial images and label the people in the images to obtain a dataset; The BackBone network in the YOLOv8 model is replaced with the improved HGNetV2 network, and the coordinate attention mechanism is added to the Neck network to obtain the improved YOLOv8 model. The improved HGNetV2 network includes the HGStem module, the GhostHGBlock module, and the DWConv module. The HGStem module is used to preprocess the input image, the GhostHGBlock module is used for feature extraction, and the DWConv module is used to perform channel-by-channel convolution on the input features to extract spatial features. The loss function CIoU is replaced by the loss function Inner-SIoU, and the improved YOLOv8 model is trained with the dataset. When the total loss function is minimized, the detection model is obtained. The detection model is deployed on the drone and the target person is detected using the detection model.

2. The lightweight network-based drone aerial photography personnel detection method according to claim 1 is characterized by: The improved HGNetV2 network includes an HGStem module, a first GhostHGBlock module, a first DWConv module, a second GhostHGBlock module, a second DWConv module, a third GhostHGBlock module, a fourth GhostHGBlock module, a fifth GhostHGBlock module, a third DWConv module, a sixth GhostHGBlock module, and an SPPF module. The input image passes through the HGStem module, the first GhostHGBlock module, the first DWConv module, and the second GhostHGBlock module in sequence to obtain the feature map T1. The feature map T1 passes through the second DWConv module, the third GhostHGBlock module, the fourth GhostHGBlock module, and the fifth GhostHGBlock module in sequence to obtain the feature map T2. The feature map T2 passes through the third DWConv module, the sixth GhostHGBlock module, and the SPPF module in sequence to obtain the feature map T3. The feature maps T1, T2, and T3 enter the Neck network.

3. The lightweight network-based drone aerial photography personnel detection method according to claim 2, characterized in that: The HGStem module includes the first Conv module, the second Conv module, the Maxpool2d module, the Concat module, and the third Conv module. The data of the input image after feature extraction by the first Conv module enters the second Conv module and the Maxpool2d module respectively. The data after downsampling by the Maxpool2d module and the output data of the second Conv module are fused in the Concat module. The fused data is input into the third Conv module for feature extraction.

4. The lightweight network-based drone aerial photography personnel detection method according to claim 2, characterized in that: The GhostHGBlock module includes multiple GhostConv modules, Concat modules, and Add modules. The input data undergoes convolution operations and feature generation operations on multiple GhostConv modules to obtain a first feature map. The output data of each GhostConv module is input into the Concat module with the first feature map for fusion. The fused data undergoes convolution operations and feature generation operations on two GhostConv modules in turn to obtain a second feature map. The output data of each GhostConv module and the second feature map are input into the Add module to obtain the output result.

5. The method for detecting people in drone aerial photography based on a lightweight network according to claim 1, characterized in that: The DWConv module is a convolution layer, and its channel-by-channel convolution parameter Conv DWConv =c×k×k, parameter Conv DWConv The number of parameters is reduced by m-1 times compared to the standard convolution, where c is the number of channels of the input feature map, k is the size of the convolution kernel, and m is the number of channels of the output feature map.

6. The method for detecting people in drone aerial photography based on a lightweight network according to claim 1, characterized in that: The improved HGNetV2 network outputs three feature maps and enters the Neck network. The feature map T3 is input into the first UpSampling module for upsampling to obtain the feature map C1. The feature map C1 and the feature map T2 are input into the first Concat module for fusion. The fused features are sequentially passed through the first C2f module, the first CA module, and the second UpSampling module to obtain the feature map C2. The feature map C2 and the feature map T1 are input into the second Concat module for fusion. The fused features are sequentially passed through the second C2f module and the second CA module to obtain the feature map T1. The feature map T1 is fused with the feature map C2 after the convolution operation of the first Conv module. The fused features are sequentially passed through the third C2f module and the third CA module to output the feature map T2. The feature map T2 is fused with the feature map C1 after the convolution operation of the second Conv module. The fused features are sequentially passed through the fourth C2f module and the fourth CA module to output the feature map T3. The feature map T1, feature map T2, and feature map T3 are respectively sent to the Detect module.

7. The method for detecting people in drone aerial photography based on a lightweight network according to claim 1, characterized in that: The CA module in the Neck network includes a Resdidual module, a first AvgPool module, and a second AvgPool module. The input data processed by the Resdidual module enters the first AvgPool module and the second AvgPool module respectively, and the input data is globally average pooled in the horizontal and vertical directions to obtain feature maps in two directions. The features after the fusion of the feature maps in the two directions are transformed by 1×1 convolution, batch normalization, and nonlinear activation to obtain horizontal attention and vertical attention. The horizontal attention and vertical attention are respectively processed by 1×1 convolution and normalization, and then weightedly merged with the data processed by the Resdidual module and output.

8. The lightweight network-based drone aerial photography personnel detection method according to claim 7, characterized in that: Output y after weighted merging c (i,j) is: Among them, x c (i, j) is the data processed by the Resdidual module, is the horizontal attention g h The data after 1×1 convolution and normalization processing, is the horizontal attention g w Data after 1×1 convolution and normalization processing; g h =σ(F h (f h )) g w =σ(F w (f w )) f=δ(F1([z h ,z w ])) Among them, δ is a nonlinear activation function, f∈R C / r×(H+W) It is represented as an intermediate feature map containing horizontal and vertical spatial information, r is the reduction factor, and F1 is the convolution change function; in, Indicates that for the c-th channel of the feature map, at position h in the height direction, the average value of all pixel values at that position in the width direction is calculated. It means that for the c-th channel of the feature map, at position w in the width direction, the average value of all pixel values in the height direction of this position is calculated.

9. The method for detecting people in drone aerial photography based on a lightweight network according to claim 1, characterized in that: Total loss function Loss improve for: Loss improve =Loss Inner-SIoU +Loss CLs +Loss DFL Among them, IoU inner is the loss function Inner-SIoU intersection-over-union ratio, Δ is the distance loss, Ω is the shape loss, Loss CLs 、Loss DFL They are classification loss and bounding box regression loss respectively; union=(w gt ×h gt )×r 2 +(w×h)×r 2 -inner in, They are the left boundary, right boundary, top boundary, and bottom boundary of the real box respectively, b l 、b r 、b t 、b b They are the left boundary, right boundary, top boundary, and bottom boundary of the prediction box, respectively, w gt 、h gt are the height and width of the real box, w and h are the height and width of the anchor box, and r is the scale factor.

10. A lightweight network-based drone aerial photography personnel detection device, characterized by: The device includes: The data processing module is used to obtain drone aerial images and annotate the people in the images to obtain a data set; The model construction module is used to replace the BackBone network in the YOLOv8 model with the improved HGNetV2 network and add a coordinate attention mechanism to the Neck network to obtain an improved YOLOv8 model. The improved HGNetV2 network includes the HGStem module, the GhostHGBlock module, and the DWConv module. The HGStem module is used to preprocess the input image, the GhostHGBlock module is used for feature extraction, and the DWConv module is used to perform channel-by-channel convolution on the input features to extract spatial features. The model training module is used to replace the CIoU loss function with the Inner-SIoU loss function, and train the improved YOLOv8 model using the dataset. When the total loss function is minimized, the detection model is obtained; The detection module is used to deploy the detection model on the drone and use the detection model to detect the target person.

Images