Improved lightweight Yolov8 pedestrian detection method and system based on unmanned aerial vehicle
Through the improved Yolov8 model, the GDConv and C2f modules are integrated, the problem of real-time and accuracy balance in drone pedestrian detection is solved, and efficient pedestrian recognition is achieved on drones, which is suitable for a variety of scenarios.
Patent Information
- Application Number
- CN202411066014.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-15
AI Technical Summary
There is a balance between real-time and accuracy in pedestrian detection tasks, and the computing power of the onboard computer is limited, making it difficult to meet the needs of efficient target detection.
The improved Yolov8 model is adopted, and the GDConv module and the C2f module are fused to form the Yolov8-GDC model. Image features are extracted through standard convolution, depth separation convolution and GELU activation functions, and the model is deployed on the drone for pedestrian detection.
It realizes fast, efficient and accurate pedestrian identification on drones, meeting the needs of lightweight architectures, and is suitable for both unmanned vehicles and unmanned ships.
Smart Images

Figure CN118968360A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of target detection carried on unmanned aerial vehicles, and in particular relates to a pedestrian detection method and system based on an improved lightweight Yolov8 of an unmanned aerial vehicle. Background Art
[0002] Drones, with their unique aerial perspective and flexible maneuverability, have shown great potential in areas such as pedestrian detection. Equipped with advanced sensors and camera technology, they can capture high-resolution image and video data in real time to provide support for accurate target identification and tracking. Compared with traditional ground monitoring systems, drones can obtain information more efficiently, reduce human resource requirements, and improve monitoring flexibility and coverage. However, due to the limitations of their size and load capacity, the computing power of their onboard computers is also limited. This limitation affects the efficiency and accuracy of drones in target detection tasks. Therefore, a lightweight and efficient model is needed to meet the needs, including a lightweight architecture, efficient target detection capabilities, and real-time detection.
[0003] The essence of target detection is to use computer vision technology to analyze and process images taken by drones to accurately detect pedestrians and other target objects in the image. Current target detection technologies can be divided into one-stage detection, such as YOLO, SSD, DETR, etc.; and two-stage detection, such as RCNN, Faster-RCNN, etc. Although these detection models have efficient detection capabilities and accuracy, they all need to be installed on high-performance computer platforms to achieve better real-time detection.
[0004] It can be seen that although the detection methods in target detection tasks have made great progress, there are still some challenges and difficulties in the problem of pedestrian detection on drones: such as the balance between real-time and accuracy, and whether the onboard computer of the drone meets the model operation requirements. Summary of the invention
[0005] In order to solve the problems existing in the prior art, the present invention provides a pedestrian detection method based on an improved lightweight Yolov8 of a drone. Using the basic framework of the Yolov8n model, a GDConv convolution module is proposed for the pedestrian detection target. At the same time, the C2f module in Yolov8n is improved, and the GDConv module is integrated to form a C2f-GDC module. While lightweighting the overall model, the accuracy of the model's pedestrian detection on the drone is improved. After training, a yolov8-GDC model is obtained, which is deployed on the drone device. The drone can recognize pedestrian targets in the images obtained by its camera, and realize fast, efficient and accurate pedestrian recognition.
[0006] In order to achieve the above object, the technical solution adopted by the present invention is: a pedestrian detection method based on an improved lightweight Yolov8 of a drone, comprising the following steps: detecting pedestrian targets in an image acquired in real time by a Yolov8-GDC model deployed on the drone, and marking the pedestrian targets respectively;
[0007] Taking Yolov8n as the basic network, the GDConv module is integrated into the C2f module in the Yolov8n backbone network to obtain the C2f-GDC module. All the C2f modules in the Backbone part of Yolov8n are replaced with C2f-GDC modules to form the Yolov8-GDC model.
[0008] Furthermore, the GDConv module includes a standard convolution module, a depth-separable convolution module, and a GELU activation function. The GDConv module extracts image features in the following steps:
[0009] First, the standard convolution module performs preliminary feature extraction x0 on the image img; the extracted features are segmented to obtain two identical feature information, which are recorded as x and v respectively;
[0010] Secondly, use the depthwise separable convolution module to further extract features from x to obtain x1. After extracting the complete features, use the GELU activation function to activate the deep feature information x2 of the image. Merge the deep feature information x2 and v to obtain the feature x3 that combines the shallow information and the deep information.
[0011] The standard convolution is used to further fuse x3 to obtain the feature y with rich information.
[0012] Furthermore, the way of C2f fusing GDConv module is to replace the residual module with GDConv module to form C2f-GDC module, and fuse C2f-GDC module into Yolov8n network to form Yolov8-GDC model; the process of extracting features by C2f-GDC module includes: extracting features of output image img through standard convolution module to obtain preliminary feature x1, extracting features of x1 through GDConv module to obtain feature x2, extracting features of x2 through standard convolution module to obtain feature x3, and finally outputting feature x3.
[0013] Furthermore, training the Yolov8-GDC model includes the following steps:
[0014] Obtain a data set and label the pedestrians in the data set;
[0015] Set the training parameters, use the training set and the test set to train the Yolov8-GDC model to obtain the trained model. In each round of model training, the Yolov8-GDC model extracts features of pedestrians in the images in the training set and associates the features with the target labels. After each round of model training, use the validation set to evaluate the model's accuracy, precision, and recall rate indicators. After the evaluation is completed, optimize the model parameters. After reaching the set accuracy, the model training is completed to obtain the trained model.
[0016] Furthermore, obtaining the data set and labeling the pedestrians in the data set include:
[0017] The drone flies high in the sky to take bird's-eye views and collect image data of scenes including densely populated urban streets, sparsely populated rural streets, bridges and parking lots. It also collects image data at different times including daytime, noon, evening and night, and data on different weather conditions including sunny days, rainy days, smog and dust.
[0018] Furthermore, the pedestrians in the acquired data set are labeled including:
[0019] To label pedestrians, use digital mapping labels:
[0020] Pedestrian → 0
[0021] Pedestrians are marked in the image by using a rectangular box to select each target. A rectangular coordinate system is established with the upper left corner of the image as the origin, the horizontal right direction as the x-axis, and the vertical downward direction as the y-axis. The absolute coordinates are normalized to obtain a relative coordinate. The change is as follows:
[0022]
[0023] (x,y,w,h)→(x norm_centor ,y norm_cenor ,w norm ,h norm )
[0024] In the formula, x morm_centor and norm_c represents the center point position of the normalized rectangular target box, w norm and h norm Represents half of the normalized relative width and relative height.
[0025] The information of a target in an image is as follows:
[0026] (x norm_centor ,y norm_cenor ,w norm ,h norm ,class)
[0027] In the formula, class represents the category of the target.
[0028] Furthermore, the trained Yolov8-GDC model is deployed to the drone to detect pedestrian targets in the images obtained by the drone camera, including:
[0029] First, build the environment configuration and necessary plug-ins for running the Yolov8-GDC model on the drone's onboard computer; after the installation is complete, upload the Yolov8-GDC model to the drone's onboard computer;
[0030] Secondly, the camera communicates data with the onboard computer of the drone. The drone can read the image information of the camera in real time. The onboard computer sends the image information topic in the form of a topic through ROS. The yolov8-GDC network reads the image information by subscribing to the topic, detects the read image information, detects pedestrian targets in the image, outputs the detected targets, and marks the targets with rectangular boxes.
[0031] The present invention can also provide a pedestrian detection system based on an improved lightweight Yolov8 of a drone, comprising a detection module, wherein the detection module detects pedestrian targets in images acquired in real time through a Yolov8-GDC model deployed on the drone, and respectively marks the pedestrian targets;
[0032] Among them, the Yolov8-GDC model takes Yolov8n as the basic network, integrates the GDConv module into the C2f module in the Yolov8n backbone network, and obtains the C2f-GDC module. The C2f modules in the Backbone part of Yolov8n are all replaced with C2f-GDC modules.
[0033] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, the improved lightweight Yolov8 pedestrian detection method based on the drone of the present invention can be implemented.
[0034] At the same time, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the pedestrian detection method based on the improved lightweight Yolov8 of the drone of the present invention can be implemented.
[0035] Compared with the prior art, the present invention has at least the following beneficial effects:
[0036] The method described in the present invention can be carried on a drone to accurately identify pedestrian targets. Secondly, the yolov8-GDC model deployed on the drone has a lightweight architecture, and can perform real-time detection on the drone onboard computer with high detection efficiency. Finally, the method is not only applicable to the drone field, but can also be used in unmanned vehicles, unmanned ships and other fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 The present invention is a flowchart of an implementable method.
[0038] Figure 2 This is a structural diagram of the GDConv module proposed in the present invention.
[0039] Figure 3 This is a structural diagram of the fusion of the GDConv module and the C2f module in the present invention.
[0040] Figure 4 This is a structural diagram of the Yolov8-GDC model improved based on the yolov8n framework of the present invention.
[0041] Figure 5 This is a detection result of an implementable method of the present invention. DETAILED DESCRIPTION
[0042] The technical solutions in the embodiments of the present invention are further described in detail below.
[0043] The process of the method proposed by the present invention is as follows Figure 1 As shown, the details are as follows:
[0044] Step 1: Obtain a data set through a camera carried by a drone; annotate pedestrians in the acquired data set and divide it into a training set, a validation set, and a test set;
[0045] Step 2, integrate the GDConv module into the Yolov8 network to form a Yolov8-GDC model;
[0046] Step 3, use the training set and the test set to train the Yolov8-GDC model to obtain the trained model;
[0047] The trained model is deployed to the drone to detect pedestrian targets in the images obtained by the drone camera, and finally pedestrian detection is achieved.
[0048] Furthermore, when acquiring data sets through the camera carried by drones, the drones fly high in the sky and take pictures from above. The drones need to collect data in different scenes, including densely populated streets in cities, sparsely populated streets in rural areas, bridges, parking lots, etc.; they need to collect data at different times, such as daytime, noon, evening, and night; they need to collect data in different weather conditions, such as sunny days, rainy days, foggy days, and dusty days. A rich variety of data sets can ensure that the trained model has good generalization to adapt to different scene capabilities.
[0049] The pedestrians in the acquired data set are labeled and divided into training set, validation set and test set including:
[0050] To label pedestrians, since computers cannot recognize Chinese characters, we use the method of digital mapping labels, as shown in formula (1).
[0051] Pedestrian → 0 (1)
[0052] Pedestrians are marked in the image by using a rectangular box to select each target. Since the image is a two-dimensional plane, a rectangular coordinate system is established with the upper left corner of the image as the origin, the horizontal right direction as the x-axis, and the vertical downward direction as the y-axis. Therefore, the position of the target in the image is represented as shown in formula (2).
[0053] ((x,y),(x+w,y),(x,y+h),(x+w,y+h)) (2)
[0054] It can be simply expressed as shown in formula (3)
[0055] (x,y,w,h) (3)
[0056] Where x and y represent the position of the upper left corner of the target rectangular box in the image rectangular coordinate system, and w and h represent the width and height of the target rectangular box, respectively.
[0057] Since there are size differences between images, using absolute coordinates will reduce the model's perception of the distribution of the target in the image. Therefore, the absolute coordinates can be normalized to obtain a relative coordinate, and the change method is shown in formula (4).
[0058]
[0059] In the formula, x morm_centor and norm_c represents the center point position of the normalized rectangular target box, w norm and h norm Represents half of the normalized relative width and relative height.
[0060] Therefore, the target information in an image is as shown in formula (5).
[0061] (x norm_centor ,y norm_cenor ,w norm ,h norm ,class) (5)
[0062] In the formula, class represents the category of the target.
[0063] Step 2: Integrate the GDConv module into the Yolov8 network to form a Yolov8-GDC model including:
[0064] A GDConv module is proposed, such as Figure 2 As shown in Figure 6, the module consists of three parts, namely, the standard convolution module (Conv), the depthwise separable convolution module (DWConv) and the GELU activation function. The feature extraction process of the GDConv module can include the following steps: First, the standard convolution function performs preliminary feature extraction x0 on the image img; the extracted features are segmented to obtain two identical feature information, which are recorded as x and v. Secondly, the depthwise separable convolution is used to further extract features from x to obtain x1, and after the complete feature extraction, the GELU activation function is used to obtain the deep feature information x2 of the image. Then, x2 and v are merged to obtain the feature x3 that combines the shallow information and the deep information. Finally, the standard convolution is used to further fuse x3 to obtain the feature y with rich information, as shown in formula (6).
[0065]
[0066] The GDConv module is integrated into the Yolov8 network. The Yolov8 network model can be divided into three parts, namely the Backbone part, the Yolo Head part and the Detect part. The Backbone part performs preliminary feature extraction on the image, including shallow, middle and deep feature information. The Yolo Head part fuses the extracted shallow, middle and deep feature information to obtain enhanced fusion feature information. The Detect part identifies and classifies the extracted features to obtain the identification results. The improved yolov8 model is based on the Yolov8n network, and the GDConv module is integrated into the C2f module in the Yolov8n backbone network to obtain the C2f-GDC module, such as Figure 3 As shown in the figure, all C2f modules in the Backbone part of Yolov8n are replaced with C2f-GDC modules to form an improved lightweight Yolov8-GDC model.
[0067] The C2f module contains two convolutional layers, which are connected by a residual module. The way to fuse C2f with the GDConv module is to replace the residual module with the GDConv module to form a C2f-GDC module. The process of extracting features in the C2f-GDC module includes: extracting features from the output image img through the standard convolution module to obtain preliminary features x1, extracting features from x1 through the GDConv module to obtain features x2, and extracting features from x2 through the standard convolution module to obtain features x3, and finally outputting features x3. The extraction process is shown in formula (7).
[0068]
[0069] After the improvement of the C2f-GDC module is completed, it is integrated into the Yolov8n network to form the Yolov8-GDC model, as shown in Figure 4 shown.
[0070] Step 3: Use the training set and test set to train the Yolov8-GDC model. The trained model includes:
[0071] First, set the training parameters, including the size of the input image, the batch size of the training image, the optimization function of the selected model, the objective function, etc. In each round of model training, the Yolov8-GDC model extracts features of pedestrians in the images in the training set and associates the features with the labels of the targets. After each round of model training, the validation set is used to evaluate the accuracy, precision, recall and other indicators of the model. After the evaluation is completed, the optimization function is used to optimize the model parameters. After reaching the set accuracy, the model training is completed and the trained model is obtained.
[0072] Among the evaluation indicators, accuracy refers to the ratio of all correctly predicted samples to the total samples, as shown in formula (8).
[0073]
[0074] In the formula, T represents the sample predicted correctly, F represents the sample predicted incorrectly, P represents the sample predicted as positive, and N represents the sample predicted as negative.
[0075] Among the evaluation indicators, precision refers to the ratio of the number of samples predicted as positive to the number of samples predicted as positive. As shown in formula (9):
[0076]
[0077] Among the evaluation indicators, recall rate refers to the ratio of samples predicted as positive to samples originally in the positive class, as shown in formula (10).
[0078]
[0079] Furthermore, the trained model is deployed to the drone to detect pedestrian targets in the images obtained by the drone camera. The final pedestrian detection includes:
[0080] First, build the environment configuration and necessary plug-ins for running the Yolov8-GDC model on the drone's onboard computer. After the installation is complete, upload the Yolov8-GDC model to the drone's onboard computer.
[0081] Secondly, the camera communicates with the onboard computer of the drone. The drone can read the visual information of the camera in real time. The onboard computer sends the image information topic in the form of a topic through ROS. The yolov8-GDC network reads the image information by subscribing to the topic, detects the read image information, detects the pedestrian target in the image, outputs the detected target, and marks the target with a rectangular box. The detection results are as follows: Figure 5 shown.
[0082] The present invention can also provide a pedestrian detection system based on an improved lightweight Yolov8 of a drone, comprising a detection module, wherein the detection module detects pedestrian targets in images acquired in real time through a Yolov8-GDC model deployed on the drone, and respectively marks the pedestrian targets;
[0083] Among them, the Yolov8-GDC model takes Yolov8n as the basic network, integrates the GDConv module into the C2f module in the Yolov8n backbone network, and obtains the C2f-GDC module. The C2f modules in the Backbone part of Yolov8n are all replaced with C2f-GDC modules.
[0084] On the other hand, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the pedestrian detection method based on the improved lightweight Yolov8 of the drone of the present invention can be implemented.
[0085] The computer device may be a laptop computer, a desktop computer or a workstation.
[0086] The present invention can also provide a computer device, including a processor and a memory, the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, the pedestrian detection method based on the improved lightweight Yolov8 of the drone described in the present invention can be implemented.
[0087] The processor may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or an off-the-shelf field programmable gate array (FPGA).
[0088] The memory described in the present invention may be an internal storage unit of a notebook computer, a desktop computer or a workstation, such as a memory or a hard disk; or an external storage unit, such as a mobile hard disk or a flash memory card.
[0089] Computer-readable storage media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media may include: read-only memory (ROM), random access memory (RAM), solid-state drive (SSD) or optical disk, etc. Among them, random access memory may include resistance random access memory (ReRAM) and dynamic random access memory (DRAM).
Claims
1. A pedestrian detection method based on improved lightweight Yolov8 for drones, It is characterized in that The following steps are involved: The Yolov8-GDC model deployed on the drone is used to detect and label pedestrian targets in real-time images. Taking Yolov8n as the basic network, the GDConv module is integrated into the C2f module in the Yolov8n backbone network to obtain the C2f-GDC module. All the C2f modules in the Backbone part of Yolov8n are replaced with C2f-GDC modules to form the Yolov8-GDC model.
2. The pedestrian detection method based on the improved lightweight Yolov8 of UAV according to claim 1, It is characterized in that The GDConv module includes a standard convolution module, a depth-separable convolution module, and a GELU activation function. The GDConv module extracts image features in the following steps: First, the standard convolution module performs preliminary feature extraction x on the image img 0 ; Segment the extracted features to obtain two identical feature information, recorded as x and v respectively; Secondly, use the depth-wise separable convolution module to further extract features from x to obtain x 1 , after extracting the complete features, use the GELU activation function to obtain the deep feature information of the image x 2 , the deep feature information x 2 Merge with v to obtain the feature x that combines shallow information and deep information 3 ; Use the standard convolution on x 3 Further fusion is performed to obtain feature y with rich information.
3. The pedestrian detection method based on the improved lightweight Yolov8 of UAV according to claim 1, It is characterized in that The way of C2f fusion GDConv module is to replace the residual module with GDConv module to form C2f-GDC module, and fuse C2f-GDC module into Yolov8n network to form Yolov8-GDC model; the process of C2f-GDC module extracting features includes: extracting features of output image img through standard convolution module to obtain preliminary features x 1 , x 1 Extract features through the GDConv module to get feature x 2 , x 2 The feature x is extracted by the standard convolution module 3 , the final output feature x 3 .
4. The pedestrian detection method based on the improved lightweight Yolov8 of UAV according to claim 1, It is characterized in that Training the Yolov8-GDC model consists of the following steps: Obtain a data set and label the pedestrians in the data set; Set the training parameters, use the training set and the test set to train the Yolov8-GDC model to obtain the trained model. In each round of model training, the Yolov8-GDC model extracts features of pedestrians in the images in the training set and associates the features with the target labels. After each round of model training, use the validation set to evaluate the model's accuracy, precision, and recall rate indicators. After the evaluation is completed, optimize the model parameters. After reaching the set accuracy, the model training is completed to obtain the trained model.
5. The pedestrian detection method based on the improved lightweight Yolov8 of UAV according to claim 4, It is characterized in that Obtaining a data set and labeling pedestrians in the data set includes: The drone flies high in the sky to take bird's-eye views and collect image data of scenes including densely populated urban streets, sparsely populated rural streets, bridges and parking lots. It also collects image data at different times including daytime, noon, evening and night, and data on different weather conditions including sunny days, rainy days, smog and dust.
6. The pedestrian detection method based on the improved lightweight Yolov8 of UAV according to claim 1, It is characterized in that Labeling pedestrians in the acquired data set includes: To label people, use digital mapping labels: Pedestrian → 0 Pedestrians are marked in the image by using a rectangular box to select each target. A rectangular coordinate system is established with the upper left corner of the image as the origin, the horizontal right direction as the x-axis, and the vertical downward direction as the y-axis. The absolute coordinates are normalized to obtain a relative coordinate. The change is as follows: (x,y,w,h)→(x norm_centor ,y norm_cenor ,w norm ,h norm ) In the formula, x norm_centor and norm_centor represents the center point position of the normalized rectangular target box, w norm and h norm Represents half of the normalized relative width and relative height. The information of a target in an image is as follows: (x norm_centor ,y norm_cenor ,w norm ,h norm ,class) In the formula, class represents the category of the target.
7. The pedestrian detection method based on the improved lightweight Yolov8 of UAV according to claim 1, It is characterized in that Deploy the trained Yolov8-GDC model to the drone to detect pedestrian targets in the images captured by the drone camera, including: First, build the environment configuration and necessary plug-ins for running the Yolov8-GDC model on the drone's onboard computer; after the installation is complete, upload the Yolov8-GDC model to the drone's onboard computer; Secondly, the camera communicates data with the onboard computer of the drone. The drone can read the image information of the camera in real time. The onboard computer sends the image information topic in the form of a topic through ROS. The yolov8-GDC network reads the image information by subscribing to the topic, detects the read image information, detects pedestrian targets in the image, outputs the detected targets, and marks the targets with rectangular boxes.
8. A pedestrian detection system based on improved lightweight Yolov8 for drones, It is characterized in that It includes a detection module, which detects pedestrian targets in real-time acquired images through the Yolov8-GDC model deployed on the drone and labels the pedestrian targets respectively; Among them, the Yolov8-GDC model takes Yolov8n as the basic network, integrates the GDConv module into the C2f module in the Yolov8n backbone network, and obtains the C2f-GDC module. The C2f modules in the Backbone part of Yolov8n are all replaced with C2f-GDC modules.
9. A computer device, It is characterized in that The invention comprises a processor and a memory, wherein the memory is used to store a computer executable program, and the processor reads part or all of the computer executable program from the memory and executes it. When the processor executes part or all of the computer executable program, the pedestrian detection method based on the improved lightweight Yolov8 of the drone as described in any one of claims 1 to 7 can be implemented.
10. A computer-readable storage medium, It is characterized in that A computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the pedestrian detection method of the improved lightweight Yolov8 based on the drone as described in any one of claims 1 to 7 can be implemented.
