Lightweight pedestrian detection method and system in complex scene based on improved YOLOv8

Abstract

The invention discloses a lightweight pedestrian detection method and system in a complex scene based on improved YOLOv8, and the method comprises the steps: optimizing a YOLOv8 model feature extraction network through an expansion residual module, and constructing comprehensive feature representation; the feature fusion network of the YOLOv8 model is optimized by introducing an OfficientRepBiPAN structure, and the feature fusion network of the YOLOv8 model is optimized; a new lightweight detection head is designed for the optimized YOLOv8 model; pruning the YOLOv8 model by adopting a channel pruning method for calculating importance scores by LAMP (Loop-Mediated Isothermal Amplification) to obtain an improved YOLOv8 model; and respectively adding the existing pedestrian data set into the YOLOv8 model before and after improvement for training and testing, and comparing the model detection results before and after improvement. According to the method adopted by the invention, the YOLOv8 with quick response is taken as a reference, the detection precision is improved, meanwhile, a model with the volume being only 0.9 MB is obtained, and the detection accuracy and the real-time performance can be effectively balanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of pedestrian detection, and in particular relates to a lightweight pedestrian detection method and system in complex scenes based on improved YOLOv8. Background Art

[0002] Pedestrian detection uses computer vision methods to obtain the location of pedestrians in images or videos. As an important branch of target detection, it is often used in crowded scenes such as large shopping malls, transportation hubs, and scenic spots, and is mostly deployed in edge devices such as surveillance.

[0003] Pedestrian detection algorithms based on deep learning can be divided into two-stage pedestrian detection algorithms and single-stage pedestrian detection algorithms. The two-stage detection algorithm is an algorithm based on candidate boxes. Representative algorithms include R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN, etc. Compared with the single-stage algorithm, it has the disadvantage of being slow. Typical single-stage detection algorithms such as RetinaNet, SSD and YOLO can complete the positioning and classification of targets at one time, run fast, and have more advantages in practical applications. Among them, the Yolo series network has received more attention because it can better balance detection accuracy and speed.

[0004] The series of methods currently proposed for pedestrian detection are effective in simple and specific scenarios, but they still perform poorly when facing small pedestrian targets and occlusion situations. Pedestrian targets are often small in scale and there are a lot of mutual occlusions, which affects the feature extraction of the network model, causing the model to miss or misdetect, bringing certain challenges to detection. In addition, in applications such as autonomous driving and real-time monitoring systems, real-time performance and deployment difficulty are important considerations. Therefore, it is urgent to develop a lightweight detection method that can improve detection accuracy to meet the needs of real-time and efficient pedestrian detection. Summary of the invention

[0005] The purpose of the present invention is to provide a lightweight pedestrian detection method and system in complex scenes based on improved YOLOv8 to solve the above problems.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a lightweight pedestrian detection method in complex scenes based on improved YOLOv8, comprising:

[0008] The feature extraction network of the YOLOv8 model is optimized through the dilated residual module to build a comprehensive feature representation;

[0009] The YOLOv8 model feature fusion network is optimized by introducing the efficient structural reparameterized bidirectional path aggregation network EfficientRepBiPAN structure;

[0010] Design a new lightweight detection head for the optimized YOLOv8 model;

[0011] The YOLOv8 model is pruned using the channel pruning method based on layer adaptive amplitude pruning LAMP to calculate the importance score, and the improved YOLOv8 model is obtained;

[0012] The existing pedestrian dataset is added to the YOLOv8 model before and after the improvement for training and testing, and the detection effects of the models before and after the improvement are compared.

[0013] Furthermore, the YOLOv8 model feature extraction network:

[0014] It is composed of a cross-stage partial dark network CSPDarknet. After the first CBS convolution layer, the image with an input size of 640*640 outputs a feature map with a size of 320*320 and a number of channels of 64, which is transmitted to the second CBS convolution layer; after the second CBS convolution layer, the feature map with a size of 160*160 and a number of channels of 128 is output, which is transmitted to the first cross-stage partial bottleneck structure C2f module with two layers of convolution; the size and number of channels of the feature map after the first C2f module remain unchanged, and it is transmitted to the third CBS convolution layer; after the third CBS convolution layer, the feature map with a size of 80*80 and a number of channels of 256 is output, which is transmitted to the second C2f module; the second cross-stage partial bottleneck structure C2f module with two layers of convolution has an output feature map with a size of 80*80, which has two directions, one On the one hand, it is passed into the fourth CBS convolutional layer, and on the other hand, it is passed into the feature fusion network; after passing through the fourth CBS convolutional layer, the output feature map with a size of 40*40 and the number of channels of 512 is transmitted to the third cross-stage partial bottleneck structure C2f module with two layers of convolution; the third cross-stage partial bottleneck structure C2f module with two layers of convolution has an output size of 40*40. The feature map still has two directions. On the one hand, it is passed into the fifth CBS convolutional layer, and on the other hand, it is passed into the feature fusion network; after passing through the fifth CBS convolutional layer, the output feature map with a size of 20*20 and the number of channels of 512 is transmitted to the fourth C2f module; the size and number of channels of the feature map after the fourth C2f module remain unchanged, and then it is processed by the spatial pyramid fast pooling SPPF module to increase the receptive field, and finally the result is passed to the feature fusion network.

[0015] Furthermore, the feature extraction network of the YOLOv8 model is optimized by dilating the residual module to construct a comprehensive feature representation:

[0016] Based on the feature extraction network, a dilated residual module is designed to improve the third and fourth C2f modules and replace the bottleneck structure in C2f. The dilated residual module first uses 3*3 convolution for preliminary feature extraction, and then uses convolutions with dilation rates of 1, 3, and 5 to perform convolution operations on the same input respectively. The results of the convolution operations are concatenated in the channel dimension, so that the model can simultaneously consider information from different dilation rates in subsequent layers. Subsequently, feature fusion is performed through 1*1 convolution, and the fusion result is added to the input feature, finally forming a residual connection to construct a comprehensive feature representation.

[0017] Furthermore, the YOLOv8 model feature fusion network is optimized by introducing the efficient reparameterized bidirectional path aggregation network EfficientRepBiPAN structure:

[0018] The feature fusion network is composed of a path aggregation network PAN and a feature pyramid network FPN. FPN passes down the strong semantic features of the high-level from top to bottom, and PAN adds a bottom-up pyramid behind FPN to pass up the strong positioning features of the low-level. The feature fusion network introduces the EfficientRepBiPAN structure. The reparameterization block RepBlock is composed of stacked modules of the reparameterization style rep-style. During training, 3*3 convolution is performed on the input feature map, and 1*1 convolution is performed at the same time. Finally, the input itself is added. Through reparameterization, the multi-branch structure is converted into a single-branch 3*3 convolution in the inference state.

[0019] Furthermore, a new lightweight detection head is designed for the optimized YOLOv8 model:

[0020] A very small target detection head with a resolution of 160*160 is added, and a redundant detection head with a resolution of 20*20 is subtracted. The P2, P3, and P4 feature maps after feature fusion are respectively passed through a convolution group CGN with a convolution kernel of 1*1. The group normalization GroupNorm is used to replace the batch normalization BN layer for normalization. The three branches pass through two convolution groups CGN with a convolution kernel of 3*3. The use of shared convolution greatly reduces the amount of parameters and calculations; the Scale layer is used to scale the features.

[0021] Furthermore, the channel pruning method of LAMP to calculate the importance score is used to prune the YOLOv8 model to obtain the improved YOLOv8 model:

[0022] Channel pruning is used to eliminate redundant parameters. Whether to subtract is determined based on the channel importance. The method for calculating the importance score uses LAMP; LAMP is a pruning score based on layer - adaptive amplitude; the score is the basis for deciding whether to retain the channel structure during the pruning process. Flatten each weight tensor into a one - dimensional vector. u and v represent the indices of the weights, and W(u) represents the weight term mapped by index u. Assume that the magnitudes of the weight terms satisfy the order of the index sizes, that is, u < v, then W(u) < W(v). Then the definition of the LAMP score is expressed as:

[0023]

[0024] (W[u]) 2 >(W[v]) 2 →score(u; W)>score(u; V) (2)

[0025] Where (W[u]) 2 represents the square of the weight amplitude of the target connection, and ∑ v≥u (W[v]) 2 represents the sum of the squares of the weight amplitudes of all remaining connections in the same layer; according to formula (2), the larger the weight term, the larger the corresponding LAMP score. The score is bound to the weight term and the channel importance. The weight terms with low LAMP scores are considered relatively unimportant parts and are pruned. After setting a target pruning ratio, LAMP selects the connections with the smallest LAMP scores for pruning, which is different from local pruning and does not necessarily prune the same ratio for each layer.

[0026] Furthermore, the existing pedestrian datasets are added to the YOLOv8 model before and after improvement for training and testing respectively, and the detection effects of the models before and after improvement are compared:

[0027] The selected pedestrian dataset is CityPersons, and the image labels are annotated as "person". The model weights trained before and after improvement are tested on the validation set, and finally the detection results are compared to draw a conclusion.

[0028] In the second aspect, the present invention provides a lightweight pedestrian detection system in complex scenarios based on the improved YOLOv8, including:

[0029] A feature extraction network optimization module for optimizing the feature extraction network of the YOLOv8 model through an expanded residual module to construct a comprehensive feature representation;

[0030] A feature fusion network optimization module for optimizing the feature fusion network of the YOLOv8 model by introducing the EfficientRepBiPAN structure;

[0031] The detection head reconstruction module is used to design a new detection head for the optimized YOLOv8 model and reconstruct the YOLOv8 model detection head;

[0032] The pruning module is used to prune the YOLOv8 model using the channel pruning method of LAMP to calculate the importance score, and obtain the improved YOLOv8 model;

[0033] The detection comparison module is used to add the existing pedestrian dataset to the YOLOv8 model before and after the improvement for training and testing, and to compare the detection effects of the models before and after the improvement.

[0034] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of a lightweight pedestrian detection method in complex scenes based on improved YOLOv8 are implemented.

[0035] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of a lightweight pedestrian detection method in complex scenes based on improved YOLOv8.

[0036] Compared with the prior art, the present invention has the following technical effects:

[0037] The method adopted by the present invention is based on the fast-response YOLOv8, which can effectively balance detection accuracy and real-time performance.

[0038] The improved feature extraction network of the present invention aims to improve the difficulty of too few or blurred visible features of occluded pedestrian targets during the detection process. The dilated residual module is combined with the deep C2f to expand the receptive field to strengthen the extracted features and construct a comprehensive feature representation.

[0039] The improved feature fusion network of the present invention improves network efficiency, fuses features of multi-level pedestrians, and makes up for the defect of single feature fusion of the original feature fusion network.

[0040] The lightweight reconstruction detection head designed in the present invention solves the problem of difficulty in extracting features of small-target pedestrians, and effectively improves the pedestrian detection accuracy while reducing model parameters.

[0041] The present invention adopts the LAMP method to calculate the importance, performs channel pruning on the YOLOv8 model, and performs global pruning according to the channel importance, thereby greatly reducing the number of algorithm parameters and calculation amount while maintaining the accuracy of pedestrian detection, thereby reducing the model size.

[0042] In summary, the proposed lightweight pedestrian detection method in complex scenes based on improved YOLOv8 can effectively cope with pedestrian detection in complex scenes compared with the existing technology, solves the problem of poor detection of long-distance and occluded pedestrians in pedestrian detection, improves the accuracy of pedestrian detection in smart community scenarios, and greatly reduces the number of parameters and calculations of the algorithm, meeting the real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A model framework diagram of a lightweight pedestrian detection method in a complex scene based on an improved YOLOv8 provided in an embodiment of the present invention;

[0044] Figure 2 A structural schematic diagram of a dilated residual module DWR and a C2f_DWR module provided in an embodiment of the present invention;

[0045] Figure 3 It is a structural schematic diagram of a lightweight detection head LS_Detect provided in an example of the present invention;

[0046] Figure 4 A before-and-after comparison diagram of a channel-based LAMP pruning channel provided in an example of the present invention;

[0047] Figure 5(a) shows the detection classification effect and prediction diagram of the original YOLOv8 model;

[0048] Figure 5(b) shows the detection classification effect and prediction diagram of the improved YOLOV8 model;

[0049] Figure 6(a) shows the detection and classification effect and prediction diagram of the original YOLOv8 model under dim conditions;

[0050] Figure 6(b) shows the detection classification effect and prediction diagram of the improved YOLOV8 model under dim conditions;

[0051] Figure 7 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described below in conjunction with the accompanying drawings:

[0053] See also Figure 7 , a lightweight pedestrian detection method in complex scenes based on improved YOLOv8, including:

[0054] The feature extraction network of the YOLOv8 model is optimized through the dilated residual module to build a comprehensive feature representation;

[0055] The YOLOv8 model feature fusion network is optimized by introducing the EfficientRepBiPAN structure;

[0056] Design a new detection head for the optimized YOLOv8 model and reconstruct the YOLOv8 model detection head;

[0057] The YOLOv8 model is pruned using the LAMP channel pruning method to calculate the importance score, and an improved YOLOv8 model is obtained;

[0058] The existing pedestrian dataset is added to the YOLOv8 model before and after the improvement for training and testing, and the detection effects of the models before and after the improvement are compared.

[0059] Specific:

[0060] The feature extraction network is composed of CSPDarknet. After the image with an input size of 640*640 passes through the first CBS convolution layer, the output size is 320*320 and the number of channels is 64, which is transmitted to the second CBS convolution layer; after passing through the second CBS convolution layer, the output size is 160*160 and the number of channels is 128, which is transmitted to the first C2f module; after passing through the first C2f module, the size and number of channels of the feature map remain unchanged and are transmitted to the third CBS convolution layer; after passing through the third CBS convolution layer, the output size is 80*80 and the number of channels is 256, which is transmitted to the second C2f module; the second C2f module outputs a feature map with a size of 80*80 with two On the one hand, it is passed into the fourth CBS convolution layer, and on the other hand, it is passed into the feature fusion network; after passing through the fourth CBS convolution layer, the output feature map with a size of 40*40 and a number of channels of 512 is transmitted to the third C2f module; the third C2f module outputs a feature map with a size of 40*40, which still has two directions. On the one hand, it is passed into the fifth CBS convolution layer, and on the other hand, it is passed into the feature fusion network; after passing through the fifth CBS convolution layer, the output feature map with a size of 20*20 and a number of channels of 512 is transmitted to the fourth C2f module; the size and number of channels of the feature map after the fourth C2f module remain unchanged, and are then processed by the SPPF module to increase the receptive field, and finally the result is passed to the feature fusion network.

[0061] Based on the feature extraction network, a dilated residual module is designed to improve the third and fourth C2f modules and replace the Bottleneck structure in C2f. The dilated residual module adopts a residual structure, first uses a convolution with a convolution kernel of 3*3 for preliminary feature extraction to obtain the approximate feature information of the input feature map, and then uses convolutions with dilation rates of 1, 3, and 5 to perform convolution operations on the same input respectively, and splices the convolution results in the channel dimension, so that the model can simultaneously consider information from different dilation rates in subsequent feature extraction. Subsequently, feature fusion is performed through 1*1 convolution, and the fusion result is added to the input feature to form a residual connection to construct a comprehensive feature representation.

[0062] The feature fusion network is composed of PAN and FPN. FPN transmits the strong semantic features of the high-level from top to bottom, but only enhances the semantic information without transmitting the positioning information. In view of this, PAN adds a bottom-up pyramid behind FPN to transmit the strong positioning features of the low-level.

[0063] The feature fusion network introduces a more efficient EfficientRepBiPAN structure, and introduces RepBlock and BuFusion structures. The structure of RepBlock is a module composed of an initial feature extraction layer, a repetitive structure, a feature fusion layer and a residual connection. It can be adjusted and designed according to the requirements of the task to build a more complex and deep neural network. RepBlock is stacked by rep-style modules. During training, three operations are performed on the input feature map, namely, 3*3 convolution, 1*1 convolution at the same time, and finally the input itself is added. After the above steps, it is re-parameterized, and the multi-branch structure is converted to a single-branch 3*3 convolution in the inference state. The BiFusion block is used in the PAN network to fuse feature maps of different scales. First, the first feature map is upsampled, then the second feature map is convolved, and finally the third feature map is convolved and downsampled. The three feature maps are then spliced ​​in the channel dimension, and the results are returned to generate an output feature map.

[0064] The YOLOv8 model has only three scale features, P3-P5, available for feature fusion, and has three corresponding detection heads with resolutions of 80*80 (P3), 40*40 (P4) and 20*20 (P5), which makes it difficult to detect pedestrians with large scale changes.

[0065] A very small target detection head with a resolution of 160*160 is added, and a redundant detection head with a resolution of 20*20 is subtracted to enhance the ability to detect small target pedestrians; the detection head in YOLOv8 is the key to detecting targets in YOLOv8, accounting for 1 / 5 of the computational effort. In order to reduce the computational pressure of model deployment, a lightweight detection head is designed; the use of shared convolutions greatly reduces the number of parameters and computational effort, making the model lighter, especially on resource-constrained devices. While using shared convolutions, in order to address the problem of inconsistent target scales detected by each detection head, the Scale layer is used to scale the features, and GroupNorm can improve the performance of detection head positioning and classification.

[0066] Channel pruning is used to eliminate redundant parameters. Whether to subtract is determined based on the importance of the channel. LAMP is used to calculate the importance score. LAMP is a global pruning method that overcomes the layer collapse problem that occurs in previous global pruning. After setting a target pruning ratio, LAMP selects the link with the minimum LAMP score for pruning. When pruning, speed_up is set to 1.9. After pruning, the model is greatly compressed, and the accuracy is improved by 0.2 percentage points.

[0067] The pedestrian dataset is CityPersons. The experimental training set has 2975 images and the validation set has 500 images. The image label is marked as "person". The training parameters are: batch size 16, initial learning rate lr0=0.01, weight decay weight_decay=0.0005, and a total of 200 epochs are trained. The model weights trained before and after the improvement are tested on the validation set, and finally the detection results are compared to draw conclusions.

[0068] Example:

[0069] The present invention proposes a lightweight pedestrian detection method in complex scenes based on improved YOLOv8, the steps are as follows:

[0070] S1. Innovatively optimize the feature extraction network of the YOLOv8 model;

[0071] S2. Optimize and improve the feature fusion network of the YOLOv8 model;

[0072] S3. Design a new detection head and reconstruct the YOLOv8 model detection head;

[0073] S4. Pruning YOLOv8 model using LAMP channel pruning method to calculate importance scores;

[0074] S5. Add the existing pedestrian dataset to the YOLOv8 model before and after the improvement for training and testing, and compare the detection effects of the models before and after the improvement.

[0075] Furthermore, in step S1, a dilated residual module is designed. The dilated residual module can enhance the feature extraction from the scalable receptive field of the high-level network. Therefore, the third and fourth C2f modules at the deeper level of the network are selected to improve the network, that is, the 3*3 convolution in the Bottleneck in the C2f structure is replaced to form the C2f_DWR module. The dilated residual module first performs preliminary feature extraction on the input feature map through a convolution with a convolution kernel of 3*3 size, and then uses a convolution operation with a convolution kernel of 3 and a dilation rate of 1, 3, and 5 to perform a convolution operation on the input. The results of the convolution operation are spliced ​​in the channel dimension, so that the model can simultaneously consider information from different dilation rates in subsequent layers. Subsequently, feature fusion is performed through 1*1 convolution, and the fusion result is added to the input feature through residual connection to construct a comprehensive feature representation. The improved feature extraction network can more effectively learn pedestrian features and improve pedestrian detection accuracy.

[0076] Furthermore, in step S2, a more efficient EfficientRepBiPAN structure is introduced in the feature fusion network. The structure of RepBlock is a module composed of an initial feature extraction layer, a repetitive structure, a feature fusion layer, and a residual connection. It can be adjusted and designed according to the requirements of the task to build a more complex and deep neural network. RepBlock is composed of a stack of rep-style modules. During training, a 3*3 convolution is performed on the input feature map, and a 1*1 convolution is performed at the same time. Finally, the input itself is added. By reparameterization, the multi-branch structure is converted to a single-branch 3*3 convolution in the inference state, which can effectively reduce memory consumption and inference latency. The BiFusion block is used in the PAN network to fuse feature maps of different scales. First, the first feature map is upsampled, then the second feature map is convolved, and finally the third feature map is convolved and downsampled. The three feature maps are then concatenated in the channel dimension and the result is returned to generate an output feature map.

[0077] Furthermore, in step S3, a very small object detection head with a resolution of 160*160 is added, and a redundant detection head with a resolution of 20*20 is subtracted to cope with the challenges brought by multi-scale targets. Especially for the P2 feature map, it is more conducive to the positioning of small target pedestrians. Finally, the output feature maps are 160*160 (P2), 80*80 (P3), and 40*40 (P4). To reduce the computational pressure of model deployment, a lightweight detection head is designed. The P2, P3, and P4 feature maps after feature fusion are respectively passed through a convolutional group CGN with a convolution kernel of 1*1, where GroupNorm is used to replace the BN layer for normalization operations to improve the performance of the detection head in positioning and classification. The three branches together pass through two convolutional groups CGN with a convolution kernel of 3*3. By using shared convolutions, the number of parameters and the amount of computation are significantly reduced, making the model more lightweight, especially on resource-constrained devices. While using shared convolutions, to address the problem of inconsistent target scales detected by each detection head, a Scale layer is used to scale the features. The reconstructed lightweight detection head can not only solve the problem that it is difficult to detect targets as pedestrians move away and the targets become smaller under the monitoring perspective, but also reduce the number of parameters.

[0078] Furthermore, in step S4, channel pruning is used to eliminate redundant parameters. Whether to subtract is determined according to the channel importance. The method for calculating the importance score uses LAMP. LAMP (Layer-Adaptive Magnitude-based Pruning) is the pruning (LAMP) score based on layer adaptive magnitude. The score is the basis for deciding whether to retain the channel structure during the pruning process. The currently widely used evaluation criterion is to compare the absolute values of the weights. The part to be pruned is equivalent to those weights with smaller absolute values, which are considered to make no significant contribution to the function. LAMP overcomes the problem of layer collapse that occurs in previous global pruning. After setting a target pruning ratio, LAMP selects the link with the smallest LAMP score for pruning without the need for prior sparsification training. Flatten each weight tensor into a one-dimensional vector. u and v represent the indices of the weights, and W(u) represents the weight term mapped by index u. Assume that the magnitudes of the weight terms satisfy the order of the index sizes, that is, u < v, then W(u) < W(v). Then the definition of the LAMP score is expressed as:

[0079]

[0080] (W[u]) 2 >(W[v]) 2 →score(u; W) > score(u; V) (2)

[0081] where (W[u]) 2 represents the square of the weight magnitude of the target connection, ∑ v≥u(W[v]) 2 It represents the sum of the squares of the weight amplitudes of all remaining connections in the same layer. According to formula (2), the larger the weight item, the larger the corresponding LAMP score. Therefore, the score is bound to the weight item and the channel importance. The weight item with a low LAMP score is considered to be a relatively unimportant part and is pruned. After setting a target pruning ratio, LAMP selects the link with the smallest LAMP score for pruning. Unlike local pruning, it does not necessarily prune the same ratio at each layer.

[0082] When pruning, speed_up is set to 1.9. The pruning operation can greatly compress the model. Under the premise of lightweight model, the accuracy does not decrease but increases by 0.2 percentage points. Figure 4 The figure shows the change in channels in each layer after pruning using LAMP (1.9). The number of channels in most convolutional layers is reduced.

[0083] Furthermore, in step S5, the pedestrian dataset selected is CityPersons, with 2975 images in the experimental training set and 500 images in the validation set, and the image label is marked as "person". The experimental environment is Ubuntu 20.04.4, and the experiment is conducted on 4 Tesla T4GPUS, with the configuration environment of cuda 11.4, pytorch 1.13.1 and python 3.8.13. During training, batch size is set to 16, initial learning rate lr0 = 0.01, weight decay weight_decay = 0.0005, and a total of 200 epochs are trained.

[0084] After training, the best weight best.pt will be saved in the training file. The trained models before and after the improvement will be tested on the validation set using best.pt. The performance of the tested target network model will be evaluated using preset performance evaluation indicators, including precision P, recall R, average precision AP, average AP value mAP, and IOU intersection-over-union ratio.

[0085]

[0086] Where TP (True Positive) represents the number of correctly detected targets, FP (False Positive) represents the number of backgrounds detected as targets, and FN (False Negative) represents the number of targets detected as backgrounds.

[0087]

[0088] Where AP is the average accuracy of each category, mAP represents the average of AP of multiple categories, m is the number of categories, and in our pedestrian detection task, m = 1.

[0089]

[0090] The intersection over union (IoU) is used to calculate the similarity between the predicted box and the true box, where the predicted box is A and the true box is B.

[0091] The present invention is compared with Yolov5, Yolov7-tiny, original YOLOv8n, Yolov8s and other algorithms in terms of parameters, computational GFLOPs, detection accuracy map and model size on the public data set CityPersons. The experimental comparison results of various algorithms are shown in Table 1:

[0092] Table 1 Comparison of experimental results of various algorithms

[0093]

[0094] It can be seen from Table 1 that although the accuracy P of the SSD algorithm has reached 91.1%, the recall rate R is extremely low, only 13.7%, and the detection accuracy is also extremely unsatisfactory. In the one-stage network, the mAP value of m among the three model sizes of YOLOv8 is the highest, but it has the largest number of parameters and the most computational effort, and the model is the largest, and does not support training on devices with insufficient computing power. Yolov9 did not have a brilliant performance, and the overall effect was average. The proposed lightweight pedestrian detection method in complex scenes based on the improved YOLOv8 has significant advantages in indicators. The average detection accuracy mAP@.5 and mAP@.5:.95 of our model for pedestrian detection reached 65.6% and 41.1% respectively, which is 6.3% and 4.9% higher than the detection accuracy of the Yolov8n model. At the same time, compared with Yolov8n, the number of parameters of the improved model is reduced from 3.01M to 0.32M, a decrease of 89%, and the model size is reduced from 5.96MB to 0.90MB, a reduction of 85%. This shows that the improved model has higher accuracy in detection and lower hardware cost requirements when deploying applications. The fps test reaches 62.5, which meets the real-time requirements and is more feasible and superior than existing technologies.

[0095] As shown in Figures 5 and 6, the test results are finally compared to draw conclusions.

[0096] Figure 5(a) and Figure 6(a) are the detection effect diagrams of the original YOLOv8 model, and Figure 5(b) and Figure 6(b) are the detection effect diagrams of the improved model. It can be seen that the original algorithm misses small-target pedestrians and occluded pedestrians in the distance, and the improved algorithm model has a better pedestrian detection effect, effectively detecting small-target pedestrians in the distance and occluded pedestrians in dense scenes, and in dim and harsh conditions, the accuracy of pedestrian detection is higher, which is 13 percentage points higher than the original algorithm.

[0097] In yet another embodiment of the present invention, a lightweight pedestrian detection system in complex scenes based on improved YOLOv8 is provided, which can be used to implement the above-mentioned lightweight pedestrian detection method in complex scenes based on improved YOLOv8. Specifically, the system includes:

[0098] Feature extraction network optimization module, which is used to optimize the feature extraction network of the YOLOv8 model through the dilated residual module to build a comprehensive feature representation;

[0099] Feature fusion network optimization module, which is used to optimize the feature fusion network of the YOLOv8 model by introducing the EfficientRepBiPAN structure;

[0100] The detection head reconstruction module is used to design a new detection head for the optimized YOLOv8 model and reconstruct the YOLOv8 model detection head;

[0101] The pruning module is used to prune the YOLOv8 model using the channel pruning method of LAMP to calculate the importance score, and obtain the improved YOLOv8 model;

[0102] The detection comparison module is used to add the existing pedestrian dataset to the YOLOv8 model before and after the improvement for training and testing, and to compare the detection effects of the models before and after the improvement.

[0103] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0104] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the lightweight pedestrian detection method in complex scenes based on the improved YOLOv8.

[0105] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, an extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the lightweight pedestrian detection method in complex scenes based on the improved YOLOv8 in the above embodiment.

[0106] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A lightweight pedestrian detection method in complex scenes based on improved YOLOv8, characterized by: include: The feature extraction network of the YOLOv8 model is optimized through the dilated residual module to build a comprehensive feature representation; The YOLOv8 model feature fusion network is optimized by introducing the efficient structural reparameterized bidirectional path aggregation network EfficientRepBiPAN structure; Design a new lightweight detection head for the optimized YOLOv8 model; The YOLOv8 model is pruned using the channel pruning method based on layer adaptive amplitude pruning LAMP to calculate the importance score, and the improved YOLOv8 model is obtained; The existing pedestrian dataset is added to the YOLOv8 model before and after the improvement for training and testing, and the detection effects of the models before and after the improvement are compared.

2. The lightweight pedestrian detection method in complex scenes based on improved YOLOv8 according to claim 1 is characterized in that: YOLOv8 model feature extraction network: It is composed of a cross-stage partial dark network CSPDarknet. After the first CBS convolution layer, the image with an input size of 640*640 outputs a feature map with a size of 320*320 and a number of channels of 64, which is transmitted to the second CBS convolution layer; after the second CBS convolution layer, the feature map with a size of 160*160 and a number of channels of 128 is output, which is transmitted to the first cross-stage partial bottleneck structure C2f module with two layers of convolution; the size and number of channels of the feature map after the first C2f module remain unchanged, and it is transmitted to the third CBS convolution layer; after the third CBS convolution layer, the feature map with a size of 80*80 and a number of channels of 256 is output, which is transmitted to the second C2f module; the second cross-stage partial bottleneck structure C2f module with two layers of convolution has an output feature map with a size of 80*80, which has two directions, one On the one hand, it is passed into the fourth CBS convolutional layer, and on the other hand, it is passed into the feature fusion network; after passing through the fourth CBS convolutional layer, the output feature map with a size of 40*40 and the number of channels of 512 is transmitted to the third cross-stage partial bottleneck structure C2f module with two layers of convolution; the third cross-stage partial bottleneck structure C2f module with two layers of convolution has an output size of 40*40. The feature map still has two directions. On the one hand, it is passed into the fifth CBS convolutional layer, and on the other hand, it is passed into the feature fusion network; after passing through the fifth CBS convolutional layer, the output feature map with a size of 20*20 and the number of channels of 512 is transmitted to the fourth C2f module; the size and number of channels of the feature map after the fourth C2f module remain unchanged, and then it is processed by the spatial pyramid fast pooling SPPF module to increase the receptive field, and finally the result is passed to the feature fusion network.

3. The lightweight pedestrian detection method in complex scenes based on improved YOLOv8 according to claim 2 is characterized in that: The feature extraction network of the YOLOv8 model is optimized through the dilated residual module to build a comprehensive feature representation: Based on the feature extraction network, a dilated residual module is designed to improve the third and fourth C2f modules, replacing the bottleneck structure in C2f; this dilated residual module first uses a 3*3 convolution for preliminary feature extraction, and then uses convolutions with dilation rates of 1, 3, and 5 to perform convolution operations on the same input respectively. The results of the convolution operations are concatenated in the channel dimension, enabling the model to consider information from different dilation rates simultaneously in subsequent layers; subsequently, a 1*1 convolution is used for feature fusion, and the fusion result is added to the input features, finally forming a residual connection to construct a comprehensive feature representation.

4. The lightweight pedestrian detection method in complex scenes based on improved YOLOv8 according to claim 1 is characterized in that: Optimize the feature fusion network of the YOLOv8 model by introducing the EfficientRepBiPAN structure of the efficient reparameterized bidirectional path aggregation network: The feature fusion network is composed of the path aggregation network PAN and the feature pyramid network FPN. FPN passes down the strong semantic features of the high layer from top to bottom, and PAN adds a bottom-up pyramid behind FPN to pass up the strong localization features of the low layer; The EfficientRepBiPAN structure is introduced into the feature fusion network. The reparameterization block RepBlock is stacked by modules in the reparameterization style. During training, a 3*3 convolution is performed on the input feature map, and a 1*1 convolution is also performed simultaneously. Finally, the input itself is added. Through reparameterization, the multi-branch structure is converted into a single-branch 3*3 convolution in the inference state.

5. The lightweight pedestrian detection method in complex scenes based on improved YOLOv8 according to claim 1, characterized in that: Design a lightweight detection head for the optimized YOLOv8 model: Add a tiny object detection head with a resolution of 160*160, subtract the redundant detection head with a resolution of 20*20. The P2, P3, and P4 feature maps after feature fusion pass through a convolution group CGN with a 1*1 convolution kernel respectively, where group normalization GroupNorm is used to replace the batch normalization BN layer for normalization operations. The three branches together pass through two convolution groups CGN with 3*3 convolution kernels, and then the classification loss and bounding box loss are calculated. By using shared convolutions, the number of parameters and the amount of computation are significantly reduced; a Scale layer is used to scale the features.

6. The lightweight pedestrian detection method in complex scenes based on improved YOLOv8 according to claim 1, characterized in that: Prune the YOLOv8 model using the channel pruning method of calculating importance scores by LAMP to obtain the improved YOLOv8 model: Channel pruning is used to eliminate redundant parameters. Whether to subtract is determined according to the channel importance, and the method of calculating importance scores uses LAMP; LAMP is the pruning score based on layer adaptive magnitude; the score is the basis for deciding whether to retain the channel structure during the pruning process; each weight tensor is flattened into a one-dimensional vector, u and v represent the indices of the weights, W(u) represents the weight term mapped by index u. Assuming that the size of the weight terms satisfies the order of the index size, that is, u < v, then W(u) < W(v). Then the definition of the LAMP score is expressed as: ( [u]) 2 >(W[v]) 2 → score(u;W)>score(u;V) (2) Among them (W[u]) 2 represents the square of the weight magnitude of the target connection, ∑ v≥u (W[v]) 2 It represents the sum of the squares of the weight amplitudes of all remaining connections in the same layer. According to formula (2), the larger the weight item, the larger the corresponding LAMP score. The score is bound to the weight item and the channel importance. The weight item with a low LAMP score is considered to be a relatively unimportant part and is pruned. After setting a target pruning ratio, LAMP selects the link with the minimum LAMP score for pruning. Unlike local pruning, it is not necessarily necessary to prune the same ratio in each layer.

7. The lightweight pedestrian detection method in complex scenes based on improved YOLOv8 according to claim 1, characterized in that: Add the existing pedestrian dataset to the YOLOv8 model before and after improvement for training and testing, and compare the detection effects of the models before and after improvement: The pedestrian dataset selected is CityPersons, and the image label is marked as "person". The model weights trained before and after the improvement are tested on the validation set, and finally the detection results are compared to draw conclusions.

8. A lightweight pedestrian detection system in complex scenes based on improved YOLOv8, characterized by: include: Feature extraction network optimization module, which is used to optimize the feature extraction network of the YOLOv8 model through the dilated residual module to build a comprehensive feature representation; Feature fusion network optimization module, which is used to optimize the feature fusion network of the YOLOv8 model by introducing the EfficientRepBiPAN structure; The detection head reconstruction module is used to design a new detection head for the optimized YOLOv8 model and reconstruct the YOLOv8 model detection head; The pruning module is used to prune the YOLOv8 model using the channel pruning method of LAMP to calculate the importance score, and obtain the improved YOLOv8 model; The detection comparison module is used to add the existing pedestrian dataset to the YOLOv8 model before and after the improvement for training and testing, and to compare the detection effects of the models before and after the improvement.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the lightweight pedestrian detection method in complex scenes based on improved YOLOv8 are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the lightweight pedestrian detection method in complex scenes based on improved YOLOv8 are implemented as described in any one of claims 1 to 7.

Images