Improved YOLOv8n-based crowded scene pedestrian detection method

Abstract

The invention discloses a crowded scene pedestrian detection method based on improved YOLOv8n. The method specifically comprises the following steps: (1) establishing a crowded scene pedestrian data set; (2) adding annotation information to the images in the data set; (3) constructing a crowded scene pedestrian detection model based on the improved YOLOv8n; (4) training the model by adopting the training set and the verification set, and storing the trained model as an optimal model; and (5) testing the optimal model by using the test set, and obtaining the final crowded scene pedestrian detection model based on the improved YOLOv8n if the test result meets the precision requirement. Compared with the prior art, the crowded scene pedestrian detection method based on the improved YOLOv8n disclosed by the invention effectively reduces the false detection rate and the missing detection rate of pedestrian detection in the crowded scene.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a pedestrian detection method in crowded scenes based on improved YOLOv8n. Background Art

[0002] In recent years, due to the improvement of people's material living standards and the acceleration of urbanization, the number of urban residents has increased rapidly, resulting in frequent crowds in shopping malls, stations, streets, hospitals, scenic spots, etc. These crowded environments have many safety hazards, such as stampedes, collision injuries, and traffic congestion. These safety hazards often cause serious property losses and casualties. In order to effectively deal with these problems and ensure public safety, it is particularly important to develop a model that can accurately and quickly identify pedestrian targets in crowded scenes.

[0003] However, pedestrian detection in crowded scenes faces multiple challenges. In addition to the common factors such as lighting changes, complex and diverse backgrounds, and different shooting angles, the characteristics of the human body, such as structural similarities, posture diversity, scale differences, and clothing differences, all increase the difficulty of detection. What is even more difficult is that there is a common occlusion between pedestrians. Occlusion makes it difficult for the model to accurately separate the target object from the background and other pedestrians, which in turn affects the quality of the detection results.

[0004] At present, there are three main methods for monitoring people in crowded scenes, namely, manual inspection, traditional pedestrian detection methods, and pedestrian detection methods based on deep learning. Manual inspection relies on security personnel patrolling on site or real-time observation through video surveillance systems. Although this method is intuitive and flexible, it is inefficient and prone to negligence or omissions due to human factors. Traditional pedestrian detection methods obtain candidate areas through sliding window traversal, extract pedestrian features manually, train a classifier, use the trained classifier to classify the scanning window area and output the target detection results. Classic algorithms include HOG+SVM, Haar+Adaboost, etc. These detection algorithms have the disadvantages of high dimension, large amount of calculation, time-consuming and labor-intensive, weak feature extraction ability and poor generalization. With the rapid development of deep learning, deep learning models have been widely used in the field of pedestrian detection. Deep learning methods are a type of neural network algorithm that extracts internal hierarchical features of images by autonomously learning a large amount of training data, avoiding the tediousness and inaccuracy of manual feature extraction in traditional methods. Representative algorithms include CNN, YOLO, SSD, etc.

[0005] In crowded scenes, when there is severe occlusion between pedestrians, small-scale occluded pedestrians only contain a single or a few pixels in the feature map, which causes the model to tend to capture large-scale pedestrian features and ignore small-scale pedestrian features. Existing models still miss or misdetect when dealing with such problems. Therefore, it is necessary to propose a pedestrian detection method to reduce the impact of occlusion and multi-scale problems in crowded scenes. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a pedestrian detection method for crowded scenes based on improved YOLOv8n. By improving the YOLOv8n algorithm, the problems of false detection and missed detection in the pedestrian detection model in crowded scenes caused by factors such as severe occlusion between pedestrians and diverse scale changes in crowded scenes are reduced.

[0007] The technical solution provided by the present invention comprises the following steps: Step 1: Obtain pedestrian images in crowded scenes to form the first data set; Step 2: adding annotation information to the images in the first data set to form a second data set, and dividing the second data set into a training set, a validation set, and a test set; Step 3: Construct a crowded scene pedestrian detection model based on improved YOLOv8n, the model includes a backbone network, a feature fusion network DCPAN and a head network, and the construction of the model further includes steps 3.1 to 3.3: Step 3.1: The backbone network consists of a convolutional layer 1, a convolutional layer 2, a C2f module 1, a convolutional layer 3, a C2f module 2, a convolutional layer 4, a C2f module 3, a convolutional layer 5, a C2f module 4 and an SPPF module connected in sequence; Using the training set and the validation set in the second data set as input of the backbone network; The backbone network outputs feature information of four different scales through C2f module 1, C2f module 2, C2f module 3 and SPPF module respectively; Step 3.2: The feature fusion network DCPAN, based on the YOLOv8n neck network, adds convolution layer 7, convolution layer 8, convolution layer 9 and a new feature scale fusion layer, and uses the DiFPN module proposed in the patent of the present invention to replace the Concat module of the YOLOv8n neck network; The new feature scale fusion layer is composed of a convolution layer 6, an upsampling layer 1, a DiFPN module 3, a C2f module 7, and a convolution layer 10 in sequence; The output of the SPPF module is used as the input of the convolutional layer 9, and the output of the convolutional layer 9 is transmitted to the top-down path. At the same time, the convolutional layer 9 is laterally connected to the DiFPN module 6 in the bottom-up path; The output of the C2f module 3 is used as the input of the convolutional layer 8, and the output of the convolutional layer 8 is used as the input of the DiFPN module 2. At the same time, the convolutional layer 8 is laterally connected to the DiFPN module 5 in the bottom-up path; The output of the C2f module 2 is used as the input of the convolutional layer 7, and the output of the convolutional layer 7 is used as the input of the DiFPN module 1. At the same time, the convolutional layer 7 is laterally connected to the DiFPN module 4 in the bottom-up path; The output of the C2f module 1 is used as the input of the convolution layer 6, and the convolution layer 6 is laterally connected to the DiFPN module 3 in the bottom-up path. At the same time, the output of the upsampling 1 is also used as the input of the DiFPN module 3; the output of the DiFPN module 3 is used as the input of the C2f module 7, and the output of the C2f module 7 is used as the input of the convolution layer 10, and the output of the convolution layer 10 is used as the input of the DiFPN module 4; Step 3.3: The head network includes 4 detection heads; the output of C2f module 10 is used as the input of detection head 4; the output of C2f module 9 is used as the input of detection head 3; the output of C2f module 8 is used as the input of detection head 2; the output of C2f module 7 is used as the input of detection head 1; Step 4: Using the training set and the validation set to train the crowded scene pedestrian detection model based on the improved YOLOv8n and saving the trained model parameters as the optimal model; Step 5: Use the test set to test the optimal model, evaluate the test results of the test set by objective evaluation indicators, meet the accuracy requirements, and obtain the final crowded scene pedestrian detection model based on the improved YOLOv8n.

[0008] Furthermore, in step 1, the images in the first data set may be collected and acquired through the Internet, photographed by a digital camera, or acquired from a surveillance video; Preferably, in step 2, the training set, validation set and test set can be divided in a ratio of 6:2:2.

[0009] Further, the DiFPN module 1, DiFPN module 2, DiFPN module 3 and DiFPN module 6 in step 3.2 have 2 inputs, and the DiFPN module with 2 inputs includes 2 DCNv4 modules and 1 weighted fusion module; DiFPN module 4 and DiFPN module 5 have 3 inputs, and the DiFPN module with 3 inputs includes 3 DCNv4 modules and 1 weighted fusion module; the internal process of the DiFPN module with 2 inputs and the DiFPN module with 3 inputs further includes steps 3.2.1 to 3.2.7: Step 3.2.1: The DiFPN module with two inputs, the input feature maps are feature map X0 and feature map X1; the feature map X0 and feature map X1 are respectively input into the two DCNv4 modules in the DiFPN module with two inputs; The DiFPN module with three inputs, the input feature maps are feature map X0, feature map X1 and feature map X2 respectively; the feature map X0, feature map X1 and feature map X2 are respectively input into the three DCNv4 modules in the DiFPN module with three inputs; Step 3.2.2: The DCNv4 module uses 1 convolutional layer to predict the offset of each pixel position in the input feature map; Step 3.2.3: Dynamically adjust the position of the sampling point according to the offset, and for each convolution kernel position (i, j), calculate the new sampling position ( , ): (1) In formula (1), offset(i,j) is the predicted offset; Step 3.2.4: For the DiFPN module with 2 inputs, feature map X0, feature map X1 and offset are used as inputs of the deformable convolution layer. Finally, 2 deformed feature maps are generated. and ; For the DiFPN module with 3 inputs, feature map X0, feature map X1, feature map X2 and offset are used as the input of the deformable convolution layer. Finally, 3 deformed feature maps are generated. , and ; Step 3.2.5: For the DiFPN module with two inputs, the two deformed feature maps are and Input to the weighted fusion module, according to and The importance of assigning weights W0 and W1 to them will Multiplying by its weight W0, Multiply it by its weight W1, and add the weighted feature maps to obtain a weighted sum. The calculation formula of the weighted sum is: (2) In formula (2), O is the output feature after weighted fusion, W i is the weight, Output features for DCNv4, i=0, 1; For the DiFPN module with 3 inputs, the 3 deformed feature maps are , and Input to the weighted fusion module, according to , and The importance of assigning weights W0, W1 and W2 to them. Multiplying by its weight W0, Multiply it by its weight W1, Multiply it by its weight W2, and add the weighted feature maps to obtain a weighted sum. The calculation formula of the weighted sum is: (3) In formula (3), O is the output feature after weighted fusion, W i is the weight, Output features for DCNv4, i=0, 1, 2; Step 3.2.6: Introduce nonlinear features into the weighted sum through the Swish activation function; Step 3.2.7: For the DiFPN module with 2 inputs, use a 1×1 convolution to operate the activated feature map to obtain an output feature map of the DiFPN module with 2 inputs; For the DiFPN module with 3 inputs, a 1×1 convolution is used to operate the activated feature map to obtain an output feature map of the DiFPN module with 3 inputs.

[0010] Furthermore, the feature fusion method of the horizontal span connection in step 3.2 is: For the i-th layer, the i-th layer feature map input to this layer is obtained according to the top-down and bottom-up feature propagation. and the i+1th layer feature map , the i+1th layer feature map Upsample and adjust to the same size as the feature map of the i-th layer , perform weighted fusion and normalization on the upsampled i+1th layer feature map and the i-th layer feature map to obtain the intermediate feature map of the i-th layer in the top-down path and the output feature map of the i-th layer in the bottom-up path; The feature fusion expression of the intermediate feature map of the i-th layer in the top-down path is: (4) In formula (4), ω1 represents the weight of the feature map of the i-th layer, ω2 represents the weight of the feature map of the i+1-th layer after upsampling, ε is a parameter used to prevent the denominator from being 0, and Conv represents feature fusion; The feature fusion expression of the output feature map of the i-th layer in the bottom-up path is: (5) In formula (5), represents the output feature map of the i-1th layer in the top-down path, It means that the output feature map of the i-1th layer is upsampled to the same size as the feature map of the i-th layer. represents the weight of the input i-th layer feature map, represents the weight of the intermediate feature map of the i-th layer in the bottom-up path, Represents the weight of the i-1th layer feature map after upsampling.

[0011] Furthermore, the training process of the crowded scene pedestrian detection model based on the improved YOLOv8n in step 4 specifically includes steps 4.1 to 4.4: Step 4.1: Setting the training parameters of the crowded scene pedestrian detection model based on the improved YOLOv8n; Model training parameters include: number of iterations, batch size, optimizer, learning rate, momentum, weight decay, and number of threads; Step 4.2: Input the training set and the validation set and the corresponding labels into the crowded scene pedestrian detection model based on the improved YOLOv8n, and use the back propagation algorithm to calculate the gradient of the loss function to the model parameters; Step 4.3: Use the optimizer to update the model parameters in the direction of gradient descent until the loss function of the training set and the validation set no longer decreases, and the evaluation indicators such as accuracy P, recall R, and mAP no longer improve; Step 4.4: Save the trained model parameters as the optimal model.

[0012] Furthermore, the step 5 specifically includes steps 5.1 to 5.3: Step 5.1: Input the test set into the optimal model described in step 4; Step 5.2: Calculate the model performance indicators: accuracy P, recall R, average precision mAP. The specific calculation formula is as follows: (6) (7) (8) (9) In formula (6) to formula (9), P is the precision, R is the recall, mAP is the average precision of all categories, AP is the average precision, m is the total number of pedestrian label categories, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples; Step 5.3: Evaluate the test results of the test set using objective evaluation indicators to meet the accuracy requirements, thus obtaining the final crowded scene pedestrian detection model based on the improved YOLOv8n.

[0013] Compared with the prior art, the present invention has the following beneficial effects: In the feature fusion network DCPAN disclosed in the patent of this invention, the horizontal spanning connection effectively retains and utilizes important detail information in the shallow layer, so that the model can significantly improve the sensitivity to small-scale pedestrian targets while maintaining good detection of large-scale pedestrian targets; The DiFPN module disclosed in the present invention can fully integrate feature information from different levels and enhance the network's adaptability to the shape and position of small-scale pedestrians and obscured pedestrians; In the feature fusion network DCPAN disclosed in the patent of this invention, four feature scale fusion layers are used to detect pedestrian targets of different scales. The strategy of using different feature scale fusion layers to process pedestrian targets of different scales is adopted to improve the detection accuracy of the model in detecting pedestrian targets of different scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of the pedestrian detection method in crowded scenes based on improved YOLOv8n of the present invention; Figure 2 This is a schematic diagram of the model structure of the pedestrian detection method in crowded scenes based on the improved YOLOv8n of the present invention; Figure 3 Schematic diagram of DiFPN structure with 2 inputs; Figure 4 Schematic diagram of DiFPN structure with 3 inputs; DETAILED DESCRIPTION

[0015] In order to make the technical solution, structural features, achieved purposes and advantages of the present invention more clearly understood, the present invention is further described in detail below in combination with specific implementation methods and accompanying drawings; it should be noted that the specific embodiments described herein are only used to explain the present invention more clearly and are not used to limit the present invention.

[0016] Figure 1 This is a flow chart of a pedestrian detection method in crowded scenes based on improved YOLOv8n disclosed in the present invention, and its implementation process is as follows: Step 1: Obtain pedestrian images in crowded scenes to form a first data set; in the first data set, the pedestrian images in crowded scenes can be collected and obtained through the Internet, photographed with a digital camera, or obtained from surveillance videos; In this embodiment, in order to better evaluate the detection effect of a crowded scene pedestrian detection method based on improved YOLOv8n disclosed in the present invention, a public data set CrowedHumen data set is used; the CrowedHumen data set is used to form the first data set; Step 2: adding annotation information to the images in the first data set to form a second data set, and dividing the second data set into a training set, a validation set, and a test set; Since the public data set CrowedHumen used in this embodiment already has annotation information, this step is skipped; the odgt format label file in the first data set described in this embodiment is converted into a txt format label file required by YOLOv8n to form a second data set, and the second data set is divided into a training set, a validation set and a test set according to 6:2:2; the divided training set includes 15,000 pictures, the validation set includes 4,370 pictures, and the test set includes 4,370 pictures. In this embodiment, the image resolution of the second data set is 640×640; Step 3: Build a crowded scene pedestrian detection model based on improved YOLOv8n. The improved YOLOv8n model structure is as follows: Figure 2 As shown, the model includes a backbone network, a feature fusion network DCPAN and a head network, and the construction of the model further includes steps 3.1 to 3.3: Step 3.1: The backbone network consists of a convolutional layer 1, a convolutional layer 2, a C2f module 1, a convolutional layer 3, a C2f module 2, a convolutional layer 4, a C2f module 3, a convolutional layer 5, a C2f module 4 and an SPPF module connected in sequence; Using the training set and the validation set in the second data set as input of the backbone network; The backbone network outputs feature information of four different scales through C2f module 1, C2f module 2, C2f module 3 and SPPF module respectively; Step 3.2: The feature fusion network DCPAN, based on the YOLOv8n neck network, adds convolution layer 7, convolution layer 8, convolution layer 9 and a new feature scale fusion layer, and uses the DiFPN module disclosed in the patent of the present invention to replace the Concat module of the YOLOv8n neck network; The new feature scale fusion layer is composed of a convolution layer 6, an upsampling layer 1, a DiFPN module 3, a C2f module 7, and a convolution layer 10 in sequence; The output of the SPPF module is used as the input of the convolutional layer 9, and the output of the convolutional layer 9 is transmitted to the top-down path. At the same time, the convolutional layer 9 is laterally connected to the DiFPN module 6 in the bottom-up path; The output of the C2f module 3 is used as the input of the convolutional layer 8, and the output of the convolutional layer 8 is used as the input of the DiFPN module 2. At the same time, the convolutional layer 8 is laterally connected to the DiFPN module 5 in the bottom-up path; The output of the C2f module 2 is used as the input of the convolutional layer 7, and the output of the convolutional layer 7 is used as the input of the DiFPN module 1. At the same time, the convolutional layer 7 is laterally connected to the DiFPN module 4 in the bottom-up path; The output of the C2f module 1 is used as the input of the convolution layer 6, and the convolution layer 6 is laterally connected to the DiFPN module 3 in the bottom-up path. At the same time, the output of the upsampling 1 is also used as the input of the DiFPN module 3; the output of the DiFPN module 3 is used as the input of the C2f module 7, and the output of the C2f module 7 is used as the input of the convolution layer 10, and the output of the convolution layer 10 is used as the input of the DiFPN module 4; Furthermore, the structure of the DiFPN module with 2 inputs is as follows Figure 3 As shown, the structure of the DiFPN module with 3 inputs is as follows Figure 4 As shown; the DiFPN module 1, DiFPN module 2, DiFPN module 3 and DiFPN module 6 in step 3.2 have 2 inputs, and the DiFPN module with 2 inputs includes 2 DCNv4 modules and 1 weighted fusion module; DiFPN module 4 and DiFPN module 5 have 3 inputs, and the DiFPN module with 3 inputs includes 3 DCNv4 modules and 1 weighted fusion module; the internal process of the DiFPN module with 2 inputs and the DiFPN module with 3 inputs further includes steps 3.2.1 to 3.2.7: Step 3.2.1: The DiFPN module with two inputs, the input feature maps are feature map X0 and feature map X1; the feature map X0 and feature map X1 are respectively input into the two DCNv4 modules in the DiFPN module with two inputs; The DiFPN module with three inputs, the input feature maps are feature map X0, feature map X1 and feature map X2 respectively; the feature map X0, feature map X1 and feature map X2 are respectively input into the three DCNv4 modules in the DiFPN module with three inputs; Step 3.2.2: The DCNv4 module uses 1 convolutional layer to predict the offset of each pixel position in the input feature map; Step 3.2.3: Dynamically adjust the position of the sampling point according to the offset, and for each convolution kernel position (i, j), calculate the new sampling position ( , ): (1) In formula (1), offset(i,j) is the predicted offset; Step 3.2.4: For the DiFPN module with 2 inputs, feature map X0, feature map X1 and offset are used as inputs of the deformable convolution layer. Finally, 2 deformed feature maps are generated. and ; For the DiFPN module with 3 inputs, feature map X0, feature map X1, feature map X2 and offset are used as the input of the deformable convolution layer. Finally, 3 deformed feature maps are generated. , and ; Step 3.2.5: For the DiFPN module with two inputs, the two deformed feature maps are and Input to the weighted fusion module, according to and The importance of assigning weights W0 and W1 to them will Multiplying by its weight W0, Multiply it by its weight W1, and add the weighted feature maps to obtain a weighted sum. The calculation formula of the weighted sum is: (2) In formula (2), O is the output feature after weighted fusion, W i is the weight, Output features for DCNv4, i=0, 1; For the DiFPN module with 3 inputs, the 3 deformed feature maps are , and Input to the weighted fusion module, according to , and The importance of assigning weights W0, W1 and W2 to them. Multiplying by its weight W0, Multiply it by its weight W1, Multiply it by its weight W2, and add the weighted feature maps to obtain a weighted sum. The calculation formula of the weighted sum is: (3) In formula (3), O is the output feature after weighted fusion, W i is the weight, Output features for DCNv4, i=0, 1, 2; Step 3.2.6: Introduce nonlinear features into the weighted sum through the Swish activation function; Step 3.2.7: For the DiFPN module with 2 inputs, use a 1×1 convolution to operate the activated feature map to obtain an output feature map of the DiFPN module with 2 inputs; For the DiFPN module with 3 inputs, a 1×1 convolutional layer is used to operate the activated feature map to obtain an output feature map of the DiFPN module with 3 inputs.

[0017] Furthermore, the horizontal span connection is Figure 2 As shown by the dotted line in , the feature fusion method of the horizontal span connection is: For the i-th layer, the i-th layer feature map input to this layer is obtained according to the top-down and bottom-up feature propagation. and the i+1th layer feature map , the i+1th layer feature map Upsample and adjust to the same size as the feature map of the i-th layer , perform weighted fusion and normalization on the upsampled i+1th layer feature map and the i-th layer feature map to obtain the intermediate feature map of the i-th layer in the top-down path and the output feature map of the i-th layer in the bottom-up path; The feature fusion expression of the intermediate feature map of the i-th layer in the top-down path is: (4) In formula (4), ω1 represents the weight of the feature map of the i-th layer, ω2 represents the weight of the feature map of the i+1-th layer after upsampling, ε is a parameter used to prevent the denominator from being 0, and Conv represents feature fusion; The feature fusion expression of the output feature map of the i-th layer in the bottom-up path is: (5) In formula (5), represents the output feature map of the i-1th layer in the top-down path, It means that the output feature map of the i-1th layer is upsampled to the same size as the feature map of the i-th layer. represents the weight of the input i-th layer feature map, represents the weight of the intermediate feature map of the i-th layer in the bottom-up path, Represents the weight of the i-1th layer feature map after upsampling.

[0018] Step 3.3: The head network includes 4 detection heads; the output of C2f module 10 is used as the input of detection head 4; the output of C2f module 9 is used as the input of detection head 3; the output of C2f module 8 is used as the input of detection head 2; the output of C2f module 7 is used as the input of detection head 1; Step 4: Use the training set and the validation set to train the crowded scene pedestrian detection model based on the improved YOLOv8n, and save the trained model as the optimal model; the crowded scene pedestrian detection model training process based on the improved YOLOv8n further includes steps 4.1 to 4.4: Step 4.1: Setting the training parameters of the crowded scene pedestrian detection model based on improved YOLOv8n; In this embodiment, the training parameters include: the iteration round Epoch is 300, the batch size batchsize is 4, the optimizer optimizer is SGD, the initial learning rate 1r0 is 0.01, the momentum momentum is 0.937, the weight decay weight_decay is 0.0005, and the number of workers is 4; Step 4.2: Input the training set and validation set images and corresponding labels into the crowded scene pedestrian detection model based on improved YOLOv8n, and use the back propagation algorithm to calculate the gradient of the loss function to the model parameters; the back propagation algorithm is an effective method for calculating gradients, which uses the chain rule to calculate the gradient of each parameter to the loss function; specifically, back propagation calculates the gradient of each parameter layer by layer by propagating the loss function backward from the output layer; in this process, the gradient of each parameter represents the rate of change of the loss function to the parameter, that is, how the loss function changes with the change of the parameter; by minimizing the loss function, the model parameters are adjusted to gradually approach the optimal solution; Step 4.3: After calculating the gradient of the model parameters, use the optimizer to update these parameters; the optimizer updates the parameters in the opposite direction of the gradient according to the gradient information of the parameters; the parameters with larger gradients are updated at a larger pace, while the parameters with smaller gradients are updated at a smaller pace; by continuously iteratively updating the model parameters, the value of the loss function can be gradually reduced; by minimizing the loss function, the values ​​of the model parameters are adjusted to gradually approach the optimal solution, that is, the parameter values ​​at which the loss function reaches the minimum value, the difference between the model's prediction results and the true value is minimized, and at the same time, evaluation indicators such as mAP, recall rate R, and precision rate P no longer improve; Step 4.4: Save the trained model parameters as the optimal model; Step 5: Use the test set to test the optimal model. The test result meets the accuracy requirement and the final model is obtained. Specifically, the step 5 further includes steps 5.1 to 5.3: Step 5.1: Input the test set into the optimal model described in step 4; Step 5.2: Calculate the model performance indicators: accuracy P, recall R, average precision mAP. The specific calculation formula is as follows: (6) (7) (8) (9) In formula (6) to formula (9), P is the precision, R is the recall, mAP is the average precision of all categories, AP is the average precision, m is the total number of pedestrian label categories, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples; Step 5.3: When the performance indicators meet the accuracy requirements, the final crowded scene pedestrian detection model based on the improved YOLOv8n is obtained.

[0019] In this embodiment, in order to verify the effect of the detection model disclosed in the patent of the present invention, this paper uses the YOLOv8n model, YOLOv8n model + BiFPN, YOLOv8n model + GlodFPN, YOLOv8n model + RepGFPN, YOLOv7-tiny model, YOLOv10n model, YOLOv11n model, GR-yolo and the detection model disclosed in the patent of the present invention to test on the CrowedHumen dataset. The evaluation results are shown in Table 1. The crowded scene pedestrian detection model based on the improved YOLOv8n proposed in this paper is superior to other comparison models in terms of accuracy P, recall R, mAP@0.5 and mAP@0.5:0.95 evaluation indicators:

[0020] The above description is only one embodiment of the present invention and does not limit the patent scope of the present invention. For those skilled in the art, the present invention may have various changes and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A pedestrian detection method in crowded scenes based on improved YOLOv8n, characterized in that: The specific steps include: Step 1: Obtain pedestrian images in a crowded scene to form a first data set; the images in the first data set can be collected through the Internet, photographed with a digital camera, or obtained from surveillance videos; Step 2: adding annotation information to the images in the first data set to form a second data set, and dividing the second data set into a training set, a validation set, and a test set; Step 3: Construct a crowded scene pedestrian detection model based on improved YOLOv8n, the model includes a backbone network, a feature fusion network DCPAN and a head network, and the construction of the model further includes steps 3.1 to 3.3: Step 3.1: The backbone network consists of a convolutional layer 1, a convolutional layer 2, a C2f module 1, a convolutional layer 3, a C2f module 2, a convolutional layer 4, a C2f module 3, a convolutional layer 5, a C2f module 4 and an SPPF module connected in sequence; Using the training set and the validation set in the second data set as input of the backbone network; The backbone network outputs feature information of four different scales through C2f module 1, C2f module 2, C2f module 3 and SPPF module respectively; Step 3.2: The feature fusion network DCPAN adds convolution layer 7, convolution layer 8, convolution layer 9 and a new feature scale fusion layer on the basis of the YOLOv8n neck network, and uses the DiFPN module disclosed in the patent of the present invention to replace the Concat module of the YOLOv8n neck network; The new feature scale fusion layer is composed of a convolution layer 6, an upsampling layer 1, a DiFPN module 3, a C2f module 7, and a convolution layer 10 in sequence; The output of the SPPF module is used as the input of the convolutional layer 9, and the output of the convolutional layer 9 is transmitted to the top-down path. At the same time, the convolutional layer 9 is laterally connected to the DiFPN module 6 in the bottom-up path; The output of the C2f module 3 is used as the input of the convolutional layer 8, and the output of the convolutional layer 8 is used as the input of the DiFPN module 2. At the same time, the convolutional layer 8 is laterally connected to the DiFPN module 5 in the bottom-up path; The output of the C2f module 2 is used as the input of the convolutional layer 7, and the output of the convolutional layer 7 is used as the input of the DiFPN module 1. At the same time, the convolutional layer 7 is laterally connected to the DiFPN module 4 in the bottom-up path; The output of the C2f module 1 is used as the input of the convolution layer 6, and the convolution layer 6 is laterally connected to the DiFPN module 3 in the bottom-up path. At the same time, the output of the upsampling 1 is also used as the input of the DiFPN module 3; the output of the DiFPN module 3 is used as the input of the C2f module 7, and the output of the C2f module 7 is used as the input of the convolution layer 10, and the output of the convolution layer 10 is used as the input of the DiFPN module 4; Step 3.3: The head network includes 4 detection heads; the output of C2f module 10 is used as the input of detection head 4; the output of C2f module 9 is used as the input of detection head 3; the output of C2f module 8 is used as the input of detection head 2; the output of C2f module 7 is used as the input of detection head 1; Step 4: Use the training set and the validation set to train the crowded scene pedestrian detection model based on the improved YOLOv8n, and save the trained model as the optimal model; the crowded scene pedestrian detection model training process based on the improved YOLOv8n further includes steps 4.1 to 4.4: Step 4.1: Setting the training parameters of the crowded scene pedestrian detection model based on improved YOLOv8n; Model training parameters include: number of iterations, batch size, optimizer, learning rate, momentum, weight decay, and number of threads; Step 4.2: Input the training set and the validation set and the corresponding labels into the crowded scene pedestrian detection model based on the improved YOLOv8n, and use the back propagation algorithm to calculate the gradient of the loss function to the model parameters; Step 4.3: Use the optimizer to update the model parameters in the direction of gradient descent until the loss function of the training set and the validation set no longer decreases, and the evaluation indicators such as accuracy P, recall R, and mAP no longer improve; Step 4.4: Save the trained model parameters as the optimal model; Step 5: Use the test set to test the optimal model, evaluate the test results of the test set by objective evaluation indicators, meet the accuracy requirements, and obtain the final crowded scene pedestrian detection model based on the improved YOLOv8n.

2. The pedestrian detection model in crowded scenes based on improved YOLOv8n according to claim 1, characterized in that: The DiFPN module 1, DiFPN module 2, DiFPN module 3 and DiFPN module 6 in step 3.2 have 2 inputs, and the DiFPN module with 2 inputs includes 2 DCNv4 modules and 1 weighted fusion module; DiFPN module 4 and DiFPN module 5 have 3 inputs, and the DiFPN module with 3 inputs includes 3 DCNv4 modules and 1 weighted fusion module; the internal process of the DiFPN module with 2 inputs and the DiFPN module with 3 inputs further includes steps 3.2.1 to 3.2.7: Step 3.2.1: The DiFPN module with two inputs, the input feature maps are feature map X0 and feature map X1; the feature map X0 and feature map X1 are respectively input into the two DCNv4 modules in the DiFPN module with two inputs; The DiFPN module with three inputs, the input feature maps are feature map X0, feature map X1 and feature map X2 respectively; the feature map X0, feature map X1 and feature map X2 are respectively input into the three DCNv4 modules in the DiFPN module with three inputs; Step 3.2.2: The DCNv4 module uses 1 convolutional layer to predict the offset of each pixel position in the input feature map; Step 3.2.3: Dynamically adjust the position of the sampling point according to the offset, and for each convolution kernel position (i, j), calculate the new sampling position ( , ): （1） In formula (1), offset(i,j) is the predicted offset; Step 3.2.4: For the DiFPN module with 2 inputs, feature map X0, feature map X1 and offset are used as inputs of the deformable convolution layer. Finally, 2 deformed feature maps are generated. and ; For the DiFPN module with 3 inputs, feature map X0, feature map X1, feature map X2 and offset are used as the input of the deformable convolution layer. Finally, 3 deformed feature maps are generated. , and ; Step 3.2.5: For the DiFPN module with two inputs, the two deformed feature maps are and Input to the weighted fusion module, according to and The importance of assigning weights W0 and W1 to them will Multiplying by its weight W0, Multiply it by its weight W1, and add the weighted feature maps to obtain a weighted sum. The calculation formula of the weighted sum is: （2） In formula (2), O is the output feature after weighted fusion, W i is the weight, Output features for DCNv4, i=0, 1; For the DiFPN module with 3 inputs, the 3 deformed feature maps are , and Input to the weighted fusion module, according to , and The importance of assigning weights W0, W1 and W2 to them. Multiplying by its weight W0, Multiply it by its weight W1, Multiply it by its weight W2, and add the weighted feature maps to obtain a weighted sum. The calculation formula of the weighted sum is: （3） In formula (3), O is the output feature after weighted fusion, W i is the weight, Output features for DCNv4, i=0, 1, 2; Step 3.2.6: Introduce nonlinear features into the weighted sum through the Swish activation function; Step 3.2.7: For the DiFPN module with 2 inputs, use a 1×1 convolutional layer to operate the activated feature map to obtain an output feature map of the DiFPN module with 2 inputs; For the DiFPN module with 3 inputs, a 1×1 convolutional layer is used to operate the activated feature map to obtain an output feature map of the DiFPN module with 3 inputs.

3. The pedestrian detection model for crowded scenes based on improved YOLOv8n according to claim 1, characterized in that: The feature fusion method for each horizontal span connection in step 3.2 is: For the i-th layer, the i-th layer feature map input to this layer is obtained according to the top-down and bottom-up feature propagation. and the i+1th layer feature map , the i+1th layer feature map Upsample and adjust to the same size as the feature map of the i-th layer , weightedly fuse and normalize the upsampled i+1th layer feature map and the i-th layer feature map to obtain the intermediate feature map of the i-th layer in the top-down path and the output feature map of the i-th layer in the bottom-up path; The feature fusion expression of the intermediate feature map of the i-th layer in the top-down path is: （4） In formula (4), ω1 represents the weight of the feature map of the i-th layer, ω2 represents the weight of the feature map of the i+1-th layer after upsampling, ε is a parameter used to prevent the denominator from being 0, and Conv represents feature fusion; The feature fusion expression of the output feature map of the i-th layer in the bottom-up path is: （5） In formula (5), represents the output feature map of the i-1th layer in the top-down path, It means that the output feature map of the i-1th layer is upsampled to the same size as the feature map of the i-th layer. represents the weight of the input i-th layer feature map, represents the weight of the intermediate feature map of the i-th layer in the bottom-up path, Represents the weight of the i-1th layer feature map after upsampling.

4. The method for pedestrian detection in crowded scenes based on improved YOLOv8n according to claim 1, characterized in that: In step 2, the second data set is divided into a training set, a validation set and a test set according to a preset ratio.

5. The method for pedestrian detection in crowded scenes based on improved YOLOv8n according to claim 1, characterized in that: The step 5 further includes steps 5.1 to 5.3: Step 5.1: Input the test set into the optimal model described in step 4; Step 5.2: Calculate the model performance indicators: accuracy P, recall R, average precision mAP. The specific calculation formula is as follows: （6） （7） （8） （9） In formula (6) to formula (9), P is the precision, R is the recall, mAP is the average precision of all categories, AP is the average precision, m is the total number of pedestrian label categories, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples; Step 5.3: Evaluate the test results of the test set using objective evaluation indicators to meet the accuracy requirements, thereby obtaining the final crowded scene pedestrian detection model based on the improved YOLOv8n.

Images