A lightweight crowded scene pedestrian detection method based on improved YOLOv11n

Abstract

The application discloses a kind of based on the lightweight crowded scene pedestrian detection method of improved YOLOv11n, specifically includes the following steps: (1) establishes crowded scene pedestrian image dataset;(2) add annotation information for the image in dataset;(3) construct the lightweight crowded scene pedestrian detection model based on improved YOLOv11n;(4) training set and verification set are used to train model and save the model trained;(5) test set is used to test model, the precision of model meets the generalization requirement, i.e. obtain the final lightweight crowded scene pedestrian detection model based on improved YOLOv11n.The application discloses a kind of based on the lightweight crowded scene pedestrian detection method of improved YOLOv11n compared to prior art, can effectively improve the precision of crowded scene pedestrian detection, simultaneously, make model have better lightweight characteristics, be convenient for deployment in resource limited mobile hardware platform.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a lightweight crowded scene pedestrian detection method based on improved YOLOv11n. BACKGROUND

[0002] With the expansion of city size and the growth of population density, public safety issues have become increasingly prominent, especially in crowded scenes such as transportation hubs, commercial centers, and large event venues, where congestion, stampedes, and other public safety incidents occur frequently. This not only poses a serious threat to the safety of citizens' lives and property, but also puts higher demands on social order maintenance. However, traditional manual inspection methods have obvious limitations in crowded scenes, not only time-consuming and labor-intensive, but also difficult to achieve comprehensive, full-process, and real-time inspection management. With the rapid development of intelligent and digital technologies, computer vision-based target detection technology has gradually become an important means of monitoring crowded scenes. This technology, by combining advanced monitoring equipment, has achieved real-time monitoring of crowded scenes, significantly improving the efficiency and accuracy of public safety management, and promoting the development of public safety management towards intelligence and refinement.

[0003] Currently, pedestrian detection, as an important means of safety monitoring in crowded scenes, is mainly divided into two categories: traditional pedestrian detection methods and deep learning-based pedestrian detection methods. Traditional pedestrian detection methods use manual feature extraction to train classifiers for distinguishing pedestrians from backgrounds. For example, the Histogram of Oriented Gradients (HOG) is used to extract features, and then the Support Vector Machine (SVM) is used to classify and detect features. Although traditional methods have achieved certain results in pedestrian detection, they have weak feature extraction capabilities and poor generalization.

[0004] In recent years, deep learning methods have gradually become the mainstream of pedestrian detection. Deep learning-based target detection algorithms can be roughly divided into two categories: two-stage (two-stage) target detection algorithms and one-stage (one-stage) target detection algorithms. The two-stage target detection algorithm is mainly the Region Convolutional Neural Network (R-CNN) series. The two-stage algorithm consists of two steps: first, generate candidate boxes, then classify and regress. Although this method has good accuracy, it requires positioning the target first and then classifying the target, which results in slow detection speed due to the complex process of the two stages. In contrast, the one-stage target detection algorithm combines target detection and classification tasks, has high detection accuracy and fast detection speed, and dominates the target detection task, such as the YOLO series.

[0005] However, in a crowded scene, the occlusion phenomenon between pedestrians leads to the loss of feature information, affecting the detection accuracy, and the traditional feature extraction method is difficult to effectively process the occlusion area. Due to the diversity of occlusion and shooting angle, there is a significant scale difference between the pedestrian target, and the existing feature fusion method has limitations in processing multi-scale features. The existing detection model has high computational complexity when processing large-scale pedestrian targets, resulting in low inference efficiency and difficulty in applying to edge devices and real-time scenes. Therefore, it is necessary to propose a pedestrian detection method that can effectively deal with the challenges of occlusion, multi-scale targets and the difficulty of deploying existing models in crowded scenes. SUMMARY

[0006] In view of the problems in the prior art, the present application provides a lightweight crowded scene pedestrian detection method based on improved YOLOv11n, which improves the YOLOv11n target detection model to solve the problem of false detection and missed detection of crowded scene pedestrian detection model caused by serious occlusion between pedestrians, diverse scale changes and large amount of calculation in crowded scenes, and at the same time, reduces the volume and parameter amount of the crowded scene pedestrian detection model, so that it can be deployed in resource-limited practical application scenarios.

[0007] The technical scheme provided by the present application comprises the following steps:

[0008] Step 1: Establish a crowded scene pedestrian image dataset to form a first dataset;

[0009] Step 2: Add annotation information to the images in the first dataset to form a second dataset, and divide the second dataset into a training set, a validation set and a test set;

[0010] Step 3: Construct a lightweight crowded scene pedestrian detection model based on improved YOLOv11n; the construction of the model further comprises steps 3.1 to 3.3:

[0011] Step 3.1: Construct an improved backbone network of the lightweight crowded scene pedestrian detection model based on improved YOLOv11n, which is composed of convolution layer 1, convolution layer 2, C3k2_EMBC module 1, convolution layer 3, C3k2_EMBC module 2, convolution layer 4, C3k2_EMBC module 3, convolution layer 5, C3k2_EMBC module 4, SPPF module and C2PSA module 1 connected in sequence;

[0012] The improved backbone network outputs pedestrian feature information of four different scales through C3k2_EMBC module 1, C3k2_EMBC module 2, C3k2_EMBC module 3 and C2PSA module 1 respectively;

[0013] Step 3.2: Constructing a light feature fusion network LDEFPN of a lightweight crowded scene pedestrian detection model based on an improved YOLOv11n;

[0014] The light feature fusion network LDEFPN specifically includes 8 convolutional layers, 6 Enhance_FPN modules, 2 DySample modules, and 5 C3k2 modules;

[0015] The output of the C3k2_EMBC module 1 is input to the convolutional layer 6, and the output of the convolutional layer 6 is input to the Enhance_FPN module 3;

[0016] The output of the C3k2_EMBC module 2 is input to the convolutional layer 7, and the output of the convolutional layer 7 is input to the Enhance_FPN module 3; the output of the convolutional layer 7 is input to the Enhance_FPN module 1, the output of the Enhance_FPN module 1 is input to the C3k2 module 2, the output of the C3k2 module 2 is input to the Enhance_FPN module 3, the output of the Enhance_FPN module 3 is input to the C3k2 module 3, and the output of the C3k2 module 3 is input to the convolutional layer 10;

[0017] The output of the C3k2_EMBC module 3 is input to the convolutional layer 8, and the output of the convolutional layer 8 is input to the Enhance_FPN module 2; the output of the Enhance_FPN module 2 is input to the C3k2 module 1, and the output of the C3k2 module 1 is fused with the output of the convolutional layer 7 in the Enhance_FPN module 1 through the DySample module 1; the outputs of the convolutional layer 8, the C3k2 module 1, and the convolutional layer 10 are fused in the Enhance_FPN module 4, the output of the Enhance_FPN module 4 is input to the C3k2 module 4, and the output of the C3k2 module 4 is input to the convolutional layer 11;

[0018] The output of the C2PSA module 1 is input to the convolutional layer 9, and the output of the convolutional layer 9 is fused with the output of the convolutional layer 8 in the Enhance_FPN module 2 through the DySample module 2; the output of the convolutional layer 9 and the output of the convolutional layer 11 are fused in the Enhance_FPN module 5, the output of the Enhance_FPN module 5 is input to the C3k2 module 5, the output of the C3k2 module 5 is input to the convolutional layer 13, the output of the convolutional layer 9 is input to the convolutional layer 12, and the output of the convolutional layer 12 and the output of the convolutional layer 13 are fused in the Enhance_FPN module 6;

[0019] Step 3.3: Constructing an improved head network of the lightweight crowded scene pedestrian detection model based on the improved YOLOv11n, the improved head network comprising 4 detection heads;

[0020] The output of the C3k2 module 3 is taken as the input of the detection head 1, the output of the C3k2 module 4 is taken as the input of the detection head 2, the output of the C3k2 module 5 is taken as the input of the detection head 3, and the output of the Enhance_FPN module 6 is taken as the output of the detection head 4;

[0021] Step 4: Training the lightweight crowded scene pedestrian detection model based on the improved YOLOv11n in step 3 using the training set and the validation set, and saving the trained model, the step 4 further comprising steps 4.1 to 4.4:

[0022] Step 4.1: Setting the training parameters of the lightweight crowded scene pedestrian detection model based on the improved YOLOv11n, the training parameters including: the number of iterations, the batch size, the optimizer, the learning rate, the momentum, the weight decay and the number of threads;

[0023] Step 4.2: Inputting the training set and validation set images and corresponding labels into the lightweight crowded scene pedestrian detection model based on the improved YOLOv11n, using the back propagation algorithm to calculate the gradient of the loss function on the model parameters, and updating the parameters of the model according to the gradient to gradually reduce the loss function;

[0024] Step 4.3: Monitoring the loss function value and performance indicators during training, when the loss function of the training set and the validation set no longer decreases, and the evaluation indicators such as the accuracy P, the recall rate R, the average precision mean mAP no longer improve, stop training to avoid overfitting;

[0025] Step 4.4: After training is completed, saving the trained model and selecting the optimal model from it;

[0026] Step 5: Testing the optimal model obtained by training using the crowded scene pedestrian test set, and obtaining the final lightweight crowded scene pedestrian detection model based on the improved YOLOv11n.

[0027] Further, in step 1, the images in the first data set can be collected through the network, photographed by a digital camera, or obtained from a monitoring video;

[0028] Preferably, in step 2, the LableImg labeling tool can be used to add annotation information to pedestrians; the training set, the validation set and the test set can be divided according to the ratio of 6:2:2.

[0029] Further, the C3k2 EMBC module in step 3.1 replaces the Bottleneck module in the original C3k2 module with an EMB convolution when C3k EMBC=False, and replaces the C3k module in the original C3k2 with the C3k EMBC module when C3k EMBC=Ture. The C3k EMBC module replaces the Bottleneck module in the C3k module with an EMB convolution;

[0030] The EMB convolution includes two convolution layers, a depth separable convolution, an eSE module and a Droupout module;

[0031] The feature map input into the EMB convolution first passes through the convolution layer 1, the feature map output by the convolution layer 1 passes through normalization and a Swish activation function, and is input into the depth separable convolution, the feature map output by the depth separable convolution passes through normalization and a Swish activation function, and is input into the eSE module, the feature map output by the eSE module is input into the convolution layer 2, the feature map output by the convolution layer 2 passes through normalization and is input into the Droupout module, the feature map output by the Droupout module is aggregated with the original input feature map, and the aggregated feature map is the output feature map of the EMB convolution.

[0032] Further, the Enhance_FPN module in step 3.2 includes a weighted fusion module and a lightweight spatial attention mechanism; the Enhance_FPN module 1, the Enhance_FPN module 2, the Enhance_FPN module 5 and the Enhance_FPN module 6 have two inputs, and the Enhance_FPN module 3 and the Enhance_FPN module 4 have three inputs; the internal process of the Enhance_FPN module with two inputs and the Enhance_FPN module with three inputs further includes steps 3.2.1 to 3.2.5:

[0033] Step 3.2.1: the Enhance_FPN module with two inputs has feature maps X0 and X1 as input feature maps, and the Enhance_FPN module with three inputs has feature maps X0, X1 and X2 as input feature maps;

[0034] Step 3.2.2: for the Enhance_FPN module with two inputs, weights W0 and W1 are assigned to X0 and X1 according to their importance, X0 is multiplied by the weight W0, X1 is multiplied by the weight W1, and the weighted feature maps are added to obtain a weighted sum, and the calculation formula of the weighted sum is:

[0035] (1)

[0036] In formula (1), O is the output feature after weighted summation, X0 and X1 are input feature maps of the Enhance_FPN module with 2 inputs respectively, W0 is the weight of the feature map X0, and W1 is the weight of the feature map X1;

[0037] For the Enhance_FPN module with 3 inputs, weights W0, W1 and W2 are assigned to X0, X1 and X2 according to their importance, X0 is multiplied by its weight W0, X1 is multiplied by its weight W1, X2 is multiplied by its weight W2, and the weighted feature maps are added to obtain the weighted sum, and the calculation formula of the weighted sum is:

[0038] (2)

[0039] In formula (2), O is the output feature after weighted summation, X0, X1 and X2 are input feature maps of the Enhance_FPN module with 3 inputs respectively, W0 is the weight of the feature map X0, W1 is the weight of the feature map X1, and W2 is the weight of the feature map X2;

[0040] Step 3.2.3: introducing the weighted sum into the non-linear feature through the Swish activation function, and the calculation formula of the Swish activation function processing is:

[0041] (3)

[0042] In formula (3), O is the output feature after weighted summation, Swish is the Swish activation function, and fused_feature is the feature map processed by the Swish activation function;

[0043] Step 3.2.4: taking the feature map fused_feature processed by the Swish activation function as the input feature map of the convolution layer in the lightweight spatial attention mechanism, and the output feature map LSA of the convolution layer is processed by the Sigmoid activation function, and the calculation formula of the Sigmoid activation function processing is:

[0044] (4)

[0045] In formula (4), LSA is the output feature map of the convolution layer in the lightweight spatial attention mechanism, Sigmoid is the Sigmoid activation function, and attention_map is the feature map processed by the Sigmoid activation function;

[0046] Step 3.2.5: Element-wise multiplication is performed between the feature map attention_map processed by the Sigmoid activation function and the feature map fused_feature processed by the Swish activation function, and the feature map obtained after element-wise multiplication is taken as the final output feature map of the Enhance_FPN module. The specific calculation formula of element-wise multiplication is as follows:

[0047] (5)

[0048] In formula (5), fused_feature is the feature map processed by the Swish activation function, attention_map is the feature map processed by the Sigmoid activation function, and Enhance_feature is the final output feature map of the Enhance_FPN module.

[0049]

[0050] Further, each DySample module in step 3.2 selects to first use the DySample up-sampling with added static range factor to reconstruct a high-resolution feature map from the input feature map, or to first use the DySample up-sampling with added dynamic range factor to reconstruct a high-resolution feature map from the input feature map before the fusion operation of the lightweight feature fusion network LDEFPN.

[0051] In which, before the fusion operation of the lightweight feature fusion network LDEFPN, the DySample up-sampling with added static range factor is first used to reconstruct a high-resolution feature map from the input feature map, which is expressed as:

[0052] (6)

[0053] (7)

[0054] (8)

[0055] In formula (6) to formula (8), x represents the input feature map, s represents the up-sampling ratio factor, grid_sample represents the grid sampling operation, represents the reconstructed high-resolution feature map, o represents the offset, g represents the original sampling grid, and linear represents the linear layer operation.

[0056] In which, before the fusion operation of the lightweight feature fusion network LDEFPN, the DySample up-sampling with added dynamic range factor is first used to reconstruct a high-resolution feature map from the input feature map, which is expressed as:

[0057] ​(9)

[0058] (10)

[0059] (11)

[0060] In the formula (9) to the formula (11), x represents an input feature map, s represents an up-sampling ratio factor; grid_sample represents a grid sampling operation, represents a reconstructed high-resolution feature map; o represents an offset, g represents an original sampling grid; linear represents a linear layer operation, and sigmoid is a sigmoid activation function;

[0061] Further, the step 5 specifically comprises a step 5.1 to a step 5.3.

[0062] The step 5.1: inputting the test set into the optimal model in the step 4.4 to perform testing.

[0063] The step 5.2: calculating model performance indexes: an accuracy P, a recall R, an average precision mean mAP, a parameter quantity, a calculation complexity GFLOPs, a frame per second FPS and a model size, and specific calculation formulas of the accuracy P, the recall R and the average precision mean mAP are as follows:

[0064] (12)

[0065] (13)

[0066] (14)

[0067] (15)

[0068] In the formula (12) to the formula (15), P is the accuracy, R is the recall, mAP is the average precision mean of all classes, AP is the average precision, m is the total number of pedestrian label categories, TP represents the number of positive samples correctly identified as positive samples, FP represents the number of negative samples incorrectly identified as positive samples, FN represents the number of positive samples incorrectly identified as negative samples, AP i is the AP of the i-th pedestrian, and P(R) is a function of P with respect to R;

[0069] The step 5.3: evaluating the performance indexes of the test set, if the accuracy of the test set is similar to that of the training set, it is indicated that the model meets the generalization requirement, and a final lightweight crowded scene pedestrian detection model based on the improved YOLOv11n is obtained.

[0070] Compared with the prior art, the beneficial effects of the present application are:

[0071] The application discloses a lightweight crowded scene pedestrian detection method based on improved YOLOv11n.

[0072] The method introduces a lightweight feature fusion network LDEFPN, improves the detection performance of multi-scale pedestrian targets and occluded pedestrian targets, and reduces the calculation overhead. BRIEF DESCRIPTION OF DRAWINGS

[0073] Figure 1 A flowchart of the lightweight crowded scene pedestrian detection method based on improved YOLOv11n of the application is shown in the figure.

[0074] Figure 2 A structure diagram of the lightweight crowded scene pedestrian detection model based on improved YOLOv11n of the application is shown in the figure.

[0075] Figure 3 A structure diagram of the C3k2_EMBC module (C3k2_EMBC=False) is shown in the figure.

[0076] Figure 4 A structure diagram of the C3k2_EMBC module (C3k2_EMBC=True) is shown in the figure.

[0077] Figure 5 A structure diagram of the C3k_EMBC module is shown in the figure.

[0078] Figure 6 A structure diagram of the EMB convolution is shown in the figure.

[0079] Figure 7 A structure diagram of the Enhance_FPN module with two inputs is shown in the figure.

[0080] Figure 8 A structure diagram of the Enhance_FPN module with three inputs is shown in the figure.

[0081] Figure 9 A structure diagram of the DySample module is shown in the figure, wherein (a) is a structure diagram of the DySample module as a whole, (b) is a structure diagram of a sampling point generator of the DySample module as a static range factor, and (c) is a structure diagram of a sampling point generator of the DySample module as a dynamic range factor. DETAILED DESCRIPTION

[0082] In order to make the technical solutions, structural features, achieved purposes and advantages of the present application clearer and more comprehensible, the present application will be further described in detail below in conjunction with specific embodiments and the accompanying drawings. It should be noted that the specific embodiments described herein are only used to more clearly explain the present application and do not limit the present application.

[0083] Figure 1 is a flowchart of the lightweight crowded scene pedestrian detection method based on improved YOLOv11n of the present application, and the implementation process is as follows:

[0084] Step 1: Establish a crowded scene pedestrian image dataset to form a first dataset; the images in the first dataset can be collected through a network, photographed by a digital camera, or obtained from a monitoring video.

[0085] In this embodiment, in order to better evaluate the detection effect of the lightweight crowded scene pedestrian detection method based on improved YOLOv11n disclosed in the present application, the public dataset CrowedHumen dataset is adopted; the first dataset is formed by using the CrowedHumen dataset.

[0086] Step 2: Add annotation information to the images in the first dataset to form a second dataset, and divide the second dataset into a training set, a validation set and a test set.

[0087] Preferably, the LableImg annotation tool is used to add annotation information to the pedestrians in the images of the first dataset; since the public dataset CrowedHumen adopted in this embodiment already has annotation information, this step is omitted.

[0088] The odgt format label file in the first dataset in this embodiment is converted into a txt format label file required by YOLOv11n to form a second dataset.

[0089] Preferably, the second dataset is divided into a training set, a validation set and a test set according to the ratio of 6:2:2; the training set after division includes 15000 images, the validation set includes 4370 images, and the test set includes 4370 images. The image resolution in the second dataset in this embodiment is 640x640.

[0090] Step 3: Construct a lightweight crowded scene pedestrian detection model based on improved YOLOv11n; the improved YOLOv11n model structure is as shown in Figure 2 The model construction process specifically includes steps 3.1 to 3.3:

[0091] Step 3.1: constructing an improved backbone network of the lightweight crowded scene pedestrian detection model based on the improved YOLOv11n, the improved backbone network being composed of a convolutional layer 1, a convolutional layer 2, a C3k2_EMBC module 1, a convolutional layer 3, a C3k2_EMBC module 2, a convolutional layer 4, a C3k2_EMBC module 3, a convolutional layer 5, a C3k2_EMBC module 4, a SPPF module and a C2PSA module 1 connected in sequence;

[0092] The improved backbone network outputs feature information of four different scales through the C3k2_EMBC module 1, the C3k2_EMBC module 2, the C3k2_EMBC module 3 and the C2PSA module 1 respectively.

[0093] Further, the structure of the C3k2_EMBC module (C3k_EMBC=False) is as shown in Figure 3 The structure of the C3k2_EMBC module (C3k_EMBC=True) is as shown in Figure 4 The structure of the C3k_EMBC module is as shown in Figure 5 When C3k_EMBC=False, the C3k2_EMBC module replaces the Bottleneck module in the original C3k2 module with an EMB convolution, and when C3k_EMBC=Ture, the C3k2_EMBC module replaces the C3k module in the original C3k2 with the C3k_EMBC module; the C3k_EMBC module replaces the Bottleneck module in the C3k module with an EMB convolution.

[0094] Further, the structure of the EMB convolution is as shown in Figure 6 The EMB convolution includes two convolutional layers, a depth separable convolution, an eSE module and a Droupout module.

[0095] The feature map input into the EMB convolution first passes through the convolutional layer 1, the feature map output by the convolutional layer 1 passes through normalization and a Swish activation function and is input into the depth separable convolution, the feature map output by the depth separable convolution passes through normalization and a Swish activation function and is input into the eSE module, the feature map output by the eSE module is input into the convolutional layer 2, the feature map output by the convolutional layer 2 is normalized and input into the Droupout module, the feature map output by the Droupout module is aggregated with the original input feature map, and the aggregated feature map is the output feature map of the EMB convolution.

[0096] Step 3.2: constructing a lightweight feature fusion network LDEFPN of the lightweight crowded scene pedestrian detection model based on the improved YOLOv11n;

[0097] The light feature fusion network LDEFPN specifically includes 8 convolutional layers, 6 Enhance_FPN modules, 2 DySample modules and 5 C3k2 modules;

[0098] The output of the C3k2_EMBC module 1 is input to the convolutional layer 6, and the output of the convolutional layer 6 is input to the Enhance_FPN module 3;

[0099] The output of the C3k2_EMBC module 2 is input to the convolutional layer 7, and the output of the convolutional layer 7 is input to the Enhance_FPN module 3; meanwhile, the output of the convolutional layer 7 is input to the Enhance_FPN module 1, the output of the Enhance_FPN module 1 is input to the C3k2 module 2, the output of the C3k2 module 2 is input to the Enhance_FPN module 3, the output of the Enhance_FPN module 3 is input to the C3k2 module 3, and the output of the C3k2 module 3 is input to the convolutional layer 10;

[0100] The output of the C3k2_EMBC module 3 is input to the convolutional layer 8, and the output of the convolutional layer 8 is input to the Enhance_FPN module 2; the output of the Enhance_FPN module 2 is input to the C3k2 module 1, and the output of the C3k2 module 1 is fused with the output of the convolutional layer 7 in the Enhance_FPN module 1 through the DySample module 1; the outputs of the convolutional layer 8, the C3k2 module 1 and the convolutional layer 10 are fused in the Enhance_FPN module 4, the output of the Enhance_FPN module 4 is input to the C3k2 module 4, and the output of the C3k2 module 4 is input to the convolutional layer 11;

[0101] The output of the C2PSA module 1 is input to the convolutional layer 9, and the output of the convolutional layer 9 is fused with the output of the convolutional layer 8 in the Enhance_FPN module 2 through the DySample module 2; the output of the convolutional layer 9 and the output of the convolutional layer 11 are fused in the Enhance_FPN module 5, the output of the Enhance_FPN module 5 is input to the C3k2 module 5, the output of the C3k2 module 5 is input to the convolutional layer 13, the output of the convolutional layer 9 is input to the convolutional layer 12, and the output of the convolutional layer 12 and the output of the convolutional layer 13 are fused in the Enhance_FPN module 6;

[0102] Further, the structure of the Enhance_FPN module with 2 inputs is as shown in Figure 7 The structure of the Enhance_FPN module with 3 inputs is as shown inFigure 8 The Enhance_FPN module includes a weighted fusion module and a lightweight spatial attention mechanism; the Enhance_FPN module 1, the Enhance_FPN module 2, the Enhance_FPN module 5 and the Enhance_FPN module 6 have two inputs, the Enhance_FPN module 3 and the Enhance_FPN module 4 have three inputs; the internal processes of the Enhance_FPN module with two inputs and the Enhance_FPN module with three inputs further include steps 3.2.1 to 3.2.5:

[0103] Step 3.2.1: The Enhance_FPN module with two inputs inputs feature maps are feature map X0 and feature map X1 respectively; the Enhance_FPN module with three inputs inputs feature maps are feature map X0, feature map X1 and feature map X2 respectively;

[0104] Step 3.2.2: For the Enhance_FPN module with two inputs, according to the importance of X0 and X1, weights W0 and W1 are assigned to it, X0 is multiplied by its weight W0, X1 is multiplied by its weight W1, and the weighted feature maps are added to obtain a weighted sum, and the specific calculation formula of the weighted sum is:

[0105] (1)

[0106] In formula (1), O is the output feature after weighted summation, X0 and X1 are input feature maps of the Enhance_FPN module with two inputs, W0 is the weight of feature map X0, and W1 is the weight of feature map X1;

[0107] For the Enhance_FPN module with three inputs, according to the importance of X0, X1 and X2, weights W0, W1 and W2 are assigned to it, X0 is multiplied by its weight W0, X1 is multiplied by its weight W1, X2 is multiplied by its weight W2, and the weighted feature maps are added to obtain a weighted sum, and the specific calculation formula of the weighted sum is:

[0108] (2)

[0109] In formula (2), O is the output feature after weighted summation, X0, X1 and X2 are input feature maps of the Enhance_FPN module with three inputs, W0 is the weight of feature map X0, W1 is the weight of feature map X1, and W2 is the weight of feature map X2;

[0110]

[0111] ​Step 3.2.3: The weighted sum is introduced into the non-linear feature through the Swish activation function, and the calculation formula of the Swish activation function is:

[0112] (3)

[0113] In formula (3), O is the output feature after weighted sum, Swish is the Swish activation function, and fused_feature is the feature map processed by the Swish activation function;

[0114] Step 3.2.4: The feature map fused_feature processed by the Swish activation function is taken as the input feature map of the convolution layer in the lightweight spatial attention mechanism, and the output feature map LSA of the convolution layer is processed by the Sigmoid activation function, and the calculation formula of the Sigmoid activation function is:

[0115] (4)

[0116] In formula (4), LSA is the output feature map of the convolution layer in the lightweight spatial attention mechanism, Sigmoid is the Sigmoid activation function, and attention_map is the feature map processed by the Sigmoid activation function;

[0117] Step 3.2.5: The feature map attention_map processed by the Sigmoid activation function is multiplied element by element with the feature map fused_feature processed by the Swish activation function, and the feature map after element-by-element multiplication is taken as the final output feature map of the Enhance_FPN module, and the specific calculation formula of the element-by-element multiplication is:

[0118] (5)

[0119] In formula (5), fused_feature is the feature map processed by the Swish activation function, and attention_map is the feature map processed by the Sigmoid activation function; and Enhance_feature is the final output feature map of the Enhance_FPN module.

[0120] Further, the structure of DySample is as follows: Figure 9As shown, wherein (a) is a schematic diagram of the overall structure of DySample, (b) is a schematic diagram of a sampling point generator of DySample as a static range factor, (c) is a schematic diagram of a sampling point generator of DySample as a dynamic range factor; each DySample module in step 3.2 selects to first use DySample up-sampling with added static range factor to reconstruct a high-resolution feature map for the input feature map before the fusion operation of the lightweight feature fusion network LDEFPN, or selects to first use DySample up-sampling with added dynamic range factor to reconstruct a high-resolution feature map for the input feature map;

[0121] wherein, before the fusion operation of the lightweight feature fusion network LDEFPN, DySample up-sampling with added static range factor is first used to reconstruct a high-resolution feature map for the input feature map, denoted as:

[0122] (6)

[0123] (7)

[0124] (8)

[0125] In formulas (6) to (8), x represents an input feature map, s represents an up-sampling scale factor; grid_sample represents a grid sampling operation, represents a reconstructed high-resolution feature map; o represents an offset, g represents an original sampling grid; linear represents a linear layer operation;

[0126] wherein, before the fusion operation of the lightweight feature fusion network LDEFPN, DySample up-sampling with added dynamic range factor is first used to reconstruct a high-resolution feature map for the input feature map, denoted as:

[0127] (9)

[0128] (10)

[0129] (11)

[0130] In formulas (9) to (11), x represents an input feature map, s represents an up-sampling scale factor, grid_sample represents a grid sampling operation, represents a reconstructed high-resolution feature map, o represents an offset, g represents an original sampling grid; linear represents a linear layer operation, sigmoid is a sigmoid activation function;

[0131] Step 3.3: Constructing an improved head network based on the improved YOLOv11n lightweight crowded scene pedestrian detection model, which includes 4 detection heads;

[0132] The output of the C3k2 module 3 is input to the detection head 1, the output of the C3k2 module 4 is input to the detection head 2, the output of the C3k2 module 5 is input to the detection head 3, and the output of the Enhance_FPN module 6 is input to the detection head 4;

[0133] Step 4: Input the training set and validation set into the improved model for training and save the trained model; the step 4 further includes steps 4.1 to 4.4:

[0134] Step 4.1: Set the training parameters of the improved YOLOv11n-based lightweight crowded scene pedestrian detection model, including: iteration rounds, batch size, optimizer, learning rate, momentum, weight decay and thread number;

[0135] In this embodiment, the training parameters include: iteration rounds Epoch is 300, batch size batchsize is 16, optimizer is SGD, initial learning rate 1r0 is 0.01, momentum is 0.937, weight decay weight_decay is 0.0005, and thread number workers is 8;

[0136] Step 4.2: Input the training set and validation set images and corresponding labels into the improved YOLOv11n-based lightweight crowded scene pedestrian detection model, and use the back propagation algorithm to calculate the gradient of the loss function to the model parameters.

[0137] Specifically, in each training iteration, first, the error between the predicted output of the model and the true label is calculated by forward propagation, and then the error is used to calculate the loss function.

[0138] Second, the gradient of the loss function to the model parameters is calculated using the back propagation algorithm.

[0139] Third, the parameters of the model are updated using the optimization algorithm.

[0140] Through repeated iterations of this process, the parameters of the model will gradually adjust to the position that minimizes the loss function, so that the model can more accurately detect pedestrians in crowded scenes.

[0141] Step 4.3: Monitor the loss function value and performance metrics during the training process. Performance metrics include accuracy P, recall R, and mean average precision mAP. When the loss function of the training set and validation set no longer decreases, and the evaluation metrics such as accuracy P, recall R, and mean average precision mAP no longer improve, stop training to avoid the model overfitting the training data and ensure the model's generalization ability on the test data.

[0142] Step 4.4: After training is complete, save the trained model and select the best model from it; ensure that the model can be used at any time for pedestrian detection tasks in crowded scenes, and can also be easily compared and evaluated with other models.

[0143] Step 5: Input the test set into the optimal model for testing. If the model's accuracy meets the generalization requirement, the final model is obtained. Step 5 further includes steps 5.1 to 5.3:

[0144] Step 5.1: Input the test set into the optimal model selected in step 4 for testing;

[0145] Step 5.2: Calculate model performance metrics: accuracy P, recall R, mean precision mAP, number of parameters, computational complexity GFLOPs, frames per second (FPS), and model size. The specific formulas for calculating accuracy P, recall R, and mean precision mAP are as follows:

[0146] (12)

[0147] (13)

[0148] (14)

[0149] (15)

[0150] In formulas (12) to (15), P is precision, R is recall, mAP is the mean precision of all classes, AP is the mean precision, m is the total number of pedestrian label classes, TP represents the number of positive samples correctly identified as positive samples, FP represents the number of negative samples incorrectly identified as positive samples, FN represents the number of positive samples incorrectly identified as negative samples, and AP i Let AP be the AP of the i-th type of pedestrian, and P(R) be a function of P with respect to R.

[0151] Step 5.3: Evaluate the performance metrics of the test set. If the accuracy of the test set is similar to that of the training set, it indicates that the model meets the generalization requirements, and the final lightweight pedestrian detection model for crowded scenes based on the improved YOLOv11n is obtained.

[0152] In this embodiment, in order to verify the effect of the improved YOLOv11n-based lightweight crowded scene pedestrian detection model disclosed in the application, the YOLOv5n model, the GS-YOLOv5 model, the YOLOv8n model, the YOLOv10n model and the YOLOv11n model and the improved YOLOv11n-based lightweight crowded scene pedestrian detection model disclosed in the application are tested on the CrowedHumen dataset, and the evaluation results are shown in Table 1.

[0153] Table 1 Comparison of experimental results

[0154] ;

[0155] According to the data in Table 1, it can be seen that the improved YOLOv11n-based lightweight crowded scene pedestrian detection model disclosed in the application has improved in the accuracy P, the recall rate R, the mAP@0.5, the mAP@0.5:0.95 and other indicators, indicating that the model has higher detection precision. At the same time, the improved model has lower performance indicators such as parameter quantity, model size and computational complexity GFLOPs compared with the original YOLOv11n model, and this improvement makes the improved YOLOv11n-based lightweight crowded scene pedestrian detection model disclosed in the application have better lightweight characteristics and be easier to deploy on mobile devices.

[0156] In this embodiment, in order to verify the effectiveness of the lightweight feature extraction network LDEFPN in the improved YOLOv11n-based lightweight crowded scene pedestrian detection model disclosed in the application, the YOLOv11n is used as the benchmark model, and the lightweight feature extraction network LDEFPN disclosed in the application is compared with several improved neck networks, including the neck network of YOLOv11n, BiFPN, HSFPN, AFPN, GSPAN and the lightweight feature extraction network LDEFPN disclosed in the application. The experimental results are shown in Table 2.

[0157] Table 2 Comparison of effectiveness of LDEFPN

[0158] ;

[0159] According to the data in Table 2, it can be seen that compared with other improved neck networks, the accuracy P, recall rate R, mAP@0.5, mAP@0.5:0.95, FPS and other indicators are improved by using the lightweight feature extraction network LDEFPN instead of the original neck network in YOLOv11n, indicating that the lightweight feature extraction network LDEFPN disclosed in the application has higher detection speed and accuracy. At the same time, the performance indicators such as parameter quantity, model size and computational complexity GFLOPs are lower, indicating that the lightweight feature extraction network LDEFPN disclosed in the application has better lightweight characteristics and is easier to deploy on mobile devices.

[0160] The above only describes one embodiment of the application, and does not limit the protection scope of the application. For those skilled in the art, the application can have various modifications and changes, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A lightweight pedestrian detection method for crowded scenes based on improved YOLOv11n, characterized in that, Specifically, the following steps are included: Step 1: Create a dataset of pedestrian images in crowded scenes to form the first dataset; In the first dataset, pedestrian images in crowded scenes can be collected and obtained through the network, captured by digital cameras, or obtained from surveillance videos; Step 2: Add annotation information to the images in the first dataset to form the second dataset, and divide the second dataset into a training set, a validation set, and a test set in a 6:2:2 ratio; Step 3: Construct a lightweight pedestrian detection model for congested scenes based on the improved YOLOv11n. The construction of the model further includes steps 3.1 to 3.3: Step 3.1: Construct an improved backbone network for a lightweight pedestrian detection model in crowded scenes based on the improved YOLOv11n. The improved backbone network consists of convolutional layer 1, convolutional layer 2, C3k2_EMBC module 1, convolutional layer 3, C3k2_EMBC module 2, convolutional layer 4, C3k2_EMBC module 3, convolutional layer 5, C3k2_EMBC module 4, SPPF module, and C2PSA module 1 connected in sequence. The improved backbone network outputs feature information at four different scales through C3k2_EMBC module 1, C3k2_EMBC module 2, C3k2_EMBC module 3 and C2PSA module 1, respectively. The C3k2_EMBC module, when C3k_EMBC=False, replaces the Bottleneck module in the original C3k2 module with EMB convolution, and when C3k_EMBC=True, replaces the C3k module in the original C3k2 with the C3k_EMBC module. The C3k_EMBC module is a C3k module in which the Bottleneck module is replaced with an EMB convolution. The EMB convolution includes two convolutional layers, a depthwise separable convolution, an eSE module, and a Dropout module. The feature map input to the EMB convolution first passes through convolutional layer 1. The feature map output from convolutional layer 1 is normalized and activated by the Swish activation function before being input into a depthwise separable convolution. The feature map output from the depthwise separable convolution is normalized and activated by the Swish activation function before being input into the eSE module. The feature map output from the eSE module is used as the input to convolutional layer 2. The feature map output from convolutional layer 2 is normalized and used as the input to the Dropout module. The feature map output from the Dropout module is aggregated with the original input feature map, and the aggregated feature map is used as the output feature map of the EMB convolution. Step 3.2: Construct a lightweight feature fusion network LDEFPN based on the improved YOLOv11n lightweight pedestrian detection model for crowded scenes; The lightweight feature fusion network LDEFPN specifically includes 8 convolutional layers, 6 Enhance_FPN modules, 2 DySample modules, and 5 C3k2 modules; The output of the C3k2_EMBC module 1 is used as the input of the convolutional layer 6, and the output of the convolutional layer 6 is used as the input of the Enhance_FPN module 3. The output of C3k2_EMBC module 2 is used as the input of convolutional layer 7, and the output of convolutional layer 7 is used as the input of Enhance_FPN module 3; the output of convolutional layer 7 is used as the input of Enhance_FPN module 1, the output of Enhance_FPN module 1 is used as the input of C3k2 module 2, the output of C3k2 module 2 is used as the input of Enhance_FPN module 3, the output of Enhance_FPN module 3 is used as the input of C3k2 module 3, and the output of C3k2 module 3 is used as the input of convolutional layer 10; The output of C3k2_EMBC module 3 serves as the input to convolutional layer 8, the output of convolutional layer 8 serves as the input to Enhance_FPN module 2, and the output of Enhance_FPN module 2 serves as the input to C3k2 module 1. The output of C3k2 module 1 is fused with the output of convolutional layer 7 in Enhance_FPN module 1 via DySample module 1. The outputs of convolutional layer 8, C3k2 module 1, and convolutional layer 10 are fused in Enhance_FPN module 4, and the output of Enhance_FPN module 4 serves as the input to C3k2 module 4. The output of C3k2 module 4 is used as the input to the convolutional layer 7. Input to layer 11; The output of C2PSA module 1 is used as the input of convolutional layer 9. The output of convolutional layer 9 is fused with the output of convolutional layer 8 in Enhance_FPN module 2 via DySample module 2. The output of convolutional layer 9 is fused with the output of convolutional layer 11 in Enhance_FPN module 5. The output of Enhance_FPN module 5 is used as the input of C3k2 module 5. The output of C3k2 module 5 is used as the input of convolutional layer 13. The output of convolutional layer 9 is used as the input of convolutional layer 12. The outputs of convolutional layer 12 and convolutional layer 13 are fused in Enhance_FPN module 6. Step 3.3: Construct an improved head network based on the lightweight pedestrian detection model for crowded scenes using the improved YOLOv11n. The improved head network includes four detection heads. The output of C3k2 module 3 is used as the input of detection head 1, the output of C3k2 module 4 is used as the input of detection head 2, the output of C3k2 module 5 is used as the input of detection head 3, and the output of Enhance_FPN module 6 is used as the output of detection head 4. Step 4: Train the lightweight pedestrian detection model for congested scenes based on the improved YOLOv11n described in Step 3 using the training set and validation set, and save the trained model. Step 4 further includes steps 4.1 to 4.4: Step 4.1: Set the training parameters of the lightweight pedestrian detection model for crowded scenes based on the improved YOLOv11n. The model training parameters include: number of iterations, batch size, optimizer, learning rate, momentum, weight decay, and number of threads. Step 4.2: Input the training set and validation set images and their corresponding labels into the lightweight pedestrian detection model for crowded scenes based on the improved YOLOv11n, use the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters, update the model parameters according to the gradient, and gradually reduce the loss function. Step 4.3: Monitor the loss function value and performance metrics during the training process. When the loss function of the training set and validation set no longer decreases, and the accuracy P, recall R, and mean precision mAP evaluation metrics no longer improve, stop training to avoid model overfitting. Step 4.4: After training is complete, save the trained model and select the optimal model from it; Step 5: Test the optimal model selected in Step 4.4 using the test set, evaluate the test results of the test set, and if the accuracy of the model meets the generalization requirement, then the final lightweight pedestrian detection model for crowded scenes based on the improved YOLOv11n is obtained.

2. The lightweight pedestrian detection method for crowded scenes based on improved YOLOv11n according to claim 1, characterized in that, The Enhance_FPN module in step 3.2 includes a weighted fusion module and a lightweight spatial attention mechanism; Enhance_FPN module 1, Enhance_FPN module 2, Enhance_FPN module 5, and Enhance_FPN module 6 have 2 inputs, while Enhance_FPN module 3 and Enhance_FPN module 4 have 3 inputs; the internal processes of the Enhance_FPN modules with 2 inputs and the Enhance_FPN modules with 3 inputs further include steps 3.2.1 to 3.2.5: Step 3.2.1: The Enhance_FPN module with two inputs has input feature maps X0 and X1 respectively; the Enhance_FPN module with three inputs has input feature maps X0, X1 and X2 respectively. Step 3.2.2: For the Enhance_FPN module with two inputs, assign weights W0 and W1 to them according to the importance of X0 and X1. Multiply X0 by its weight W0, multiply X1 by its weight W1, and then sum the weighted feature maps to obtain a weighted sum. The specific formula for calculating the weighted sum is as follows: （1） In formula (1), O is the output feature after weighted summation, X0 and X1 are the input feature maps of the Enhance_FPN module with two inputs, W0 is the weight of feature map X0, and W1 is the weight of feature map X1. For the Enhance_FPN module with three inputs, weights W0, W1, and W2 are assigned to X0, X1, and X2 based on their importance. X0 is multiplied by its weight W0, X1 by its weight W1, and X2 by its weight W2. The weighted feature maps are summed to obtain a weighted sum, and the specific formula for calculating the weighted sum is as follows: （2） In formula (2), O is the output feature after weighted summation, X0, X1 and X2 are the input feature maps of the Enhance_FPN module with 3 inputs, W0 is the weight of feature map X0, W1 is the weight of feature map X1 and W2 is the weight of feature map X2. Step 3.2.3: The weighted sum introduces nonlinear features through the Swish activation function. The calculation formula after processing with the Swish activation function is: （3） In formula (3), O is the output feature after weighted summation, Swish is the Swish activation function, and fused_feature is the feature map processed by the Swish activation function. Step 3.2.4: Use the feature map fused_feature processed by the Swish activation function as the input feature map of the convolutional layer in the lightweight spatial attention mechanism. The output feature map LSA of the convolutional layer is processed by the Sigmoid activation function. The calculation formula for the Sigmoid activation function is as follows: （4） In formula (4), LSA is the output feature map of the convolutional layer in the lightweight spatial attention mechanism, Sigmoid is the Sigmoid activation function, and attention_map is the feature map processed by the Sigmoid activation function. Step 3.2.5: Multiply the feature map attention_map processed by the Sigmoid activation function and the feature map fused_feature processed by the Swish activation function element-wise. The feature map obtained after element-wise multiplication is used as the final output feature map of the Enhance_FPN module. The specific calculation formula for the element-wise multiplication is as follows: （5） In formula (5), fused_feature is the feature map processed by the Swish activation function, attention_map is the feature map processed by the Sigmoid activation function, and Enhance_feature is the final output feature map of the Enhance_FPN module.

3. The lightweight pedestrian detection method for crowded scenes based on improved YOLOv11n according to claim 1, characterized in that, In step 3.2, each DySample module, before performing the fusion operation in the lightweight feature fusion network LDEFPN, chooses to first reconstruct a high-resolution feature map from the input feature map using DySample upsampling with a static range factor, or chooses to first reconstruct a high-resolution feature map from the input feature map using DySample upsampling with a dynamic range factor. Resolution feature map; Before performing the fusion operation, the lightweight feature fusion network LDEFPN first reconstructs a high-resolution feature map from the input feature map using DySample upsampling with a static range factor, as shown below: (6) (7) (8) In formulas (6) to (8), x represents the input feature map, s represents the upsampling scaling factor, and grid_sample represents the grid sampling operation. Represents the reconstructed high-resolution feature map; o represents the offset, g represents the original sampling grid; linear represents the linear layer operation; Before performing the fusion operation, the lightweight feature fusion network LDEFPN first reconstructs a high-resolution feature map from the input feature map using DySample upsampling with an added dynamic range factor, as shown below: (9) (10) (11) In formulas (9) to (11), x represents the input feature map, s represents the upsampling scaling factor, and grid_sample represents the grid sampling operation. represents the reconstructed high-resolution feature map; o represents the offset, g represents the original sampling grid; linear represents the linear layer operation, and sigmoid is the sigmoid activation function.

4. The lightweight pedestrian detection method for crowded scenes based on improved YOLOv11n according to claim 1, characterized in that, Step 5 further includes steps 5.1 to 5.3: Step 5.1: Input the test set into the optimal model selected in step 4 for testing; Step 5.2: Calculate model performance metrics: accuracy P, recall R, mean precision mAP, number of parameters, computational complexity GFLOPs, frames per second (FPS), and model size. The specific formulas for calculating accuracy P, recall R, and mean precision mAP are as follows: （12） （13） （14） （15） In formulas (12) to (15), P is precision, R is recall, mAP is the mean precision of all classes, AP is the mean precision, m is the total number of pedestrian label classes, TP represents the number of positive samples correctly identified as positive samples, FP represents the number of negative samples incorrectly identified as positive samples, FN represents the number of positive samples incorrectly identified as negative samples, and AP i Let AP be the AP of the i-th type of pedestrian, and P(R) be a function of P with respect to R. Step 5.3: Evaluate the performance metrics of the test set. If the accuracy of the test set is similar to that of the training set, it indicates that the model meets the generalization requirements, and the final lightweight pedestrian detection model for crowded scenes based on the improved YOLOv11n is obtained.

Images