A pedestrian detection method in crowded scenes based on improved YOLOv8n

Abstract

The present invention discloses a method for detecting pedestrians in crowded scenes based on an improved YOLOv8n. The method specifically comprises the following steps: Step 1: Inputting the image to be detected into an improved backbone network to extract features at different scales; Step 2: Inputting the features extracted in Step 1 into the model's feature fusion network DCPAN for feature fusion; Step 3: Inputting the fused features from Step 2 into the model's improved head network to predict and filter the results of pedestrian detection in crowded scenes, obtaining a final prediction result, and thus implementing pedestrian detection in crowded scenes. Compared to the existing technology, the method for detecting pedestrians in crowded scenes based on an improved YOLOv8n disclosed in the present invention can effectively improve the accuracy of pedestrian detection in crowded scenes.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a pedestrian detection method in crowded scenes based on improved YOLOv8n. Background Art

[0002] With the acceleration of urbanization, crowds are increasingly common in public places like shopping malls, train stations, and streets. This not only creates congestion but also poses numerous safety risks, such as stampedes, collisions, and traffic jams, which can even lead to casualties in serious cases. Therefore, developing accurate and rapid pedestrian detection technology in crowded environments is crucial to improving public safety.

[0003] Pedestrian detection in crowded scenes faces multiple challenges. Besides varying lighting conditions and complex backgrounds, the similarities, diverse poses, and sizes of pedestrians significantly increase the difficulty of detection. In particular, mutual occlusion between pedestrians makes it difficult for detection models to accurately distinguish and identify targets within complex backgrounds, thus affecting overall detection performance.

[0004] Currently, there are three main methods for monitoring the safety of people in crowded scenes: manual inspections, traditional pedestrian detection methods, and deep learning-based pedestrian detection methods. Manual inspections rely on on-site patrols by security personnel or real-time observation through video surveillance systems. While this method is intuitive and flexible, it is inefficient and prone to oversight or omissions due to human factors. Traditional methods such as HOG+SVM and Haar+Adaboost extract manual features through sliding windows and perform classification training. These detection algorithms suffer from high computational complexity, weak feature extraction, and poor generalization. In contrast, deep learning-based methods such as CNN, YOLO, and SSD can automatically learn image features, avoiding the tedious manual feature extraction.

[0005] In crowded scenes, pedestrians vary in scale, and occlusions hinder distinguishability of pedestrian features. Furthermore, complex environmental factors can cause significant feature interference. Existing models still suffer from missed or false detections when addressing these challenges. Therefore, it is necessary to develop a pedestrian detection method that can effectively address challenges such as occlusions, multi-scale objects, and complex backgrounds in crowded scenes. Summary of the Invention

[0006] In response to the problems existing in the prior art, the present invention provides a pedestrian detection method for crowded scenes based on an improved YOLOv8n algorithm. By improving the YOLOv8n algorithm, the problem of false detection and missed detection caused by factors such as severe occlusion between pedestrians, diverse scale changes, and complex background in crowded scenes can be solved.

[0007] The technical solution provided by the present invention comprises the following steps:

[0008] Step 1: Input the image to be detected into the improved backbone network of the EMDC-YOLOv8n model to extract features of different scales;

[0009] The improved backbone network consists of a sequentially connected convolutional layer 1, convolutional layer 2, C2f module 1, convolutional layer 3, C2f module 2, convolutional layer 4, C2f module 3, convolutional layer 5, C2f module 4, residual EMA module 1 and SPPF module;

[0010] The improved backbone network outputs four different scale feature information through C2f module 1, C2f module 2, C2f module 3 and SPPF module respectively;

[0011] Step 2: Input the features extracted in step 1 into the feature fusion network DCPAN of the EMDC-YOLOv8n model for feature fusion;

[0012] The feature fusion network DCPAN of the EMDC-YOLOv8n model adds convolutional layers 7, 8, 9, residual EMA module 3, and a new feature scale fusion layer to the YOLOv8n neck network, and uses the DiFPN module to replace the Concat module of the YOLOv8n neck network.

[0013] The new feature scale fusion layer is composed of a convolutional layer 6, an upsampling layer 1, a DiFPN module 3, a C2f module 7, a residual EMA module 2 and a convolutional layer 10 in sequence;

[0014] The output of the SPPF module is used as the input of the convolutional layer 9. The output of the convolutional layer 9 is transmitted to the top-down path. At the same time, the convolutional layer 9 is laterally connected to the DiFPN module 6 in the bottom-up path.

[0015] The output of the C2f module 3 is used as the input of the convolutional layer 8, and the output of the convolutional layer 8 is used as the input of the DiFPN module 2. At the same time, the convolutional layer 8 is laterally connected to the DiFPN module 5 in the bottom-up path;

[0016] The output of the C2f module 2 is used as the input of the convolutional layer 7, and the output of the convolutional layer 7 is used as the input of the DiFPN module 1. At the same time, the convolutional layer 7 is laterally connected to the DiFPN module 4 in the bottom-up path; the output of the DiFPN module 4 is used as the input of the C2f module 8; the output of the C2f module 8 is used as the input of the residual EMA module 3; the output of the residual EMA module 3 is used as the input of the convolutional layer 11;

[0017] The output of the C2f module 1 is used as the input of the convolutional layer 6, and the convolutional layer 6 is horizontally connected to the DiFPN module 3 in the bottom-up layer. At the same time, the output of the upsampling 1 is also used as the input of the DiFPN module 3; the output of the DiFPN module 3 is used as the input of the C2f module 7, and the output of the C2f module 7 is used as the input of the residual EMA module 2; the output of the residual EMA module 2 is used as the input of the convolutional layer 10, and the output of the convolutional layer 10 is used as the input of the DiFPN module 4;

[0018] Step 3: Input the fused features from step 2 into the improved head network of the EMDC-YOLOv8n model to predict and filter the pedestrian detection results in crowded scenes, obtaining the final prediction results for pedestrian detection in crowded scenes, and thus realizing pedestrian detection in crowded scenes.

[0019] An improved head network of the EMDC-YOLOv8n model, including four ASFF detection heads;

[0020] ASFF detection head 4 receives the feature map output by C2f module 10; ASFF detection head 3 receives the feature map output by C2f module 9; ASFF detection head 2 receives the feature map output by residual EMA module 3; ASFF detection head 1 receives the feature map output by residual EMA module 2.

[0021] Furthermore, the residual EMA module in step 1 includes one EMA module, one depthwise separable convolution and one 1×1 convolution layer; the internal process of the residual EMA module further includes steps 1.1 to 1.3:

[0022] Step 1.1: Input the input feature map to the EMA module; the feature map output by the EMA module is used as the input of the depthwise separable convolution; the feature map after the depthwise separable convolution operation is aggregated with the feature map output by the EMA module;

[0023] Furthermore, the internal process of the EMA module in step 1.1 includes steps 1.1.1 to 1.1.6:

[0024] Step 1.1.1: The EMA module divides the input feature map into g sub-features and learns different semantic information;

[0025] Step 1.1.2: Extract the attention weight descriptor of the grouped feature map from the g sub-features using three parallel paths; the first and second paths of the three parallel paths are routed through 1×1 branches, and the third path of the three parallel paths uses a 3×3 branch;

[0026] Step 1.1.3: There are two global average pooling operations in the 1×1 branch, which encode the channels along two spatial directions respectively and concatenate the two encoded features so that their dimensions remain unchanged in the 1×1 branch;

[0027] Step 1.1.4: Decompose the concatenated features into two vectors after point-wise convolution. Aggregate the two channel attentions through simple multiplication to achieve cross-channel interaction between the two parallel paths in the 1×1 branch.

[0028] Step 1.1.5: Use a single 3×3 convolution in the 3×3 branch to capture local cross-channel interactions to expand the feature space, and use global average pooling to encode the global spatial information output by the two branches;

[0029] Step 1.1.6: The output features of the parallel path are converted into the aggregation of two spatial attention weights through the matrix dot product operation, and then the output feature map is obtained after the Sigmoid function.

[0030] Step 1.2: Input the aggregated result into the 1×1 convolutional layer;

[0031] Step 1.3: Aggregate the feature map of the original input with the feature map output by the 1×1 convolutional layer to obtain the output feature map of the residual EMA module.

[0032] Furthermore, the structure and internal process of the residual EMA module in step 2 are the same as those of the residual EMA module in step 1.

[0033] Furthermore, the DiFPN module 1, DiFPN module 2, DiFPN module 3 and DiFPN module 6 in step 2 have 2 inputs, and the DiFPN module with 2 inputs includes 2 DCNv4 modules and 1 weighted fusion module; DiFPN module 4 and DiFPN module 5 have 3 inputs, and the DiFPN module with 3 inputs includes 3 DCNv4 modules and 1 weighted fusion module; the internal process of the DiFPN module with 2 inputs and the DiFPN module with 3 inputs further includes steps 2.1 to 2.7:

[0034] Step 2.1: The DiFPN module with two inputs, wherein the input feature maps are feature map X0 and feature map X1; the feature map X0 and feature map X1 are respectively input into the two DCNv4 modules in the DiFPN module with two inputs;

[0035] The DiFPN module with three inputs has input feature maps X0, X1, and X2; the feature maps X0, X1, and X2 are respectively input into the three DCNv4 modules in the DiFPN module with three inputs;

[0036] Step 2.2: The DCNv4 module uses one convolutional layer to predict the offset of each pixel position in the input feature map;

[0037] Step 2.3: Dynamically adjust the position of the sampling point according to the offset. For each convolution kernel position (i, j), calculate the new sampling position (i′, j′):

[0038] (i′,j′)=(i,j)+offset(i,j) (1)

[0039] In formula (1), offset(i,j) is the predicted offset;

[0040] Step 2.4: For the DiFPN module with two inputs, the feature map X0, the feature map X1 and offset are used as the input of the deformable convolution layer. Finally, two deformed feature maps X′0 and X′1 are generated.

[0041] For the DiFPN module with 3 inputs, the feature map X0, feature map X1, feature map X2 and offset are used as the input of the deformable convolution layer. Finally, 3 deformed feature maps X′0, X′1 and X′2 are generated.

[0042] Step 2.5: For the DiFPN module with two inputs, the two deformed feature maps X′0 and X′1 are input into the weighted fusion module. According to the importance of X′0 and X′1, weights W0 and W1 are assigned to them. X′0 is multiplied by its weight W0, and X′1 is multiplied by its weight W1. The weighted feature maps are added together to obtain a weighted sum. The calculation formula of the weighted sum is:

[0043] O=X′0·W0+X′1·W1 (2)

[0044] In formula (2), O is the output feature after weighted fusion, W i is the weight, X′ i Output features for DCNv4, i = 0, 1;

[0045] For the DiFPN module with three inputs, the three deformed feature maps X′0, X′1, and X′2 are input into the weighted fusion module. According to the importance of X′0, X′1, and X′2, weights W0, W1, and W2 are assigned to them. X′0 is multiplied by its weight W0, X′1 is multiplied by its weight W1, and X′2 is multiplied by its weight W2. The weighted feature maps are added together to obtain a weighted sum. The calculation formula of the weighted sum is:

[0046] O=X′0·W0+X′1·W1+X′2·W2 (3)

[0047] In formula (3), O is the output feature after weighted fusion, W i is the weight, X′ i Output features for DCNv4, i = 0, 1, 2;

[0048] Step 2.6: The weighted sum is introduced into nonlinear features through the Swish activation function;

[0049] Step 2.7: For the DiFPN module with two inputs, use a 1×1 convolutional layer to operate on the activated feature map to obtain the output feature map of the DiFPN module with two inputs;

[0050] For the DiFPN module with 3 inputs, a 1×1 convolutional layer is used to operate the activated feature map to obtain the output feature map of the DiFPN module with 3 inputs.

[0051] Furthermore, the feature fusion method of the horizontal span connection in step 2 is:

[0052] For the i-th layer, the i-th layer feature map P of the input layer is obtained according to the top-down and bottom-up feature propagation i in and the i+1th layer feature map The i+1th layer feature map Upsample and adjust to the same size as the feature map of layer i The upsampled i+1th layer feature map is weightedly fused and normalized with the i-th layer feature map to obtain the intermediate feature map of the i-th layer in the top-down path and the output feature map of the i-th layer in the bottom-up path;

[0053] The feature fusion expression of the intermediate feature map of the i-th layer in the top-down path is:

[0054]

[0055] In formula (4), ω1 represents the weight of the feature map of the i-th layer, ω2 represents the weight of the feature map of the i+1-th layer after upsampling, ε is a parameter used to prevent the denominator from being 0, and Conv represents feature fusion;

[0056] The feature fusion expression of the output feature map of the i-th layer in the bottom-up path is:

[0057]

[0058] In formula (5), represents the output feature map of the i-1th layer in the top-down path, Indicates that the output feature map of the i-1th layer is upsampled and adjusted to the same size as the feature map of the i-th layer, ω′1 represents the weight of the input feature map of the i-th layer, ω′2 represents the weight of the intermediate feature map of the i-th layer in the bottom-up path, and ω′3 represents the weight of the feature map of the i-1th layer after upsampling.

[0059] Furthermore, the improved head network in step 3.3 includes four ASFF detection heads; wherein each ASFF detection head includes constant scaling, adaptive fusion, one ASFF layer, and one detection head; the internal process of the improved head network further includes steps 3.1 to 3.4:

[0060] Step 3.1: The improved head network receives four feature layers of different scales output by the feature fusion network DCPAN; the four feature layers of different scales are: the feature map output by the C2f module 10, the feature map output by the C2f module 9, the feature map output by the residual EMA module 3, and the feature map output by the residual EMA module 2;

[0061] Step 3.2: Perform constant scaling on the four feature layers of different scales;

[0062] Step 3.3: Adaptively fuse the identically scaled features to obtain four ASFF layers, corresponding to ASFF layer 1, ASFF layer 2, ASFF layer 3, and ASFF layer 4, respectively. The features obtained by each detection head have the same resolution and number of channels. The features adaptively fused at the level 1 feature layer are adaptively fused using the following method:

[0063]

[0064] In formula (6), Represents the feature map y l The output at (i,j), The feature vector representing the position (i, j) of the feature map from n to l, is the learnable weight coefficient of the four-level feature map, satisfying the following relationship:

[0065]

[0066]

[0067] In formula (7) to formula (11), e is the Euler number, and 1×1 convolutional layers are used to respectively and Calculate the weighted scalar mapping They are learned via standard backpropagation, Use As a control parameter, it is defined by the Softmax function;

[0068] Step 3.4: Input the output feature map of the ASFF layer 1 into the detection head 1; input the output feature map of the ASFF layer 2 into the detection head 2; input the output feature map of the ASFF layer 3 into the detection head 3; input the features of the ASFF layer 4 into the detection head 4;

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] (1) The present invention discloses a pedestrian detection method for crowded scenes based on an improved YOLOv8n. This method introduces the residual EMA module disclosed in the present invention patent. This module captures the key features of pedestrians of different scales by combining channel information and context information, effectively distinguishing pedestrians from the background.

[0071] (2) This method enhances the multi-scale pedestrian feature fusion capability through the feature fusion network DCPAN disclosed in the patent of this invention, and can also more flexibly focus on capturing the key features of occluded pedestrians;

[0072] (3) This method introduces the ASFF detection head to adaptively fuse the four pedestrian feature maps of different scales output by the feature fusion network DCPAN, thereby improving the model's recognition ability for pedestrian targets of various scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 Schematic diagram of the EMDC-YOLOv8n model structure of the present invention;

[0074] Figure 2 Schematic diagram of the residual EMA module and EMA module structure;

[0075] Figure 3 Schematic diagram of the DiFPN module structure with 2 inputs;

[0076] Figure 4 Schematic diagram of the DiFPN module structure with 3 inputs;

[0077] Figure 5 To improve the internal structure diagram of the head network; DETAILED DESCRIPTION

[0078] In order to make the technical solutions, structural features, achieved objectives and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and accompanying drawings. It should be noted that the specific embodiments described herein are only used to more clearly explain the present invention and are not intended to limit the present invention.

[0079] Figure 1 This is a schematic diagram of the EMDC-YOLOv8n model structure disclosed in the present invention, and its implementation process is as follows:

[0080] Step 1: Input the image to be detected into the improved backbone network of the EMDC-YOLOv8n model to extract features at different scales;

[0081] The improved backbone network consists of a sequentially connected convolutional layer 1, convolutional layer 2, C2f module 1, convolutional layer 3, C2f module 2, convolutional layer 4, C2f module 3, convolutional layer 5, C2f module 4, residual EMA module 1 and SPPF module;

[0082] The improved backbone network outputs four different scale feature information through C2f module 1, C2f module 2, C2f module 3 and SPPF module respectively;

[0083] Furthermore, the residual EMA module structure is as follows Figure 2 As shown, the residual EMA module includes 1 EMA module, 1 depth-separable convolution and 1 1×1 convolution layer; the internal process of the residual EMA module further includes steps 1.1 to 1.3:

[0084] Step 1.1: Input the input feature map to the EMA module; the feature map output by the EMA module is used as the input of the depthwise separable convolution; the feature map after the depthwise separable convolution operation is aggregated with the feature map output by the EMA module;

[0085] Furthermore, the EMA module structure is as follows Figure 2 As shown, the internal process of the EMA module includes steps 1.1.1 to 1.1.6:

[0086] Step 1.1.1: The EMA module divides the input feature map into g sub-features and learns different semantic information;

[0087] Step 1.1.2: Extract the attention weight descriptor of the grouped feature map from the g sub-features using three parallel paths; the first and second paths of the three parallel paths are routed through 1×1 branches, and the third path of the three parallel paths uses a 3×3 branch;

[0088] Step 1.1.3: There are two global average pooling operations in the 1×1 branch, which encode the channels along two spatial directions respectively and concatenate the two encoded features so that their dimensions remain unchanged in the 1×1 branch;

[0089] Step 1.1.4: Decompose the concatenated features into two vectors after point-wise convolution. Aggregate the attention of the two channels through a simple multiplication to achieve cross-channel interaction between the two parallel paths in the 1×1 branch.

[0090] Step 1.1.5: Use a single 3×3 convolution in the 3×3 branch to capture local cross-channel interactions to expand the feature space, and use global average pooling to encode the global spatial information output by both branches;

[0091] Step 1.1.6: The output features of the parallel path are converted into the aggregation of two spatial attention weights through the matrix dot product operation, and then the output feature map is obtained after the Sigmoid function.

[0092] Step 1.2: Input the aggregated result into the 1×1 convolutional layer;

[0093] Step 1.3: Aggregate the feature map of the original input with the feature map output by the 1×1 convolutional layer to obtain the output feature map of the residual EMA module;

[0094] Step 2: Input the features extracted in step 1 into the feature fusion network DCPAN of the EMDC-YOLOv8n model for feature fusion;

[0095] The feature fusion network DCPAN of the EMDC-YOLOv8n model adds convolutional layers 7, 8, 9, residual EMA module 3, and a new feature scale fusion layer to the YOLOv8n neck network, and uses the DiFPN module to replace the Concat module of the YOLOv8n neck network.

[0096] The new feature scale fusion layer is composed of a convolutional layer 6, an upsampling layer 1, a DiFPN module 3, a C2f module 7, a residual EMA module 2 and a convolutional layer 10 in sequence;

[0097] The output of the SPPF module is used as the input of the convolutional layer 9. The output of the convolutional layer 9 is transmitted to the top-down path. At the same time, the convolutional layer 9 is laterally connected to the DiFPN module 6 in the bottom-up path.

[0098] The output of the C2f module 3 is used as the input of the convolutional layer 8, and the output of the convolutional layer 8 is used as the input of the DiFPN module 2. At the same time, the convolutional layer 8 is laterally connected to the DiFPN module 5 in the bottom-up path;

[0099] The output of the C2f module 2 is used as the input of the convolutional layer 7, and the output of the convolutional layer 7 is used as the input of the DiFPN module 1. At the same time, the convolutional layer 7 is laterally connected to the DiFPN module 4 in the bottom-up path; the output of the DiFPN module 4 is used as the input of the C2f module 8; the output of the C2f module 8 is used as the input of the residual EMA module 3; the output of the residual EMA module 3 is used as the input of the convolutional layer 11;

[0100] The output of the C2f module 1 is used as the input of the convolutional layer 6, and the convolutional layer 6 is horizontally connected to the DiFPN module 3 in the bottom-up layer. At the same time, the output of the upsampling 1 is also used as the input of the DiFPN module 3; the output of the DiFPN module 3 is used as the input of the C2f module 7, and the output of the C2f module 7 is used as the input of the residual EMA module 2; the output of the residual EMA module 2 is used as the input of the convolutional layer 10, and the output of the convolutional layer 10 is used as the input of the DiFPN module 4;

[0101] Furthermore, the structure of the DiFPN module with 2 inputs is as follows Figure 3 As shown, the structure of the DiFPN module with 3 inputs is as follows Figure 4 As shown, the DiFPN module 1, DiFPN module 2, DiFPN module 3 and DiFPN module 6 in step 2 have 2 inputs, and the DiFPN module with 2 inputs includes 2 DCNv4 modules and 1 weighted fusion module; DiFPN module 4 and DiFPN module 5 have 3 inputs, and the DiFPN module with 3 inputs includes 3 DCNv4 modules and 1 weighted fusion module; the internal process of the DiFPN module with 2 inputs and the DiFPN module with 3 inputs further includes steps 2.1 to 2.7:

[0102] Step 2.1: The DiFPN module with two inputs, wherein the input feature maps are feature map X0 and feature map X1; the feature map X0 and feature map X1 are respectively input into the two DCNv4 modules in the DiFPN module with two inputs;

[0103] The DiFPN module with three inputs has input feature maps X0, X1, and X2; the feature maps X0, X1, and X2 are respectively input into the three DCNv4 modules in the DiFPN module with three inputs;

[0104] Step 2.2: The DCNv4 module uses one convolutional layer to predict the offset of each pixel position in the input feature map;

[0105] Step 2.3: Dynamically adjust the position of the sampling point according to the offset. For each convolution kernel position (i, j), calculate the new sampling position (i′, j′):

[0106] (i′,j′)=(i,j)+offset(i,j) (1)

[0107] In formula (1), offset(i,j) is the predicted offset;

[0108] Step 2.4: For the DiFPN module with two inputs, the feature map X0, the feature map X1 and the offset are used as the input of the deformable convolution layer. Finally, two deformed feature maps X′0 and X′1 are generated.

[0109] For the DiFPN module with three inputs, the feature map X0, feature map X1, feature map X2 and offset are used as the input of the deformable convolution layer. Finally, three deformed feature maps X′0, X′1 and X′2 are generated.

[0110] Step 2.5: For the DiFPN module with two inputs, the two deformed feature maps X′0 and X′1 are input into the weighted fusion module. According to the importance of X′0 and X′1, weights W0 and W1 are assigned to them. X′0 is multiplied by its weight W0, and X′1 is multiplied by its weight W1. The weighted feature maps are added together to obtain a weighted sum. The calculation formula of the weighted sum is:

[0111] O=X′0·W0+X′1·W1 (2)

[0112] In formula (2), O is the output feature after weighted fusion, W i is the weight, X′ i Output features for DCNv4, i = 0, 1;

[0113] For the DiFPN module with three inputs, the three deformed feature maps X′0, X′1, and X′2 are input into the weighted fusion module. According to the importance of X′0, X′1, and X′2, weights W0, W1, and W2 are assigned to them. X′0 is multiplied by its weight W0, X′1 is multiplied by its weight W1, and X′2 is multiplied by its weight W2. The weighted feature maps are added together to obtain a weighted sum. The calculation formula of the weighted sum is:

[0114] O=X′0·W0+X′1·W1+X′2·W2 (3)

[0115] In formula (3), O is the output feature after weighted fusion, W i is the weight, X′ i Output features for DCNv4, i = 0, 1, 2;

[0116] Step 2.6: The weighted sum is introduced into nonlinear features through the Swish activation function;

[0117] Step 2.7: For the DiFPN module with two inputs, use a 1×1 convolutional layer to operate on the activated feature map to obtain the output feature map of the DiFPN module with two inputs;

[0118] For the DiFPN module with 3 inputs, a 1×1 convolutional layer is used to operate the activated feature map to obtain the output feature map of the DiFPN module with 3 inputs.

[0119] Furthermore, horizontal spanning connections such as Figure 1 As shown by the dotted line in , the feature fusion method of the horizontal span connection is:

[0120] For the i-th layer, the i-th layer feature map P of the input layer is obtained according to the top-down and bottom-up feature propagation i in and the i+1th layer feature map The i+1th layer feature map Upsample and adjust to the same size as the feature map of layer i The upsampled i+1th layer feature map is weightedly fused and normalized with the i-th layer feature map to obtain the intermediate feature map of the i-th layer in the top-down path and the output feature map of the i-th layer in the bottom-up path;

[0121] The feature fusion expression of the intermediate feature map of the i-th layer in the top-down path is:

[0122]

[0123] In formula (4), ω1 represents the weight of the feature map of the i-th layer, ω2 represents the weight of the feature map of the i+1-th layer after upsampling, ε is a parameter used to prevent the denominator from being 0, and Conv represents feature fusion;

[0124] The feature fusion expression of the output feature map of the i-th layer in the bottom-up path is:

[0125]

[0126] In formula (5), represents the output feature map of the i-1th layer in the top-down path, Indicates that the output feature map of the i-1th layer is upsampled and adjusted to the same size as the feature map of the i-th layer, ω′1 represents the weight of the input feature map of the i-th layer, ω′2 represents the weight of the intermediate feature map of the i-th layer in the bottom-up path, and ω′3 represents the weight of the feature map of the i-1th layer after upsampling.

[0127] Step 3: Input the fused features from step 2 into the improved head network of the EMDC-YOLOv8n model to predict and filter the pedestrian detection results in crowded scenes, obtaining the final prediction results for pedestrian detection in crowded scenes, and thus realizing pedestrian detection in crowded scenes.

[0128] An improved head network of the EMDC-YOLOv8n model, including four ASFF detection heads;

[0129] ASFF detection head 4 receives the feature map output by C2f module 10; ASFF detection head 3 receives the feature map output by C2f module 9; ASFF detection head 2 receives the feature map output by residual EMA module 3; ASFF detection head 1 receives the feature map output by residual EMA module 2.

[0130] Furthermore, the internal structure of the head network is improved. Figure 5 As shown, the improved head network includes four ASFF detection heads; each ASFF detection head includes constant scaling, adaptive fusion, one ASFF layer, and one detection head; the internal process of the improved head network further includes steps 3.1 to 3.4:

[0131] Step 3.1: The improved head network receives four feature layers of different scales output by the feature fusion network DCPAN; the four feature layers of different scales are: the feature map output by the C2f module 10, the feature map output by the C2f module 9, the feature map output by the residual EMA module 3, and the feature map output by the residual EMA module 2;

[0132] Step 3.2: Perform constant scaling on the four feature layers of different scales;

[0133] Step 3.3: Adaptively fuse the identically scaled features to obtain four ASFF layers, corresponding to ASFF layer 1, ASFF layer 2, ASFF layer 3, and ASFF layer 4, respectively. The features obtained by each detection head have the same resolution and number of channels. The features adaptively fused at the level 1 feature layer are adaptively fused using the following method:

[0134]

[0135] In formula (6), Represents the feature map y l The output at (i,j), The feature vector representing the position (i, j) of the feature map from n to l, is the learnable weight coefficient of the four-level feature map, satisfying the following relationship:

[0136]

[0137]

[0138] In formula (7) to formula (11), e is the Euler number, and 1×1 convolutional layers are used to respectively and Calculate the weighted scalar mapping They are learned via standard backpropagation, Use As a control parameter, it is defined by the Softmax function;

[0139] Step 3.4: Input the output feature map of the ASFF layer 1 into the detection head 1; input the output feature map of the ASFF layer 2 into the detection head 2; input the output feature map of the ASFF layer 3 into the detection head 3; input the output feature map of the ASFF layer 4 into the detection head 4;

[0140] In this embodiment, in order to verify the effect of the EMDC-YOLOv8n model disclosed in the present invention, this paper uses the crowded scene pedestrian detection model of the YOLOv8n model, YOLOv7-tiny model, YOLOv10n model, YOLOv11n model, GR-yolo model and EMDC-YOLOv8n model to test on the CrowedHumen dataset. The evaluation results are shown in Table 1. The EMDC-YOLOv8n model proposed in this paper outperforms other comparison models in terms of precision P, recall R, mAP@0.5 and mAP@0.5:0.95 evaluation indicators.

[0141] Table 1 Comparative experimental results

[0142] Model Accuracy P Recall R mAP@0.5 mAP@0.5:0.95 YOLOv8n model 83.91 64.28 75.01 47.41 YOLOv7-tiny model 84.39 71.17 78.23 45.49 YOLOv10n model 82.2 65.73 75.41 47.52 YOLOv11n model 83.68 64 74.78 47.14 GR-yolo model 84.48 65.43 76.06 48.41 EMDC-YOLOv8n model 86.05 72.68 82.21 54.65

[0143] In this embodiment, in order to verify the effect of the feature fusion network DCPAN in the EMDC-YOLOv8n model disclosed in the present invention, this paper uses the YOLOv8n model, YOLOv8n model + BiFPN, YOLOv8n model + GlodFPN, YOLOv8n model + RepGFPN and YOLOv8n model + the feature extraction network DCPAN disclosed in the patent of this invention to perform tests on the CrowedHumen dataset. The evaluation results are shown in Table 2. The feature fusion network DCPAN proposed in this paper outperforms other comparison models in terms of precision P, recall R, mAP@0.5 and mAP@0.5:0.95 evaluation indicators.

[0144] Table 2. Comparative experimental results of feature fusion network

[0145]

[0146] In this example, in order to verify the effect of the residual EMA module in the EMDC-YOLOv8n model disclosed in the present invention, Table 3 shows the experimental results of the EMDC-YOLOv8n model performance using different attention mechanisms. The residual EMA module proposed in this article outperforms other attention mechanisms in the precision P, recall R, mAP@0.5, and mAP@0.5:0.95 evaluation indicators of the EMDC-YOLOv8n model.

[0147] Table 3 Comparative experimental results of attention mechanism

[0148] Attention Mechanism Accuracy P Recall R mAP@0.5 mAP@0.5: 0.95 EMA 85.79 72.66 82.03 54.33 CBAM 86.01 72.55 82.09 54.52 SEAM 86 72.43 81.99 54.51 Residual EMA module 86.05 72.68 82.21 54.65

[0149] The above description is only one embodiment of the present invention and does not limit the patent scope of the present invention. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A pedestrian detection method in crowded scenes based on improved YOLOv8n, characterized by: The EMDC-YOLOv8n pedestrian detection model for crowded scenes based on the improved YOLOv8n includes an improved backbone network, a feature fusion network DCPAN, and an improved head network. The construction of the EMDC-YOLOv8n model further includes steps 1 to 3: Step 1: Input the image to be detected into the improved backbone network of the EMDC-YOLOv8n model to extract features of different scales; The improved backbone network consists of a sequentially connected convolutional layer 1, convolutional layer 2, C2f module 1, convolutional layer 3, C2f module 2, convolutional layer 4, C2f module 3, convolutional layer 5, C2f module 4, residual EMA module 1 and SPPF module; The improved backbone network outputs feature information of four different scales through C2f module 1, C2f module 2, C2f module 3 and SPPF module respectively; Step 2: Input the features extracted in step 1 into the feature fusion network DCPAN of the EMDC-YOLOv8n model for feature fusion; The feature fusion network DCPAN of the EMDC-YOLOv8n model adds convolutional layers 7, 8, 9, residual EMA module 3, and a new feature scale fusion layer to the YOLOv8n neck network, and uses the DiFPN module to replace the Concat module of the YOLOv8n neck network. The new feature scale fusion layer is composed of a convolutional layer 6, an upsampling layer 1, a DiFPN module 3, a C2f module 7, a residual EMA module 2 and a convolutional layer 10 in sequence; The output of the SPPF module is used as the input of the convolutional layer 9. The output of the convolutional layer 9 is transmitted to the top-down path. At the same time, the convolutional layer 9 is laterally connected to the DiFPN module 6 in the bottom-up path. The output of the C2f module 3 is used as the input of the convolutional layer 8, and the output of the convolutional layer 8 is used as the input of the DiFPN module 2. At the same time, the convolutional layer 8 is laterally connected to the DiFPN module 5 in the bottom-up path; The output of the C2f module 2 is used as the input of the convolutional layer 7, and the output of the convolutional layer 7 is used as the input of the DiFPN module 1. At the same time, the convolutional layer 7 is laterally connected to the DiFPN module 4 in the bottom-up path; the output of the DiFPN module 4 is used as the input of the C2f module 8; the output of the C2f module 8 is used as the input of the residual EMA module 3; the output of the residual EMA module 3 is used as the input of the convolutional layer 11; The output of the C2f module 1 is used as the input of the convolutional layer 6, and the convolutional layer 6 is horizontally connected to the DiFPN module 3 in the bottom-up path. At the same time, the output of the upsampling 1 is also used as the input of the DiFPN module 3; the output of the DiFPN module 3 is used as the input of the C2f module 7, and the output of the C2f module 7 is used as the input of the residual EMA module 2; the output of the residual EMA module 2 is used as the input of the convolutional layer 10, and the output of the convolutional layer 10 is used as the input of the DiFPN module 4; Step 3: Input the fused features from step 2 into the improved head network of the EMDC-YOLOv8n model to predict and filter the pedestrian detection results in crowded scenes, obtaining the final prediction results for pedestrian detection in crowded scenes, and thus realizing pedestrian detection in crowded scenes. An improved head network of the EMDC-YOLOv8n model, including four ASFF detection heads; ASFF detection head 4 receives the feature map output by C2f module 10; ASFF detection head 3 receives the feature map output by C2f module 9; ASFF detection head 2 receives the feature map output by residual EMA module 3; ASFF detection head 1 receives the feature map output by residual EMA module 2.

2. The method for pedestrian detection in crowded scenes based on improved YOLOv8n according to claim 1, characterized in that: The residual EMA module in step 1 includes one EMA module, one depthwise separable convolution and one 1×1 convolution layer; the internal process of the residual EMA module further includes steps 1.1 to 1.3: Step 1.1: Input the input feature map to the EMA module, and the feature map output by the EMA module is used as the input of the depthwise separable convolution; the feature map output by the depthwise separable convolution is aggregated with the feature map output by the EMA module; Step 1.2: Input the aggregated result into the 1×1 convolutional layer; Step 1.3: Aggregate the feature map of the original input with the feature map output by the 1×1 convolutional layer to obtain the output feature map of the residual EMA module.

3. The method for pedestrian detection in crowded scenes based on improved YOLOv8n according to claim 1, characterized in that: The structure and internal process of the residual EMA module in step 2 are the same as those of the residual EMA module in step 1.

4. The method for pedestrian detection in crowded scenes based on improved YOLOv8n according to claim 1, characterized in that: The DiFPN module 1, DiFPN module 2, DiFPN module 3 and DiFPN module 6 in step 2 have 2 inputs, and the DiFPN module with 2 inputs includes 2 DCNv4 modules and 1 weighted fusion module; DiFPN module 4 and DiFPN module 5 have 3 inputs, and the DiFPN module with 3 inputs includes 3 DCNv4 modules and 1 weighted fusion module; the internal process of the DiFPN module with 2 inputs and the DiFPN module with 3 inputs further includes steps 2.1 to 2.7: Step 2.1: The DiFPN module with two inputs, wherein the input feature maps are feature map X0 and feature map X1; the feature map X0 and feature map X1 are respectively input into the two DCNv4 modules in the DiFPN module with two inputs; The DiFPN module with three inputs has input feature maps X0, X1, and X2; the feature maps X0, X1, and X2 are respectively input into the three DCNv4 modules in the DiFPN module with three inputs; Step 2.2: The DCNv4 module uses one convolutional layer to predict the offset of each pixel position in the input feature map; Step 2.3: Dynamically adjust the position of the sampling point according to the offset, and for each convolution kernel position (i, j), calculate the new sampling position ( , ): （1） In formula (1), offset(i,j) is the predicted offset; Step 2.4: For the DiFPN module with 2 inputs, the feature map X0, feature map X1 and offset are used as the input of the deformable convolution layer. Finally, two deformed feature maps are generated. and ; For the DiFPN module with 3 inputs, the feature map X0, feature map X1, feature map X2 and offset are used as the input of the deformable convolution layer. Finally, 3 deformed feature maps are generated. 、 and ; Step 2.5: For the DiFPN module with 2 inputs, transform the two deformed feature maps and Input to the weighted fusion module, according to and The importance of is assigned weights W0 and W1, Multiplying by its weight W0, Multiply it by its weight W1 and add the weighted feature maps to get the weighted sum. The calculation formula of the weighted sum is: （2） In formula (2), O is the output feature after weighted fusion, W i is the weight, Output features for DCNv4, i=0, 1; For the DiFPN module with 3 inputs, the 3 deformed feature maps are 、 and Input to the weighted fusion module, according to 、 and The importance of is assigned weights W0, W1 and W2. Multiplying by its weight W0, Multiplying by its weight W1, Multiply it by its weight W2 and add the weighted feature maps to get the weighted sum. The calculation formula of the weighted sum is: （3） In formula (3), O is the output feature after weighted fusion, W i is the weight, Output features for DCNv4, i=0, 1, 2; Step 2.6: The weighted sum is introduced into nonlinear features through the Swish activation function; Step 2.7: For the DiFPN module with two inputs, use a 1×1 convolutional layer to operate on the activated feature map to obtain the output feature map of the DiFPN module with two inputs; For the DiFPN module with 3 inputs, a 1×1 convolutional layer is used to operate the activated feature map to obtain the output feature map of the DiFPN module with 3 inputs.

5. The method for pedestrian detection in crowded scenes based on improved YOLOv8n according to claim 1, characterized in that: The feature fusion method for each horizontal span connection in step 2 is: For the i-th layer, the i-th layer feature map of the input layer is obtained according to the top-down and bottom-up feature propagation and the i+1th layer feature map , the i+1th layer feature map Upsample and adjust to the same size as the feature map of layer i , perform weighted fusion and normalization on the upsampled i+1th layer feature map and the i-th layer feature map to obtain the intermediate feature map of the i-th layer in the top-down path and the output feature map of the i-th layer in the bottom-up path; The feature fusion expression of the intermediate feature map of the i-th layer in the top-down path is: （4） In formula (4), ω1 represents the weight of the feature map of the i-th layer, ω2 represents the weight of the feature map of the i+1-th layer after upsampling, ε is a parameter used to prevent the denominator from being 0, and Conv represents feature fusion; The feature fusion expression of the output feature map of the i-th layer in the bottom-up path is: （5） In formula (5), represents the output feature map of the i-1th layer in the top-down path, Indicates that the output feature map of the i-1th layer is upsampled and adjusted to the same size as the feature map of the i-th layer. represents the weight of the input i-th layer feature map, represents the weight of the intermediate feature map of layer i in the bottom-up path, Represents the weight of the i-1th layer feature map after upsampling.

6. The method for pedestrian detection in crowded scenes based on improved YOLOv8n according to claim 1, characterized in that: The improved head network of step 3 includes four ASFF detection heads; each ASFF detection head includes constant scaling, adaptive fusion, one ASFF layer, and one detection head; the internal process of the improved head network further includes steps 3.1 to 3.4: Step 3.1: The improved head network receives four feature layers of different scales output by the feature fusion network DCPAN; the four feature layers of different scales are: the feature map output by the C2f module 10, the feature map output by the C2f module 9, the feature map output by the residual EMA module 3, and the feature map output by the residual EMA module 2; Step 3.2: Perform constant scaling on the four feature layers of different scales; Step 3.3: Adaptively fuse the identically scaled features to obtain four ASFF layers, corresponding to ASFF layer 1, ASFF layer 2, ASFF layer 3, and ASFF layer 4, respectively. The features obtained by each detection head have the same resolution and number of channels. The features adaptively fused at the level 1 feature layer are adaptively fused using the following method: （6） In formula (6), Represents the feature map y l The output at (i,j), The feature vector representing the position (i, j) of the feature map from n to l, 、 、 、 is the learnable weight coefficient of the four-level feature map, satisfying the following relationship: （7） （8） （9） （10） （11） In formula (7) to formula (11), e is the Euler number; 1×1 convolutional layers are used to respectively 、 、 and Calculate the weighted scalar mapping 、 、 、 , learn them through standard back-propagation, 、 、 、 Use 、 、 、 As a control parameter, it is defined by the Softmax function; Step 3.4: Input the output feature map of the ASFF layer 1 into the detection head 1; input the output feature map of the ASFF layer 2 into the detection head 2; input the output feature map of the ASFF layer 3 into the detection head 3; input the output feature of the ASFF layer 4 into the detection head 4.

7. The method for pedestrian detection in crowded scenes based on improved YOLOv8n according to claim 2, characterized in that: The internal process of the EMA module in step 1.1 includes steps 1.1.1 to 1.1.6: Step 1.1.1: The EMA module divides the input feature map into g sub-features and learns different semantic information; Step 1.1.2: Extract the attention weight descriptor of the grouped feature map from the g sub-features using three parallel paths; the first and second paths of the three parallel paths are routed through 1×1 branches, and the third path of the three parallel paths uses a 3×3 branch; Step 1.1.3: There are two global average pooling operations in the 1×1 branch, which encode the channels along two spatial directions respectively and concatenate the two encoded features so that their dimensions remain unchanged in the 1×1 branch; Step 1.1.4: Decompose the concatenated features into two vectors after point-wise convolution. Aggregate the two channel attentions through simple multiplication to achieve cross-channel interaction between the two parallel paths in the 1×1 branch. Step 1.1.5: Use a single 3×3 convolution in the 3×3 branch to capture local cross-channel interactions to expand the feature space, and use global average pooling to encode the global spatial information output by the two branches; Step 1.1.6: The output features of the parallel path are converted into the aggregation of two spatial attention weights through the matrix dot product operation, and then the output feature map is obtained after the Sigmoid function.

Images