Dense pedestrian detection method based on improved YOLOv5s lightweight model

Abstract

The invention belongs to the technical field of image pedestrian detection, and particularly relates to a dense pedestrian detection method based on an improved YOLOv5s lightweight model. Comprising the following steps: acquiring a pedestrian image, and inputting the pedestrian image into an improved YOLOv5 model to obtain an initial dense pedestrian detection result; performing structured pruning on the improved YOLOv5 model according to the initial dense pedestrian detection result to obtain an intermediate improved YOLOv5 model; calculating the total loss of the model according to the initial dense pedestrian detection result, and adjusting the parameters of the intermediate improved YOLOv5 model according to the total loss of the model to obtain a trained dense pedestrian detection model; and screening the detection results of the trained dense pedestrian detection model by using a Soft-NMS algorithm to obtain a final dense pedestrian detection result. The pedestrian recognition accuracy can be improved, the problem of shielding between pedestrians can be effectively relieved, and the method can be operated on equipment with limited resources and has a good effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of image pedestrian detection, and in particular relates to a dense pedestrian detection method based on an improved YOLOv5s lightweight model. Background Art

[0002] As an important research topic in the field of target detection, pedestrian detection has wide applications in many engineering and technical fields such as visual tracking, pedestrian re-identification and behavior analysis. In recent years, with the rapid development of deep learning technology, a large number of related target detection algorithms have also emerged. Target detectors have also achieved good results, but most of these models require too much computing resources; this highlights the strong demand for lightweight model design. The main reason is that although existing algorithms have made remarkable achievements in pursuing high detection accuracy, they often ignore the model size and the computational burden it brings, which is precisely the key factor in judging the actual feasibility of the algorithm and whether it can be successfully applied to edge computing resource-constrained devices.

[0003] Under the framework of deep learning, object detection algorithms can be roughly classified into two schools according to their operating mechanisms: two-stage algorithms and one-stage algorithms. The former is typified by region-based convolutional neural networks. This type of algorithm is known for its high detection accuracy, but it is accompanied by a significant increase in model size and computational complexity, which may be limited in real-time applications such as autonomous driving and intelligent transportation systems. In contrast, one-stage detection algorithms simplify the detection process and directly predict the location of objects and their bounding box attributes from the input image, converting the detection task into a regression problem. Although these algorithms may be slightly inferior to the two-stage method in terms of accuracy, they are more streamlined and faster in execution, so they are widely used in the industry. In particular, the YOLO model stands out for its extremely high detection speed, showing an advantage of several orders of magnitude over most other object detection technologies. The core innovation of YOLO lies in its architectural design. The model outputs an object category probability and position offset for each preset anchor box at the same time, and only retains those bounding boxes whose category probabilities exceed the preset threshold as the detection results, so as to quickly and effectively locate the target object in the image, which is different from the traditional methods that rely on complex region proposals.

[0004] Most pedestrian detection tasks are usually tested and implemented on platforms equipped with high-performance graphics processing units (GPUs). However, in comparison, embedded devices have significant differences in computing power and cannot bear the burden of large-scale convolution operations, which means that GPU-driven solutions face implementation challenges in resource-constrained embedded environments. Therefore, it is a difficult problem to ensure model performance while ensuring that the model can be used on resource-constrained devices. Many improvements to the YOLO model series in the prior art have led to a slight increase in detection accuracy, but the problem of controlling the model size is still prominent, and the harmonious unity of lightweight and high accuracy has not been achieved satisfactorily. Considering the significant difference in computing performance between embedded systems and high-end servers, in real deployments, many edge computing devices and mobile devices are difficult to meet the resource requirements of cutting-edge deep learning models. Especially in places with large traffic, the frequent overlap between targets exacerbates the difficulty of pedestrian recognition and reduces detection accuracy.

[0005] In summary, a lightweight pedestrian detection model designed for resource-constrained devices is proposed, aiming to achieve efficient and real-time pedestrian recognition capabilities even in densely populated areas. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a dense pedestrian detection method based on an improved YOLOv5s lightweight model, the method comprising: obtaining a pedestrian image and inputting it into a trained dense pedestrian detection model for processing to obtain a dense pedestrian detection result;

[0007] The training process of the dense pedestrian detection model includes:

[0008] S1: Obtain pedestrian images, input the pedestrian images into the improved YOLOv5 model, and obtain the initial dense pedestrian detection results;

[0009] S2: Perform structured pruning on the improved YOLOv5 model based on the initial dense pedestrian detection results to obtain an intermediate improved YOLOv5 model;

[0010] S3: Calculate the total model loss based on the initial dense pedestrian detection results, adjust the intermediate improved YOLOv5 model parameters based on the total model loss, and obtain a trained dense pedestrian detection model;

[0011] S4: Use the Soft-NMS algorithm to filter the detection results of the trained dense pedestrian detection model to obtain the final dense pedestrian detection results.

[0012] Preferably, the improved YOLOv5 model includes a Backbone network, a Neck network and a Head network; the Backbone network consists of 2 one-dimensional convolutional layers, 4 C3_MLCA ​​layers and 3 LWWConv layers, and an SPPF layer that are alternately connected to each other; the Neck network includes 2 one-dimensional convolutional layers, 2 upsampling layers, 2 LWWConv layers, 4 C3 layers and 4 Concat layers; the Head network includes 3 Detect layers.

[0013] Furthermore, the calculation expression of the LWWConv layer is:

[0014] X avg =Conv 1×1 (AvgPool(X))

[0015]

[0016] F LWW_Max =WTConv 1×1 (WTConv 1×1 (MaxPool(X)))

[0017] F LWW =Softmax(F LWW_Avg +F LWW_Max )·GConv k×k

[0018] Among them, F LWW Represents the LWWConv layer output, Conv 1×1 Represents a convolution with a kernel size of 1, WTConv 1×1 represents WTConv with a convolution kernel size of 1, X represents the input feature map, AvgPool represents average pooling, and MaxPool represents maximum pooling. Represents X avg The i-th pixel in, N represents X avg The number of pixels in b l is the lth learnable visual codeword, s l is the lth scale factor, L is the total number of visual centers, Represents BN layer, GConv k×k represents a group convolution with a kernel size of k; X avg 、e k 、F LWW_Avg 、F LWW_Max Represent the first, second, third and fourth intermediate parameters respectively.

[0019] Preferably, the total model loss is the weighted sum of the classification loss and the regression loss; the classification loss is:

[0020]

[0021] Among them, VFL(p,q) represents classification loss, q represents binary variable, p represents the probability predicted by the model, γ represents the curve curvature control parameter; a represents the scale factor;

[0022] The regression loss is:

[0023]

[0024] Among them, L EIoU represents the regression loss, IoU represents the intersection over union ratio, b represents the center of the prediction box, and b gt represents the center point of the target box, ρ 2 (b,b gt ) represents the square of the Euclidean distance between the coordinates of the center point of the predicted box of the object and the target box of the object, ρ 2 (w,w gt ) represents the square of the width difference between the predicted box and the target box of the object, w represents the width of the predicted box, and w gt represents the width of the target box, ρ 2 (h,h gt ) represents the square of the height difference between the predicted box and the target box of the object, h represents the height of the predicted box, and h gt Indicates the height of the target box, w c Indicates the width of the minimum bounding rectangle that can be formed by the object's prediction box and the object's target box, h c Indicates the height of the minimum bounding rectangle that can be formed by the object's prediction box and the object's target box.

[0025] Preferably, the process of screening the initial dense pedestrian detection results includes: calculating the confidence score of the inspection box in the detection results of the trained dense pedestrian detection model; setting the confidence score threshold of the inspection box, removing the inspection box whose confidence score is lower than the confidence score threshold, and obtaining the final dense pedestrian detection result.

[0026] Furthermore, the formula for calculating the confidence score of the inspection box is:

[0027]

[0028] Among them, SN i Represents the confidence score of the inspection box, S i Represents the score of the i-th detection box output by the classic NMS algorithm, IoU(M,b i ) represents the intersection-over-union ratio between two test boxes, M represents the test box with the highest score, and b i represents the i-th inspection box, N trepresents the attenuation threshold, and α represents the hyperparameter of the Gaussian penalty function.

[0029] Preferably, the process of performing structured pruning on the intermediate improved YOLOv5 model includes: grouping network parameters and performing model pruning according to the grouping; calculating the pruning loss, and when the pruning loss converges, obtaining a trained dense pedestrian detection model.

[0030] The beneficial effects of the present invention are:

[0031] The present invention introduces a lightweight convolution LWWConv, which adaptively adjusts the importance of different spatial positions by introducing a spatial attention mechanism. At the same time, it uses a group convolution structure to divide the input channels into several groups to achieve the purpose of reducing the parameters and calculation amount of the model;

[0032] The present invention incorporates an efficient hybrid local channel attention (MLCA) module. This module has fewer parameters and lower computational complexity, which can effectively enhance the performance of the lightweight backbone network in feature extraction and enhance the overall detection accuracy of the YOLOv5 model;

[0033] The present invention uses the VariFocal-EIoU loss function, which significantly alleviates the imbalance problem of the positive and negative sample ratios during the training phase. Especially when performing bounding box regression, the VariFocal-EIoU loss function optimizes the learning focus by giving more attention to high-quality anchor boxes, thereby not only accelerating the convergence speed of the model, but also enhancing the adaptability and stability of the overall model to complex scenarios;

[0034] The present invention uses Soft-NMS to screen the detection frame, which can effectively improve the detection accuracy of the model without adding extra computational burden. This method has excellent performance in processing overlapping targets and occluded objects, so it can significantly reduce the occurrence of missed detection events while ensuring detection efficiency, and accelerate the overall calculation process;

[0035] The present invention uses structured pruning based on the DepGraph algorithm to compress the model. Neurons and filters with small weights or insignificant contributions to the final prediction are removed from the model, thereby achieving model slimming without significantly sacrificing detection performance. This not only reduces the computational burden and makes the model more suitable for edge devices with limited resources, but also improves the speed and efficiency of object detection in dense crowd scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of dense pedestrian detection model training in the present invention;

[0037] Figure 2This is a schematic diagram of the improved YOLOv5 model structure in the present invention;

[0038] Figure 3 This is a schematic diagram of the LWWConv structure in the improved YOLOv5 model in the present invention;

[0039] Figure 4 Schematic diagram of the mixed local channel attention mechanism in the present invention;

[0040] Figure 5 Schematic diagram of the GAP, LAP and UNAP processes in the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] This paper proposes a dense pedestrian detection method based on an improved YOLOv5s lightweight model. Figure 1 As shown, the method includes: obtaining a pedestrian image and inputting the image into a trained dense pedestrian detection model for processing to obtain a dense pedestrian detection result.

[0043] The training process of the dense pedestrian detection model includes:

[0044] S1: Obtain a pedestrian image and input the pedestrian image into the improved YOLOv5 model to obtain the initial dense pedestrian detection result.

[0045] The present invention realizes dense pedestrian detection based on the improved YOLOv5 model, namely the YOLOv5s-LMD model. The improved YOLOv5 model structure is as follows: Figure 2 As shown; the improved YOLOv5 model of the present invention includes a Backbone network, a Neck network and a Head network; the Backbone network consists of 2 one-dimensional convolutional layers, 4 C3_MLCA ​​layers and 3 LWWConv layers alternately connected to each other, and an SPPF layer (spatial pyramid pooling layer); the Neck network includes 2 one-dimensional convolutional layers, 2 upsampling layers, 2 LWWConv layers, 4 C3 layers (bottleneck layers) and 4 Concat layers; the Head network includes 3 Detect layers (detection layers).

[0046] The present invention designs a lightweight convolution LWWConv based on the spatial attention mechanism, such as Figure 3As shown, LWWConv is divided into two branches, namely, feature weight calculation and feature extraction. In the feature weight calculation branch, the first step is to compress the feature map. Using global average pooling alone may cause some local peak features to be averaged, thereby losing important detail information. Therefore, the present invention additionally introduces global maximum pooling as a supplement. Through these two pooling functions, two one-dimensional vectors can be obtained respectively. Avgpool has feedback for every global pixel on the feature map, and when the global maximum pooling performs gradient back propagation, only the position with the local maximum response in the feature map has gradient feedback, which can serve as a supplement to Avgpool. In the second step, the one-dimensional vector obtained after compression is processed with a 1x1 convolution, because the advantage of 1x1 convolution is that it can adjust the number of channels without changing the size of the input feature map, and there will be no excessive increase in parameters and calculation amount, and information interaction is carried out.

[0047] AvgPool is followed by an encoder with an intrinsic dictionary, which has two components: 1) Intrinsic codebook: B = {b1, b2, .., b L}, where N = H × W is the total number of input features, H and W represent the height and width of the feature map space respectively; 2) a set of scaling factors S = {s1, s2, .., s L} for learnable visual centers. Specifically, the feature map is first passed through Conv 1×1 and BR block processing, the BR block consists of a BN layer and a ReLU activation function. Through the above steps, the encoded features are input into the codebook. To this end, the present invention uses a set of scaling factors s to sequentially and b l Map the corresponding position information. The information of the entire image about the lth codeword can be calculated by formula; WTConv is used in MaxPool 1×1The module cooperates with the BR block to extract features. Maxpool adds a layer of WTConv module with a convolution kernel size of 1x1 on the basis of Avgpool. The WTConv (wavelet convolution) module is based on wavelet transform. Wavelet transform can decompose the feature map into sub-bands of different frequencies, which enables the model to capture low-frequency (smooth) and high-frequency (detail) information at the same time, which helps the model to better understand and process features at different levels at multiple scales, thereby further enhancing these local features and capturing more relevant detail information. Finally, softmax is used to emphasize the importance of each feature in the feature. In the feature extraction branch, the group convolution method is used instead of convolution. Group convolution greatly reduces the number of parameters by dividing the input channels into multiple groups and performing independent convolution operations in each group. Group convolution enhances the sparsity of the model by reducing the connections across groups, which helps the model better capture local features and reduce redundant calculations. The number of groups is set to ch / / ratio (ch is the input channel value). Experiments show that when ratio = 16, this division method ensures that there are enough channels in each group for effective feature extraction, while also reducing redundant calculations across groups. If the group is set too small, the effect of group convolution may not be obvious; if it is too large, cross-channel information interaction may be lost. The calculation expression of LWWConv is:

[0048] X avg =Conv 1×1 (AvgPool(X))

[0049]

[0050] F LWW_Max =WTConv 1×1 (WTConv 1×1 (MaxPool(X)))

[0051] F LWW =Softmax(F LWW_Avg +F LWW_Max )·GConv k×k

[0052] Among them, F LWW Represents the LWWConv layer output, Conv 1×1 Represents a convolution with a kernel size of 1, WTConv 1×1 represents WTConv with a convolution kernel size of 1, X represents the input feature map, AvgPool represents average pooling, and MaxPool represents maximum pooling. Represents X avgThe i-th pixel in, N represents X avg The number of pixels in the middle; b l is the lth learnable visual codeword, Represents the information of each pixel position relative to the codeword; s l is the lth scale factor, L is the total number of visual centers, Represents a BN layer with ReLU and mean layers, GConv k×k represents a group convolution with a kernel size of k; X avg 、e k 、F LWW_Avg 、F LWW_Max Represent the first, second, third and fourth intermediate parameters respectively.

[0053] A hybrid mechanism combining local and channel attention, also called the hybrid local channel attention (MLCA) mechanism, is added to the C3 layer to form the C3_MLCA ​​layer; Figure 4 As shown in the figure, the processing flow of the MLCA mechanism begins with two pooling steps on the input feature vector. First, the input feature map is converted into the form of 1*C*ks*ks. The purpose here is to capture local spatial information through local pooling. On this basis, the input data is processed through two branches: the first branch is used to obtain global context information, while the second branch focuses on refining local spatial features. Next, the two branches are respectively subjected to one-dimensional convolution operations, which helps to reduce computational complexity while retaining key features. After completing the one-dimensional convolution, the two vectors are restored to their original resolution through the inverse pooling process. Finally, the information from the two branches is integrated together, thereby realizing a mixture of global and local spatial attention, and improving the model's ability to capture multi-scale features.

[0054] Figure 4 In , Conv1d represents a one-dimensional convolution, where the convolution kernel size k is proportional to the channel dimension C. This means that when capturing local cross-channel interaction information, the mechanism only considers the relationship between each channel and its k adjacent channels. The specific choice of k is expressed by the following formula:

[0055]

[0056] Where k represents the size of the one-dimensional convolution kernel, C represents the number of channels of the input feature map, y represents the first preset parameter, b represents the second preset parameter, |·| odd It means that the default value of k is odd. If k is even, add 1.

[0057] Figure 5 Shown Figure 4The main features of GAP, LAP, and UNAP in . GAP is the process of performing global average pooling on the entire feature map. When the output size of adaptive pooling is set to 1, the feature map is compressed to a size of 1*1. When direct multiplication or addition operations with the source input are required, Expand or UNAP can be used for size expansion. LAP refers to local average pooling. Unlike GAP, LAP does not perform global pooling on the entire feature map, but divides the feature map into k*k patches and performs average pooling on each patch separately. This process can be completed with the output of k*k adaptive average pooling. It should be noted that when expanding the LAP result, the Expand operation cannot be used directly because its output size is not 1*1. Instead, the UNAP process must be used to restore the feature map to its original size. UNAP, or un-average pooling, is mainly used to adjust the size of the feature map on demand, and fills the values ​​in the patch according to the weights assigned to each patch. UNAP can be achieved by setting the output size of adaptive pooling to the same as the source feature map size.

[0058] Input the pedestrian image into the improved YOLOv5 model, and the predicted detection box (bounding box) and the detection box score, which are the initial dense pedestrian detection results, can be output.

[0059] S2: Perform structured pruning on the improved YOLOv5 model based on the initial dense pedestrian detection results to obtain the intermediate improved YOLOv5 model.

[0060] A major challenge of deep neural networks (DNNs) is that their huge computing requirements lead to an increase in the size of the model, which has become a major obstacle to the large-scale use of products based on deep learning technology, especially in the application of edge computing devices. Since edge devices usually do not have the ability to handle high computing loads, directly deploying complex DNN models will cause problems such as a surge in energy consumption and response delays, which restricts its practicality. Therefore, how to solve the problems of missing small target pedestrians and mutual occlusion between pedestrians when detecting dense pedestrians in complex background environments, while satisfying the need to deploy on resource-constrained devices to achieve high-precision detection of pedestrians, is an urgent need of the current market.

[0061] Structured pruning is an important model compression algorithm that reduces the number of parameters by removing redundant structures in neural networks, thereby reducing the time and space costs of model reasoning. In the neural network structure, each neuron is associated with multiple connections and corresponding parameter sets. When implementing neuron pruning to accelerate the network, all relevant parameters of the neuron must be removed collaboratively. These parameter sets constitute the basic unit of structured pruning. It is worth noting that depending on the network architecture, the organization of parameters is highly diverse and can be nested to form a more complex grouping layout. In traditional pruning practice, manually defining these parameter groupings is a cumbersome and inefficient task. It not only requires a high degree of manual intervention to deal with a time-consuming and tedious process, but also relies heavily on the designer's deep understanding of the network architecture and the precise grasp of the complex dependencies between layers. Therefore, parameter grouping has become one of the reasons why pruning cannot be implemented.

[0062] In order to solve the parameter grouping problem that traditional structured pruning cannot implement, this paper introduces the structured pruning of DepGraph graph algorithm to model the parameter dependencies in any network. In the structured pruning strategy, the parameters within the group are closely connected to each other. If any parameter is to be removed, all other parameters in the group must also be removed to maintain the integrity and correctness of the network structure. In order to ideally manage this coupling relationship, a binary grouping matrix G is created to record all relationships. This matrix can systematically map and represent this inseparable pairing relationship between all parameter pairs, expressed as g(i)={j|G ij =1}, where if the parameters of the i-th layer and the j-th layer are coupled to each other, G ij =1 to indicate.

[0063] However, the interdependence between parameters does not exist in isolation, but is also deeply affected by the intermediate hierarchical structure and the upper and lower layers where these parameters are located. Because there may be two different pruning methods for the same layer, the structured pruning f of each layer i Divided into input pruning f i - and output pruning f i + A simple network can be represented as The local dependency relationship between adjacent layers is called the dependency graph D, which is expressed as:

[0064]

[0065] Among them, the symbol represents network connection, ∨ and ∧ represent logical "OR" and "AND" respectively. When the input and output of a layer can be pruned independently, they are recorded as sch(f i- ) and sch(f i + ).

[0066] After the parameters of the network are grouped by the DepGraph algorithm, an importance evaluation criterion is introduced to evaluate the importance of the parameters. The batch normalization (BN) layer uses the statistical information of the mini-batch to normalize the activation output of the neuron. The output result undergoes a specific scaling and translation operation. The scaling factor γ′ plays a key role here because it determines the extent to which the normalized result can be magnified or reduced, which indirectly reflects the importance of each channel. In the training phase, a regularization penalty designed to induce sparsity is incorporated into the loss function. This penalty is designed based on the L1 norm of γ′ (the sum of the absolute values ​​of all γ values), and its purpose is to encourage the network to learn to identify and mark those unimportant channels so that their corresponding γ′ values ​​are close to zero. Finally, according to the pre-set pruning ratio, channels with small γ′ values ​​are removed from the network.

[0067] Therefore, the network is pruned according to the grouping; the pruning loss is calculated. When the pruning loss converges, the trained dense pedestrian detection model is obtained, and the pruning loss is:

[0068]

[0069] Wherein, L represents the pruning loss, f(x,W) represents the network prediction output, (x,y) represents the training input and target, W represents the trainable weight in the network, λ′ represents the balance factor, which is used to weigh the relationship between normal training loss and regularization, γ′ represents the scaling factor, Γ represents the set of all channel scaling factors, where each channel has a scaling factor y, l(f(x,W),y) represents the task loss during training, that is, the total loss calculated using the loss function of the original YOLOv5 model based on the initial dense pedestrian detection results; g(γ′) is the sparsity penalty term imposed on the scaling factor γ′. In the present invention, g(s)=|s|, i.e., the L1 norm, is selected to promote the sparse distribution of γ′ values.

[0070] S3: Calculate the total model loss based on the initial dense pedestrian detection results, adjust the intermediate improved YOLOv5 model parameters based on the total model loss, and obtain a trained dense pedestrian detection model.

[0071] The total loss of the model of the present invention is the weighted sum of the classification loss and the regression loss, where the regression loss (EIoUloss) is:

[0072]

[0073] Among them, the EIoU loss function is composed of three types of losses, namely overlap loss, center point distance loss and width and height loss. The width and height loss is taken into account by the EIoU loss function, which can achieve faster convergence speed and thus detect more information; L EIoU represents regression loss, IoU represents intersection over union, ρ 2 (b,b gt ) represents the square of the Euclidean distance between the coordinates of the center point of the predicted box of the object and the target box of the object, ρ 2 (w,w gt ) represents the square of the width difference between the predicted box of the object and the target box of the object, ρ 2 (h,h gt ) represents the square of the height difference between the predicted box of the object and the target box of the object, w c Indicates the width of the minimum bounding rectangle that can be formed by the object's prediction box and the object's target box, h c represents the height of the smallest enclosing rectangle that can be formed by the prediction box and the target box of the object; b represents the center of the prediction box, b gt represents the center point of the target box, w represents the width of the prediction box, and w gt represents the width of the target box, h represents the height of the prediction box, and h gt Indicates the height of the target box.

[0074] The classification loss (VariFocal Loss) is:

[0075]

[0076] Among them, VFL(p,q) represents the classification loss, q represents a binary variable used to distinguish positive samples from negative samples, and p represents the probability predicted by the model; γ represents the curve curvature control parameter, and when γ=0.5, the model can achieve the best trade-off; a represents an adjustable proportional factor to balance the loss between positive examples and negative examples.

[0077] The YOLOv5 model parameters are adjusted and improved according to the total model loss, that is, back propagation is performed according to the total model loss. When the loss converges or reaches the maximum number of iterations, the model parameters are saved to obtain a trained dense pedestrian detection model.

[0078] In crowded pedestrian scenes, target pedestrians frequently overlap and occlude each other. The present invention uses VariFocal-EIoU Loss (classification loss and regression loss), which can more effectively guide model learning based on EIoU, reduce excessive attention to easy-to-classify samples similar to the background, and enhance the ability to distinguish targets that are severely overlapped and difficult to distinguish. It not only improves the accuracy of dense target detection, but also enhances the robustness of the model in the face of occlusion and overlap challenges, ultimately promoting a significant improvement in pedestrian detection performance in crowded environments.

[0079] S4: Use the Soft-NMS algorithm to filter the detection results of the trained dense pedestrian detection model to obtain the final dense pedestrian detection results.

[0080] In a complex environment with dense pedestrians, how to solve the occlusion and overlap between pedestrians is one of the key issues to be solved in this study. During the training stage of the target detection model, a frequent situation is that a large number of overlapping prediction bounding boxes are generated, and the number of these boxes far exceeds the real box count of the object in the actual image. When there are multiple scores exceeding the threshold in adjacent prediction boxes, simply clearing the confidence of all boxes below the highest score and with an overlap exceeding the set threshold may inadvertently eliminate some prediction boxes with higher scores that should actually be retained. This approach is very likely to cause missed detection (i.e., failure to identify actual targets) and false detection (erroneously predicting non-existent targets), which ultimately has an adverse effect on the overall detection accuracy. Therefore, the present invention uses the Soft-NMS algorithm to screen the detection results of the trained dense pedestrian detection model. Soft-NMS is to calculate the score of each detection box. If its IoU with the marked retained box exceeds the threshold, it is not directly removed, but its confidence (score) is proportionally reduced according to the size of the IoU. Since the confidence is gradually decayed instead of being directly set to zero, even boxes with a lot of overlap may still be retained as long as their initial confidence is high enough, thereby reducing missed detections.

[0081] The Soft-NMS algorithm introduces the Gaussian penalty function as a solution. This function can smoothly increase the penalty strength. As the overlap degree M between the detection frame and the reference frame increases, the penalty strength imposed also increases accordingly, thereby avoiding the fault phenomenon and achieving a smooth transition in the prediction and screening process.

[0082] The process of screening the detection results of the trained dense pedestrian detection model includes: calculating the confidence score of the inspection box in the results of the trained dense pedestrian detection model; setting the confidence score threshold of the inspection box, removing the inspection box whose confidence score is lower than the confidence score threshold, and obtaining the final dense pedestrian detection result.

[0083] The formula for calculating the confidence score of the inspection box is:

[0084]

[0085] Among them, SN i Represents the confidence score of the inspection box, S i Represents the score of the i-th detection box output by the classic NMS algorithm, IoU(M,b i ) represents the intersection-over-union ratio between two bounding boxes, M represents the detection box with the highest score, and b i represents the i-th inspection box, N t represents the decay threshold, which determines how much overlap will cause the bounding box score to decay, and the Gaussian penalty function The hyperparameter α is tuned to 0.5.

[0086] The present invention uses the Soft-NMS algorithm with a set penalty strategy, which can more effectively alleviate the problem of missed detection caused by overlapping areas and improve the accuracy and stability of dense pedestrian detection in complex scenes.

[0087] The present invention can effectively improve the efficiency of dense pedestrian detection by reducing redundancy and unnecessary calculations in the neural network through pruning. It reduces the complexity of the model, especially in scenes with frequent target overlap, and improves detection accuracy and real-time performance by removing feature map channels that contribute little to detection, so that dense crowds can be effectively identified even on resource-constrained devices.

[0088] After obtaining the trained dense pedestrian detection model, the pedestrian image to be detected is obtained and input into the trained dense pedestrian detection model for processing, so that the dense pedestrian detection result can be obtained efficiently and quickly.

[0089] In summary, the present invention designs the YOLOv5s-LMD model, and adds the mixed local channel attention (MLCA) module to the original YOLOv5 model, in order to enhance the ability to extract the feature information of small target pedestrians when detecting images, and effectively reduce the situation of missed detection. It is proposed to replace the loss function of YOLOv5 with the VariFocal-EIoU loss function. First, the EloU loss function is used, which divides the error of detection frame positioning into regional area loss and overlap loss, and VariFocal-EIoU Loss reduces the excessive attention to easy-to-classify samples similar to the background on the basis of EIoU, while strengthening the ability to distinguish those targets with serious overlap and difficult to distinguish. The non-maximum suppression Soft-NMS algorithm is used, and the penalty strategy set can more effectively reduce the problem of missed detection caused by overlapping areas. Finally, the structured pruning based on DepGraph is used to identify unimportant channels in the network, and then the channels are pruned according to the value of the pruning ratio, so as to achieve the purpose of compressing the model. Compared with the original YOLOv5s model, the YOLOv5s-LMD model can improve the accuracy of pedestrian recognition, effectively alleviate the occlusion problem between pedestrians and the problem of missed detection of small-target pedestrians. In addition, through the pruning method, the YOLOv5s-LMD model can run on resource-constrained devices and achieve good results.

[0090] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A dense pedestrian detection method based on an improved YOLOv5s lightweight model, characterized in that: include: Obtain pedestrian images and input them into the trained dense pedestrian detection model for processing to obtain dense pedestrian detection results; The training process of the dense pedestrian detection model includes: S1: Obtain pedestrian images, input the pedestrian images into the improved YOLOv5 model, and obtain the initial dense pedestrian detection results; S2: Perform structured pruning on the improved YOLOv5 model based on the initial dense pedestrian detection results to obtain an intermediate improved YOLOv5 model; S3: Calculate the total model loss based on the initial dense pedestrian detection results, adjust the intermediate improved YOLOv5 model parameters based on the total model loss, and obtain a trained dense pedestrian detection model; S4: Use the Soft-NMS algorithm to filter the detection results of the trained dense pedestrian detection model to obtain the final dense pedestrian detection results.

2. According to claim 1, a dense pedestrian detection method based on an improved YOLOv5s lightweight model is characterized in that: The improved YOLOv5 model includes a Backbone network, a Neck network and a Head network; the Backbone network consists of two one-dimensional convolutional layers, four C3_MLCA ​​layers and three LWWConv layers and an SPPF layer that are alternately connected to each other; the Neck network includes two one-dimensional convolutional layers, two upsampling layers, two LWWConv layers, four C3 layers and four Concat layers; the Head network includes three Detect layers.

3. According to claim 2, a dense pedestrian detection method based on an improved YOLOv5s lightweight model is characterized in that: The calculation expression of the LWWConv layer is: X avg =Conv 1×1 (AvgPool(X)) F LWW_Max =WTConv 1×1 (WTConv 1×1 (MaxPool(X))) F LWW =Softmax(F LWW_Avg +F LWW_Max )·GConv k×k Among them, F LWW Represents the LWWConv layer output, Conv 1×1 Represents a convolution with a kernel size of 1, WTConv 1×1 represents WTConv with a convolution kernel size of 1, X represents the input feature map, AvgPool represents average pooling, and MaxPool represents maximum pooling. Represents X avg The i-th pixel in, N represents X avg The number of pixels in b l is the lth learnable visual codeword, s l is the lth scale factor, L is the total number of visual centers, Represents BN layer, GConv k×k represents a group convolution with a kernel size of k; X avg 、e k 、F LWW_Avg 、F LWW_Max Represent the first, second, third and fourth intermediate parameters respectively.

4. According to claim 1, a dense pedestrian detection method based on an improved YOLOv5s lightweight model is characterized in that: The total loss of the model is the weighted sum of classification loss and regression loss; the classification loss is: Among them, VFL(p,q) represents classification loss, q represents binary variable, p represents the probability predicted by the model, γ represents the curve curvature control parameter; a represents the scale factor; The regression loss is: Among them, L EIoU represents regression loss, IoU represents intersection over union, b represents the center point of the prediction box, and b gt represents the center point of the target box, ρ 2 (b,b gt ) represents the square of the Euclidean distance between the coordinates of the center point of the predicted box of the object and the target box of the object, ρ 2 (w,w gt ) represents the square of the width difference between the predicted box and the target box of the object, w represents the width of the predicted box, and w gt represents the width of the target box, ρ 2 (h,h gt ) represents the square of the height difference between the predicted box and the target box of the object, h represents the height of the predicted box, and h gt Indicates the height of the target box, w c Indicates the width of the minimum enclosing rectangle that can be formed by the predicted bounding box of the object and the target box of the object, h c Represents the height of the minimum enclosing rectangle that can be formed by the predicted bounding box of the object and the target box of the object.

5. The dense pedestrian detection method based on the improved YOLOv5s lightweight model according to claim 1 is characterized in that: The process of screening the initial dense pedestrian detection results includes: calculating the confidence score of the inspection box in the detection result of the trained dense pedestrian detection model; setting the confidence score threshold of the inspection box, removing the inspection box whose confidence score is lower than the confidence score threshold, and obtaining the final dense pedestrian detection result.

6. The dense pedestrian detection method based on the improved YOLOv5s lightweight model according to claim 5 is characterized in that: The formula for calculating the confidence score of the inspection box is: Among them, SN i Represents the confidence score of the inspection box, S i Represents the score of the i-th detection box output by the classic NMS algorithm, IoU(M,b i ) represents the intersection-over-union ratio between two test boxes, M represents the test box with the highest score, and b i represents the i-th inspection box, N t represents the attenuation threshold, and α represents the hyperparameter of the Gaussian penalty function.

7. The dense pedestrian detection method based on the improved YOLOv5s lightweight model according to claim 1 is characterized in that: The process of structured pruning of the intermediate improved YOLOv5 model includes: grouping network parameters and pruning the model according to the grouping; calculating the pruning loss, and when the pruning loss converges, a trained dense pedestrian detection model is obtained.

Images