Lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort

Abstract

The invention discloses a lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort. According to the method, an improved YOLOv10n model is adopted as a detector to obtain a target detection result in a video scene. The influence of outdoor weather on a detection result is considered, and the detection result is more accurate under a complex background by fusing an attention feature fusion structure DC-FFA of dynamic convolution. A DeepSort multi-target tracking model is fused, pedestrians are tracked through Hungary matching and Kalman filtering, an SEAMNet36 pedestrian re-recognition network is designed, and appearance features of the pedestrians can be better extracted. In the counting aspect, the counting algorithm based on biconvex trajectory measurement and calculation can count the number of pedestrians in the picture and calculate the number of pedestrians entering and leaving the monitoring picture. According to the method, the model parameter quantity is reduced while the tracking accuracy is improved, the stronger apparent feature extraction capability is realized through a lighter network, the problems of missing detection, false detection, too many ID hopping times and the like in the prior art are solved, and the detection and tracking effects are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and pedestrian detection, and in particular to a scenic area pedestrian detection, tracking and counting system based on improved YOLOv10n and DeepSort. Background Art

[0002] The study of pedestrian counting in scenic spots is an important technical means to promote the intelligent management of scenic spots, and has a far-reaching impact on improving tourist experience, optimizing scenic spot resource allocation, and ensuring safety. Through accurate pedestrian counting technology, managers can grasp the dynamics of passenger flow in scenic spots in real time, reasonably arrange security personnel, cleaning services and tourist guidance, and ensure the safety of pedestrians in scenic spots, providing reliable data support for scenic spot planning and operation decisions. This data-driven management method can not only improve the operational efficiency of scenic spots, but also provide a basis for the improvement of economic benefits, thereby promoting the long-term prosperity and stable development of scenic spots. Pedestrians in scenic spots may be difficult to detect accurately due to changes in clothing, posture, angle, weather, lighting and other external conditions. Traditional detection methods often show low robustness in the face of these changes. With the rapid development of computer hardware performance, artificial intelligence technology based on deep learning has made significant progress in the field of pedestrian detection. Deep learning models can automatically extract high-level features from massive data, get rid of the dependence of traditional methods on manually designed features, and better adapt to the diverse and complex pedestrian behaviors and environmental changes in scenic spots. Through training in a large number of different scenarios, the deep learning model can automatically learn how to distinguish pedestrians from the background, and has stronger adaptability and detection accuracy. The application of these new technologies provides more efficient and accurate tools for the intelligent management of scenic spots, which is of great significance to improving the overall management level of scenic spots, ensuring the safety of pedestrians in scenic spots, and promoting the continued prosperity of the tourism industry.

[0003] In the current deep learning, YOLOv10n is the latest generation of object detection models in the YOLO series, continuing the series' superiority in real-time detection. Although YOLOv10n has good detection effects, there are still some problems in identifying pedestrians in complex scenes. For example, pedestrian counting in scenic spots not only needs to deal with complex dynamic backgrounds, such as trees, buildings, other tourists or moving objects in scenic spots, but also needs to achieve high-precision counting in a variety of environments, such as rainy days, foggy days and crowded scenes, to ensure the stability of detection results. Summary of the invention

[0004] In view of the above problems, the present invention discloses a lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort. The present invention integrates the improved YOLOv10n model and the improved DeepSort multi-target tracking algorithm. Under the premise of ensuring the accuracy of pedestrian detection, it can effectively reduce model parameters and achieve accurate counting, thereby solving the problems of missed detection and false detection caused by complex scenic environment background and dense crowds.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is to improve the lightweight scenic spot pedestrian counting system based on YOLOv10n and DeepSort, which is implemented in the following steps:

[0006] S1: Obtain pedestrian images in crowded places such as scenic spots, shopping malls, and streets to build a dataset;

[0007] S2: Perform data enhancement on pedestrian images in the data set to obtain a data enhanced data set;

[0008] S3: Build the YOLOv10n pedestrian detection network model;

[0009] S4: Improve the YOLOv10n model to obtain an improved YOLOv10n model;

[0010] S5: Use the data-enhanced dataset obtained in S2 to train the improved YOLOv10n network structure to obtain a trained pedestrian detection model;

[0011] S6: Send the pedestrian image to be detected into the trained YOLOv10n model to obtain the pedestrian's position information and detection box information;

[0012] S7: improving the DeepSort model to obtain an improved DeepSort model;

[0013] S8: Send the information detected by S5 to the improved DeepSort model to obtain the sequence number of the pedestrian target box;

[0014] S9: The number of pedestrians entering and leaving is obtained through a counting algorithm based on double convex track measurement;

[0015] The present invention is also characterized in that

[0016] S1 is specifically as follows: a data set is constructed by acquiring pedestrian images using a Canon 800D camera under normal lighting conditions. The constructed data set is divided into a training set, a validation set, and a test set in an 8:1:1 manner. The pedestrian images in the data set are then finely annotated using the Makesense online annotation tool. If the pedestrian image contains a pedestrian, the pedestrian is marked as people.

[0017] S2 specifically includes: performing image enhancement methods such as image cropping, translation, rotation, and changing image brightness on the pedestrian images in the data set, enhancing the quantity of each pedestrian image, and expanding the original one pedestrian image to four.

[0018] S3 is specifically as follows: constructing a YOLOv10n network model, whose overall structure consists of four parts, which are connected in this way to complete the target detection task; first, the input end includes functions such as mosaic, data enhancement and adaptive anchor frame calculation; next is the backbone network Backbone adopts the CSPDarknet architecture, which is composed of multiple Conv, C2f, SCDown, SPPF and other modules, which are used to extract image features; followed by the Neck part, the neck network consists of two parts: the feature pyramid network FPN and the path aggregation network PAN, which are used to fuse the high-level and low-level features; finally, the Head part is used to generate the prediction results of target detection.

[0019] S4 is specifically as follows: adding the attention feature fusion structure DC-FFA based on lightweight dynamic convolution to the backbone network Backbone of the original YOLOv10n. This structure can retain shallow information and pass it to the deep layer without increasing the number of model parameters, thereby enhancing the representation ability of the model. It can not only fuse all features, but also adaptively learn different weights of feature information at different levels; using the DC-FFA structure as the first layer of the improved model network structure, and replacing the original SPPF module in the last layer of Backbone with the SimSPPF module. The SimSPPF module reduces unnecessary computational complexity by simplifying operations, optimizes the calculation and storage operations in the feature fusion process, reduces the overall weight of the network, and reduces the computational complexity and improves the inference speed of the model; finally, the Concat module of the Neck part FPN is integrated into the repeatedly weighted bidirectional feature pyramid network BiFPN to obtain the Concat_BiFPN module, allowing features to flow bidirectionally between different scales in the network.

[0020] S5 is as follows: the training set is input into the improved YOLOv10n model for training; and the training parameters are set as follows: learning rate = 0.001, momentum = 0.937, input image size is 640×640, training batch is 8, number of training rounds is 300, SGD optimizer is used, and Mosaic image enhancement is turned off in the last 10 rounds.

[0021] Furthermore, the improved YOLOv10n model after training is evaluated by the recall rate, precision rate and average precision mean, and the evaluation formulas are respectively:

[0022] Accuracy:

[0023] Recall:

[0024] Average Precision: AP = ∫0 1 P(R)dr

[0025] Mean average precision:

[0026] In the formula: TP represents the number of samples correctly judged as positive, FP represents the number of samples incorrectly judged as positive, FN represents the number of samples incorrectly judged as negative, n represents the total number of categories, AP represents the area enclosed by the PR curve, and mAP represents the mean AP of all categories.

[0027] S7 is specifically as follows: First, a new pedestrian re-identification network SEAMNet36 is designed to replace the original pedestrian re-identification network. The core innovation of SEAMNet36 lies in its structural optimization, which upgrades the original three-layer BasicBlock to 5 layers, which can better capture the subtle differences in pedestrian appearance, thereby improving the recognition ability in complex scenes. In addition, its BasicBlock integrates the SEAM attention mechanism, which enables the network to capture the contextual appearance information of each pixel, further enhancing the model's ability to distinguish similar pedestrians. Finally, the original IOU matching strategy is replaced by a more advanced Powerful-IOU matching. This new method significantly improves the accuracy and robustness of target matching, especially in scenes with partial occlusion and dense pedestrians, where Powerful-IOU can better handle complex matching situations. By introducing a more flexible matching strategy, this method performs well in re-identification tasks and ensures reliability and stability in changing environments.

[0028] S8 is specifically as follows: according to the pedestrian detection frame of the first frame in S6, an initial trajectory is created and an initial ID is set; the position of the pedestrian detection frame of the next frame is predicted using Kalman filtering to obtain a pedestrian prediction frame; cascade matching and Powerful-IOU matching are performed on the pedestrian detection frame of the next frame and the pedestrian prediction frame to obtain a matching result; and the ID is updated according to the matching result.

[0029] Furthermore, the improved DeepSort model after training is evaluated by evaluation indicators such as multi-target tracking accuracy MOTA, multi-target tracking precision MOTP and number of encoding transformations IDS. The evaluation formulas are:

[0030] Multi-target tracking accuracy:

[0031] Multi-target tracking accuracy:

[0032] In the formula: N FN Represents the number of undetected pedestrians, N FP represents the number of samples that are misdetected as pedestrians, N IDS represents the number of identity switches in a trajectory, N GT represents the total number of targets, d t,i represents the average distance between target i and the labeled position in the dataset, c t Indicates the number of successful matches of t frames.

[0033] S9 is as follows: first, draw two irregular detection lines, a blue line and a yellow line, on the video; second, compare the pedestrian position coordinates with the coordinates of the two lines continuously, and count when the coordinates of the monitoring point cross the two detection lines; finally, if the pedestrian position coordinates first cross the blue line and then the yellow line, it is determined to have entered the monitoring area, and the number of pedestrians increases by 1. If the pedestrian position coordinates first cross the yellow line and then the blue line, it is determined to have left the monitoring area, and the number of pedestrians decreases by 1.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The present invention significantly improves the performance of pedestrian detection and counting by deeply integrating the improved YOLOv10n model with the optimized DeepSort multi-target tracking algorithm. Compared with the existing technology, this solution not only effectively reduces the size of the model and improves the detection speed, but also significantly improves the detection accuracy and counting accuracy of pedestrians. In a complex scenic environment, the system can realize real-time and accurate counting of pedestrians, monitor the dynamics of human flow in real time, and identify congestion in time, greatly reducing the risk of stampede accidents caused by too many people. In addition, the improved algorithm has stronger anti-interference ability and can maintain stable recognition performance under the influence of lighting changes, occlusion and other complex factors. The application of this technology not only improves the safety management level of scenic spots, but also provides tourists with a safer and more comfortable tour experience, laying a solid foundation for the development of smart cities and smart tourism. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The overall flow chart of the pedestrian counting method provided by the present invention

[0037] Figure 2 Schematic diagram of some data sets provided by the present invention

[0038] Figure 3 The network structure diagram of the improved YOLOv10n provided by the present invention

[0039] Figure 4 Schematic diagram of the DC-FFA module structure provided by the present invention

[0040] Figure 5 Schematic diagram of the DynamicConvolution dynamic convolution structure provided by the present invention

[0041] Figure 6 Schematic diagram of the SimSPPF module structure provided by the present invention

[0042] Figure 7 Schematic diagram of the BiFPN structure provided by the present invention

[0043] Figure 8 Schematic diagram of the SEAMNet36 structure provided by the present invention

[0044] Fig. 9 Schematic diagram of the SEAM attention mechanism structure provided by the present invention

[0045] Fig.10 Flowchart of DeepSort algorithm provided by the present invention

[0046] Fig.11 Schematic diagram of the Powerful-IOU loss function provided by the present invention

[0047] Fig.12 Schematic diagram of system detection results provided by the present invention DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The embodiment of the present invention discloses a lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort. The system flow chart is as follows: Figure 1 To make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0050] Current target detection algorithms are mainly divided into two categories: one-stage detection algorithms and two-stage detection algorithms. One-stage target detection algorithms include YOLO, SSD, etc., and two-stage target detection models include R-CNN, Faster R-CNN, etc. Since the one-stage detection algorithm realizes end-to-end prediction, it is usually better than the two-stage algorithm in detection speed, but its accuracy is slightly lower than the two-stage algorithm. Given that the task of this article requires high real-time performance, the typical model YOLOv10 of the YOLO series is selected as the benchmark model. YOLOv10 is the latest version of the current YOLO series, which further improves the performance of the model in target detection tasks.

[0051] In the field of multi-target tracking, the DeepSORT algorithm has become an indispensable core tool due to its stability and efficiency, and its performance is highly dependent on the accuracy of the target detection algorithm. Therefore, the combination of YOLOv10n and DeepSort can not only achieve efficient multi-target tracking, but also ensure the accuracy of real-time detection. In practical applications, the combination of YOLOv10n and DeepSORT has formed a super strong effect, showing good complementarity and synergy, making the real-time counting of pedestrians more accurate and efficient.

[0052] Specifically, the technical solution adopted by the present invention is to implement a lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to the following steps:

[0053] S1: Obtain pedestrian images in crowded places such as scenic spots, shopping malls, and streets to build a dataset;

[0054] S2: Perform data enhancement on pedestrian images in the data set to obtain a data enhanced data set;

[0055] S3: Build the YOLOv10n pedestrian detection network model;

[0056] S4: Improve the YOLOv10n model to obtain an improved YOLOv10n model;

[0057] S5: Use the data-enhanced dataset obtained in S2 to train the improved YOLOv10n network structure to obtain a trained pedestrian detection model;

[0058] S6: Send the pedestrian image to be detected into the trained YOLOv10n model to obtain the pedestrian's position information and detection box information;

[0059] S7: improving the DeepSort model to obtain an improved DeepSort model;

[0060] S8: Send the information detected by S5 to the improved DeepSort model to obtain the sequence number of the pedestrian target box;

[0061] S9: The number of pedestrians entering and leaving is obtained through a counting algorithm based on double convex track measurement;

[0062] The following is a detailed explanation of each step:

[0063] In this implementation, Figure 2 For some dataset images created for this article, step S1 of constructing the dataset can be implemented in the following ways:

[0064] S11: In this study, a Canon 800D camera was used to shoot multiple scenes and multiple time periods in crowded places such as scenic spots, shopping malls, and railway stations, capturing pedestrian photos in different environments and time points. Through the shooting strategy of different time periods and scenes, a total of 6,849 high-quality image data were obtained. These photos have rich scene changes and crowd density differences, providing a variety of samples for subsequent target detection and tracking algorithm testing, ensuring the wide applicability and reliability of the experimental results.

[0065] S12: In the data processing phase, the Makesense online annotation tool was used to finely annotate all images. In the specific operation, all pedestrians in each picture were selected with the mouse to ensure that each pedestrian was completely enclosed in a rectangular frame and its boundaries were accurately marked. In order to unify the labels, "person" was assigned to each frame as the target label. This process strictly annotated every detail of the image to ensure the accuracy and consistency of the data. After the annotation was completed, the export path and format were set to export all annotation results as XML format files. In the end, 6849 annotation files with the suffix xml were generated for 6849 pictures. These XML files contain the position coordinates and label information of the pedestrian box in each picture, providing standardized and high-quality training data for subsequent target detection and tracking algorithms, ensuring the reliability and accuracy of model training.

[0066] S13: Since the annotation files with the suffix txt are used in model training, the annotation file format needs to be converted to convert the files with the suffix xml to the files with the suffix txt. Use Python program scripts for batch conversion, use importos, importglob and importxml.etree.ElementTree to import related library files, traverse all xml files in the specified folder, convert the pixel coordinates in the xml file to the normalized coordinates in txt, and the pedestrian type is represented by the number 0.

[0067] Specifically, each line of the txt file indicates the order of target category, target center point x, target center point y, target width, and target height. The conversion formula from the xml file coordinates to the txt coordinates is as follows:

[0068] Target center point:

[0069] Target center point:

[0070] Target width:

[0071] Aim high:

[0072] In the formula, (x, y) represents the normalized coordinates of the center point of the box target, w represents the normalized width of the box target, h represents the normalized height of the box target, (x min ,y min ) represents the coordinates of the upper left corner of the box, (x max ,y max) represents the coordinates of the lower right corner of the box, width represents the width of the entire image, and height represents the height of the entire image.

[0073] In this implementation, step S2 can be implemented in the following manner:

[0074] In this example, geometric transformation technology is used to enhance the data to improve the robustness and generalization ability of the model. Geometric transformations include flipping, rotation, translation, cropping, deformation, and scaling. These operations can effectively increase the diversity of training data, thereby helping the model to better adapt to different visual scenes and conditions. Use Python program scripts for data enhancement. The transforms module of the torchvision library provides a variety of data enhancement methods, including horizontal flipping, vertical flipping, random cropping, etc.

[0075] In this implementation, step S3 can be implemented in the following manner:

[0076] The YOLOv10n network model is constructed. Its overall structure consists of four parts, which are connected in this way to complete the target detection task; first, the input end includes functions such as mosaic, data enhancement and adaptive anchor frame calculation; next is the backbone network Backbone adopts the CSPDarknet architecture, which is composed of multiple Conv, C2f, SCDown, SPPF and other modules, which are used to extract image features; followed by the Neck part, the neck network consists of two parts: the feature pyramid network FPN and the path aggregation network PAN, which are used to fuse the high-level and low-level features; finally, the Head part is used to generate the prediction results of target detection.

[0077] In this implementation, Figure 3 For the improved YOLOv10n structure diagram, step S4 can be implemented in the following way:

[0078] S41: Figure 4 As shown in the figure, this figure is a structural diagram of the attention feature fusion structure DC-FFA based on lightweight convolution, which is an end-to-end feature fusion attention network. Figure 4 It can be seen that there are three block structures in the network. A basic block structure consists of local residual learning and feature attention, and the feature attention FA module combines the channel attention (ChannelAttention) and pixel attention (PixelAttention) mechanisms. Considering that the weight information contained in different channel features is completely different and the haze is unevenly distributed on different image pixels, FA treats different features and pixels specially, providing additional flexibility for processing different types of information and expanding the representation ability of CNN.

[0079] Traditional convolution operations extract local features based on fixed-size convolution kernels, and the same convolution kernel is applied to all data sets, which may lead to incomplete information capture in some cases. And if traditional convolution wants to improve the performance of the model, it must expand the depth, width, resolution, etc. of the model. To solve this problem, this paper proposes to replace the convolution of the feature attention FA module with dynamic convolution DynamicConvolution. The principle diagram of dynamic convolution is shown in the figure below. Figure 5 As shown in the figure. The core idea of ​​dynamic convolution is to enable the parameters of the convolution kernel to be dynamically adjusted according to the input data. For each input data, dynamic convolution will calculate a set of specific convolution kernel parameters based on the characteristics of the data and use these parameters to perform convolution operations. In this way, for different input data, dynamic convolution will use different convolution kernel parameters, thereby improving the representation ability of the model.

[0080] exist Figure 5 In the convolutional kernel, weighted processing is performed, which is specifically manifested in using different convolutional kernels for different inputs, and then paying attention to these different convolutional kernels. The Conv1 convolutional module contains elements such as w1, b1, and the Conv2 convolutional module contains elements such as w2, b2, and so on. The ConvK convolutional module contains w k ,b k Different w represents different weight dynamics. The formula for dynamic convolution is as follows:

[0081] Dynamic convolution: y = g(w T (x)x+b(x))

[0082] in, The scope is 0≤π k (x)≤1,

[0083] S42: Figure 6 As shown in the figure, this is the structure diagram of the SimSPPF module, which includes convolutional layers, pooling layers, and splicing layers. The input feature map is first extracted through the convolutional layer, then multi-scale pooling is performed through the pooling layer, and finally all pooling results and the original feature map are spliced, and then feature fusion is performed through a convolutional layer. SimSPPF is only different from SPPF in one activation function. The SiLU activation function in SPPF is replaced by the ReLU activation function. The formulas of the SiLU activation function and the ReLU activation function are:

[0084] SiLU activation function: y = x × sigmoid(x)

[0085] ReLU activation function: y = max(0,x)

[0086] Among them, the output range of the sigmoid activation function is limited to (0,1). It is equivalent to a normalization operation.

[0087] S43: Figure 7 As shown in the figure, this is the structural diagram of the BiFPN module of the repeated weighted bidirectional feature pyramid network. BiFPN is added to the Concat module. BiFPN introduces a bidirectional connection mechanism, that is, a bottom-up and top-down feature fusion path, which helps to establish connections between features of different scales. BiFPN optimizes cross-scale connections by removing nodes with only one input edge, adding additional edges between input and output nodes at the same level, and treating each bidirectional path as a feature network layer. Through this bidirectional fusion, the Concat module can make more full use of the information of low-level and high-level features and improve the detection ability of multi-scale targets.

[0088] In this implementation, step S5 can be implemented in the following manner:

[0089] The enhanced training set was input into the improved YOLOv10n model for training, and the training parameters were set as follows: learning rate = 0.001, momentum = 0.937, input image size was 640×640, training batch was 8, number of training rounds was 300, SGD optimizer was used, and Mosaic image enhancement was turned off in the last 10 rounds.

[0090] In the S5 training phase, pedestrian images were systematically and meticulously processed in the improved YOLOv10n network structure to improve the model's performance in pedestrian detection tasks. The 0th layer of the backbone network BackBone uses the attention feature fusion structure DC-FFA based on lightweight convolution. This structure can not only effectively retain shallow information, but also pass it to the deep layer through the attention mechanism, thereby enhancing the model's perception of important features. Next, the 1st and 2nd layers are designed as convolutional layers, where the output channels of the 1st layer are 16 and the 2nd layer is 32. Both use a convolution kernel of size 3 and stride 2, combining the functions of expanding channels and reducing feature maps. This design enables the network to better extract diverse features when dealing with complex scenes. In the 3rd, 5th, 7th and 9th layers, C2f modules are used, and the output channels of these layers are 32, 64, 128 and 256 respectively. The number of output channels of each layer is twice that of the previous level, while the amount of calculation is reduced to 0.5 of the previous level. This design not only improves the network's expressiveness, but also significantly reduces the consumption of computing resources, making the model more suitable for real-time applications. The 4th layer performs convolution operations, with an output channel of 64, a convolution kernel size of 3, and a stride of 2, which further enhances the depth of feature extraction. The 6th and 8th layers use the SCDown spatial channel decoupling downsampling module. SCDown adjusts the number of channels through 1×1 point-by-point convolution, and then combines 3×3 depth convolution for spatial downsampling. The output channels are 128 and 256, respectively, which can effectively reduce the dimension of the feature map while retaining important information. The 10th layer introduces the improved spatial pyramid pooling structure SimSPPF to achieve multi-scale feature fusion, output 256 channels, and enhance the detection ability of objects of different scales. The 11th layer is the self-attention module PSA, and the number of output channels is also 256, which further enhances the contextual understanding ability of the model. In subsequent layers, the 12th and 15th layers convert low-resolution feature maps into high-resolution feature maps through Upsample upsampling, thereby improving the clarity of details. The Concat_BiFPN modules in the 13th and 16th layers enhance the spatial details in the low-level feature maps and the semantic information in the high-level feature maps through bidirectional information flow, making the model perform better in multi-scale object capture. The 14th, 17th and 20th layers continue to use the C2f module, with output channels of 128, 64 and 128 respectively, ensuring the continuity of features and the effective transmission of information. The 18th layer is a convolutional layer with an output of 64 channels, which further enhances the feature extraction effect. The next 19th and 22nd layers are concatenation layers, which promote feature integration. The 21st layer is the SCDown downsampling module with an output channel of 128, which continues to reduce the dimension of features.The 23rd layer introduces the C2fCIB module, which replaces the original Bottleneck structure in C2f and uses a combination of deep convolution and point-by-point convolution to further improve the overall performance of the model. This series of carefully designed structures and processing steps ensures that the YOLOv10n network has stronger adaptability and accuracy in pedestrian detection tasks.

[0091] In this implementation, step S6 can be implemented in the following manner:

[0092] The improved YOLOv10n target detection algorithm obtains the coordinate information of the pedestrian target. The specific result is det=(x, y, w, h), where (x, y) represents the center point coordinates of the detected target, w represents the width of the target object, and h represents the height of the target object.

[0093] Furthermore, the improved YOLOv10n model after training is evaluated by the recall rate, precision rate and average precision mean, and the evaluation formulas are respectively:

[0094] Accuracy:

[0095] Recall:

[0096] Average Precision:

[0097] Mean average precision:

[0098] In the formula: TP represents the number of samples correctly judged as positive, FP represents the number of samples incorrectly judged as positive, FN represents the number of samples incorrectly judged as negative, n represents the total number of categories, AP represents the area enclosed by the PR curve, and mAP represents the mean AP of all categories.

[0099] In order to evaluate the performance superiority of the improved target detection algorithm, different algorithm combinations were used to perform the same test. The measured data results are shown in the following table: Model P / ％ R / ％ mAP@0.5 / % YOLOv10n 89.3 83.1 92.3 YOLOv10n+DC-FFA 89.5 83.4 92.8 YOLOv10n+SimSPPF 89.3 83.5 92.4 YOLOv10n+Concat_BiFPN 89.6 83.4 92.8 This algorithm 90.2 84.1 93.2

[0100] As can be seen from the table, the improved algorithm model in this paper has greatly improved the recall rate, precision and average precision, which can meet the actual production needs.

[0101] In this implementation, step S7 can be implemented in the following manner:

[0102] S71: Figure 8As shown in Figure 1, this is the structure diagram of the SEAMNet36 module of the pedestrian re-identification network. The core innovation of SEAMNet36 lies in its structural optimization, which upgrades the original three-layer BasicBlock to five layers. The input feature map size of the first layer is 32×64×32, the feature map size of the second layer is 64×32×16, the feature map size of the third layer is 128×16×8, the feature map size of the fourth layer is 256×8×4, and the feature map size of the fifth layer is 512×8×4. It can be seen that the original re-identification network can only generate a 128-dimensional feature map, which is not able to capture the subtle differences in the appearance of pedestrians. To solve this problem, this paper increases the feature map dimension to 512 dimensions, thereby improving the recognition ability in complex scenes.

[0103] Specifically, the SEAM attention mechanism is integrated into the BasicBlock of the SEAMNet36 person re-identification network, enabling the network to capture the contextual appearance information of each pixel, further enhancing the model's ability to distinguish similar pedestrians. The structure diagram of the SEAM attention mechanism is shown in the figure below. Fig. 9 As shown, first, the network obtains the enhanced image by performing image enhancement changes, i.e., affine transformation, on the original input image, and inputs the original image and the enhanced image into a twin network. The twin network refers to a network with the same structure and shared weights, which is used to evaluate the similarity of inputs. The twin network consists of three modules, namely CNN for extracting image features, CAM for weakly supervised positioning, and pixel correlation module PCM. The associated attention map is obtained by performing self-attention calculation on the feature map extracted by CNN. This associated attention map captures the contextual information for each pixel. The original CAM is modified using the associated attention map to obtain the modified CAM map. Secondly, the original CAM and the modified CAM of the two branches are equivariantly regularized through ERLoss and ECRLoss. The equivariant regularization formula is as follows:

[0104] Equivariant Regularization: R ER =||F(A(I))-A(F(I))||

[0105] Where F(·) represents the network and A(·) represents any spatial affine transformation, such as redrawing, rotation, and flipping.

[0106] The loss function of SEAM is divided into three parts, where the Cls classification loss is used to roughly locate the object, the ER loss is used to narrow the gap between pixel-level and image-level supervision, and the ECR loss is used to integrate PCM with the network to make consistent predictions for various affine transformations. The loss function formula is as follows: Loss function L = L Cls +L ER +L ECR

[0107] S72: Fig.10 As shown in the figure, the execution process of DeepSort multi-target tracking is divided into three cases:

[0108] S721: The first case is to start from tracking and use Kalman filtering to predict the target frame. If it is certain, that is, the current target is a person, not a complex background, then cascade matching is performed with the target detection of the current detection frame. If the cascade matching is successful, it is updated.

[0109] S722: The second case is when the cascade matching of the first case fails, the track that failed to match and the detection that failed to match are IOU matched. If the IOU match is successful, they are updated.

[0110] S723: The third case is when the IOU matching of the second case fails. There are two cases: if it is a detection that fails to match, the possible reason is that the current target is a new target and its track has not appeared before, so the solution is to create a new pedestrian track and set it to uncertain; if it is a track that fails to match, the possible reason is that the detector has missed detection. At this time, check whether it is certain. If it is uncertain, delete it directly. Otherwise, set a time for it and inspect it three times within this time. If there is no change within this time, delete it.

[0111] IOU matching refers to the ratio of the intersection area and the union area of ​​the target prediction bounding box and the true bounding box. IOU only considers the overlapping area of ​​the two bounding boxes, and does not consider the shape information of the target. In addition, IOU is only an evaluation indicator based on the degree of bounding box overlap. It cannot capture other features and details of the target. To solve the above problems, the original IOU matching strategy of DeepSort multi-target tracking was replaced with a more advanced Powerful-IOU matching. The traditional IOU loss function has unreasonable penalty factors, which causes the anchor box to expand during the regression process and significantly slows down the convergence speed. Powerful-IOU introduces a target size adaptive penalty factor and a gradient adjustment function based on the quality of the anchor box.

[0112] like Fig.11 As shown in the figure, this is the principle diagram of Powerful-IOU matching, and its formula is as follows, where each item is marked in the figure:

[0113] Powerful-IOU loss function:

[0114] in, L IOU The IOU loss function measures the degree of overlap by calculating the intersection area of ​​the predicted bounding box and the true bounding box divided by their union area.

[0115] In this implementation, step S8 can be implemented in the following manner:

[0116] S81: Use the improved YOLOv10n model to extract the pedestrian detection box of the first frame from the video stream and create the initial track based on it. At the same time, assign a unique initial ID to each detected pedestrian for subsequent tracking and identification;

[0117] S82: Use Kalman filtering to predict the position of the pedestrian detection frame of the next frame to obtain a pedestrian prediction frame;

[0118] S83: Perform cascade matching and IOU matching on the pedestrian detection frame of the next frame and the prediction frame. By calculating the overlap between the detection frame and the prediction frame, the system can effectively identify the matching pedestrian.

[0119] S84: Apply the Hungarian algorithm to analyze the matching results and update the pedestrian's ID accordingly.

[0120] Furthermore, the improved DeepSort model after training is evaluated by evaluation indicators such as multi-target tracking accuracy MOTA, multi-target tracking precision MOTP and number of encoding transformations IDS. The evaluation formulas are:

[0121] Multi-target tracking accuracy:

[0122] Multi-target tracking accuracy:

[0123] In the formula: N FN Represents the number of undetected pedestrians, N FP represents the number of samples that are misdetected as pedestrians, N IDS represents the number of identity switches in a trajectory, N GT represents the total number of targets, d t,i represents the average distance between target i and the labeled position in the dataset, c t Indicates the number of successful matches of t frames.

[0124] In this implementation, step S9 can be implemented in the following manner:

[0125] like Fig.11As shown, first draw two detection lines on the video: blue line and yellow line. Secondly, compare the pedestrian position coordinates with the coordinates of the two lines. When the coordinates of the monitoring point cross the two detection lines, count them. Finally, if the pedestrian position coordinates cross the blue line first and then the yellow line, it is judged as entering the monitoring area, and the number of pedestrians increases by 1; if the pedestrian position coordinates cross the yellow line first and then the blue line, it is judged as leaving the monitoring area, and the number of pedestrians decreases by 1.

[0126] In order to evaluate the superiority of the improved target tracking algorithm in performance, different algorithm combinations were used to perform the same test. The measured data results are shown in the following table: Model MOTA / % MOTP / % DeepSort 60.7 78.4 DeepSort+SEAMNet36 65.4 80.3 DeepSort+Powerful-IOU 62.9 79.5 This algorithm 66.2 81.4

[0127] It can be seen from the table that the improved algorithm model in this paper has been greatly improved in both MOTA and MOTP, and can meet actual production needs.

Claims

1. A lightweight scenic spot pedestrian counting system based on YOLOv10n and DeepSort, characterized by: The specific steps are as follows: S1: Obtain pedestrian images in crowded places such as scenic spots, shopping malls, and streets to build a dataset; S2: Perform data enhancement on pedestrian images in the data set to obtain a data enhanced data set; S3: Build the YOLOv10n pedestrian detection network model; S4: Improve the YOLOv10n model to obtain an improved YOLOv10n model; S5: Use the data-enhanced dataset obtained in S2 to train the improved YOLOv10n network structure to obtain a trained pedestrian detection model; S6: Send the pedestrian image to be detected into the trained YOLOv10n model to obtain the pedestrian's position information and detection box information; S7: improving the DeepSort model to obtain an improved DeepSort model; S8: Send the information detected by S5 to the improved DeepSort model to obtain the sequence number of the pedestrian target box; S9: The number of pedestrians entering and leaving is obtained through a counting algorithm based on double convex track measurement.

2. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1 is characterized in that: The S1 step is specifically as follows: in a normal lighting environment, pedestrian images are acquired by a Canon 800D camera to construct a data set, the constructed data set is divided into a training set, a validation set and a test set in an 8:1:1 manner, and then the pedestrian images in the data set are finely annotated using the Makesense online annotation tool. If the pedestrian image contains a pedestrian, the pedestrian is marked as person.

3. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1 is characterized in that: The step S2 specifically includes: performing image enhancement methods such as image cropping, translation, rotation, and changing image brightness on the pedestrian images in the data set, enhancing the quantity of each pedestrian image, and expanding the original one pedestrian image to four.

4. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1 is characterized in that: The S3 step is specifically as follows: constructing the original YOLOv10n network model, whose overall structure consists of four parts, which are connected in this way to complete the target detection task; first, the input end includes functions such as mosaic, data enhancement and adaptive anchor frame calculation; next, the backbone network Backbone adopts the CSPDarknet architecture, which is composed of multiple Conv, C2f, SCDown, SPPF and other modules for extracting image features; followed by the Neck part, the neck network consists of two parts: the feature pyramid network FPN and the path aggregation network PAN, which are used to fuse the high-level and low-level features; finally, the Head part is used to generate the prediction results of target detection.

5. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1 is characterized in that: The S4 step is specifically as follows: adding an attention feature fusion structure DC-FFA based on lightweight dynamic convolution to the backbone network Backbone of the original YOLOv10n. The structure can retain shallow information and pass it to the deep layer without increasing the number of model parameters, thereby enhancing the representation ability of the model. It can not only fuse all features, but also adaptively learn different weights of feature information at different levels; using the DC-FFA structure as the first layer of the improved model network structure, and replacing the original SPPF module in the last layer of Backbone with the SimSPPF module. The SimSPPF module reduces unnecessary computational complexity by simplifying operations, optimizes the calculation and storage operations in the feature fusion process, reduces the overall weight of the network, and reduces the computational complexity and improves the inference speed of the model; finally, the Concat module of the Neck part FPN is integrated into the repeated weighted bidirectional feature pyramid network BiFPN to obtain the Concat_BiFPN module, allowing features to flow bidirectionally between different scales in the network.

6. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1 is characterized in that: The S5 step is specifically as follows: inputting the training set into the improved YOLOv10n model for training; and the training parameters are set as follows: learning rate = 0.001, momentum = 0.937, input image size is 640×640, training batch is 8, number of training rounds is 300, SGD optimizer is used, and Mosaic image enhancement is turned off in the last 10 rounds.

7. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1 is characterized in that: The improved YOLOv10n model after training is evaluated by precision, recall and average precision, and the evaluation formulas are: Accuracy Recall Average precision Mean average precision In the formula: P is the accuracy, T p is the number of correctly predicted personnel, F p is the number of people predicted incorrectly. R is the recall rate, F N is the number of people who have not been detected, AP is the average accuracy, AP is the area under the PR curve of a certain category, mAP is the mean average precision, AP i is the average accuracy of the i-th category, and N is the number of target categories in the sample.

8. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1 is characterized in that: The S7 step is specifically as follows: first, a new pedestrian re-identification network SEAMNet36 is designed to replace the original ReID network. On the basis of the original re-identification network, the residual layer is added to five layers, and the BasicBlock of each layer is embedded in the self-supervised equivariant attention mechanism SEAM to improve the accuracy of pedestrian re-identification, especially in complex backgrounds and pedestrians with similar appearances. It can better distinguish pedestrians; secondly, the original IOU matching is replaced by Powerful-IOU matching to improve the accuracy and robustness of target matching, especially in scenes with partial occlusion and dense pedestrians, the matching effect is better.

9. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1 is characterized in that: The step S8 specifically includes: creating an initial trajectory and setting an initial ID according to the pedestrian detection frame of the first frame in S6; using Kalman filtering to predict the position of the pedestrian detection frame of the next frame to obtain a pedestrian prediction frame; Perform cascade matching and Powerful-IOU matching on the pedestrian detection frame of the next frame and the pedestrian prediction frame to obtain a matching result; and update the ID according to the matching result.

10. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1, characterized in that: The improved DeepSort model after training is evaluated by evaluation indicators such as multi-target tracking accuracy MOTA, multi-target tracking precision MOTP and number of encoding transformations IDS. The evaluation formulas are: TYPE MOTP In the formula: N FN Represents the number of undetected pedestrians, N FP represents the number of samples that are misdetected as pedestrians, N IDS represents the number of identity switches in a trajectory, N GT Indicates the total number of targets. t,i represents the average distance between target i and the labeled position in the dataset, c t Indicates the number of successful matches of t frames.

11. The lightweight scenic spot pedestrian counting system based on improved YOLOv10n and DeepSort according to claim 1, characterized in that: The S9 step is specifically as follows: first, two irregular detection lines are drawn on the video: a blue line and a yellow line; the pedestrian position coordinates are continuously compared with the coordinates of the two lines, and when the coordinates of the monitoring point intersect the two detection lines, a count is performed; if the pedestrian position coordinates first cross the blue line and then the yellow line, it is determined to have entered the monitoring area, and the number of pedestrians is increased by 1; if the pedestrian position coordinates first cross the yellow line and then the blue line, it is determined to have left the monitoring area, and the number of pedestrians is decreased by 1.

Images