Pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment

Abstract

The invention discloses a pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment, and aims at the problem that a deep convolutional neural network in a backbone network is frequently excessively parameterized, a composite scaling mechanism is introduced to reduce the parameter quantity and improve the feature extraction capability of the network; meanwhile, considering that targets with different scales exist in a data set, a lightweight cross-scale feature fusion module is fused at the neck of the network, so that the adaptability of the model to scale change is enhanced; besides, aiming at the problems of small targets and shielding targets, a new loss function InnerWiseWIoU is designed, and in combination with internal region optimization and context information, the target positioning precision is improved. According to the method, the new model is applied to pedestrian tracking, the shielding degree between pedestrians is calculated and the matching threshold is dynamically adjusted aiming at the condition that a fixed confidence threshold in a tracking task is not suitable for centralized and rapid change of a target, so that the tracking performance is effectively improved, and the adaptability in a complex scene is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of pedestrian target detection, relates to a pedestrian target detection method and multi-target tracking thereof, and specifically relates to a pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment. Background Art

[0002] Multi-target pedestrian detection and tracking aims to locate, identify, and associate multiple pedestrian targets in real time from a video or image sequence, while also generating the motion trajectory of each target. By accurately detecting and continuously tracking pedestrian targets, this technology can understand the behavioral patterns, motion states, and interrelationships of pedestrians in a scene, providing foundational support for higher-level semantic analysis. It is widely used in intelligent video surveillance, autonomous driving, pedestrian re-identification, and crowd behavior analysis. With the rapid development of deep learning technology, multi-target pedestrian detection and tracking has achieved significant improvements in accuracy and efficiency. However, it still faces numerous challenges in terms of robustness in complex scenarios, real-time performance, and handling of target occlusion.

[0003] The core tasks of multi-target pedestrian detection and tracking consist of two phases: target detection and target tracking. In the target detection phase, the algorithm needs to locate and classify pedestrian targets in the image. Common detectors include two-stage deep learning-based methods (such as Faster R-CNN) and single-stage methods (such as the YOLO series). In the target tracking phase, the algorithm uses data association technology to match detection results from different frames to generate the motion trajectory of each target. In recent years, the tracking-by-detection paradigm has become mainstream. Algorithms such as SORT, DeepSort, and ByteTrack achieve efficient target association by combining detection results with Kalman filtering or the Hungarian algorithm.

[0004] In practical applications, the main challenges faced by multi-target pedestrian detection and tracking technology include target occlusion, scale variation, background interference, and real-time requirements. For example, in dense crowds, mutual occlusion between pedestrians can lead to reduced detection and tracking accuracy; while in surveillance videos, the rapid movement and multi-scale variations of pedestrians also place higher demands on the robustness of the algorithm. Traditional detectors (such as Faster R-CNN and YOLO series) perform well in target positioning, but still have shortcomings in multi-scale target detection and occlusion processing. Existing trackers (such as DeepSORT) improve tracking performance through appearance features and motion models, but their adaptability in dynamic scenes still needs to be improved. Summary of the Invention

[0005] The problem solved by the present invention is to provide a pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment, which realizes the deep fusion of detection and tracking, solves the shortcomings of existing methods in multi-scale target adaptation, occlusion processing and dynamic adjustment of complex scenes, and significantly improves the accuracy and robustness of pedestrian multi-target detection and tracking.

[0006] The present invention is achieved through the following solutions:

[0007] The pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment includes the following operations:

[0008] 1) Build the YOLOV10-EfficientCCFM model of the object detector, which includes the backbone network, neck network and detection head;

[0009] The backbone network replaces the YOLOV10 backbone network with EfficientNetV1, introduces a composite scaling mechanism, and uses a fixed scaling factor to uniformly scale the network width, depth, and resolution. The neck network integrates the lightweight cross-scale feature fusion module (CCFM) into the YOLOV10 neck network to fuse feature maps of different scales. At the same time, the attention mechanism and multi-scale feature aggregation are used to improve the contextual information expression capability of the feature maps. The detection head uses the YOLO detection head to detect and locate multi-target pedestrians, and outputs the pedestrian position and confidence results.

[0010] The loss function is replaced by the Inner_Wise_WIoU loss function shown in formula (4). The closer the calculation result of the loss function is to 0, the better the match between the predicted area and the true area.

[0011]

[0012] Where B is the predicted bounding box area, B gt is the area of ​​the true bounding box, W total is the dynamic weight matrix, W total ·(B∩B gt ) takes into account the importance of different locations and weights the contribution of the intersection area; ∑W total ·(B∪B gt ) is a double-weighted summation of the union area, taking into account the weights of the union areas at different positions, and r is the dynamic gradient gain factor;

[0013] 2) Train the target detector YOLOV10-EficientCCFM model based on the dataset;

[0014] 3) ByteTrack is selected as the multi-target tracking algorithm. The tracker first detects the target bounding box and confidence score from the video frame using the target detector. Based on the confidence score, the detected target boxes are divided into high-scoring detection boxes and low-scoring detection boxes. Then, the matching threshold of the high-scoring detection boxes and the low-scoring detection boxes is dynamically adjusted according to the occlusion ratio between the targets.

[0015] Then, the first data association is performed, and the high-scoring detection box targets are matched with the existing trajectories predicted by the Kalman filter using the Hungarian algorithm to perform IoU matching, and the trajectory position and motion state are updated. High-confidence targets that are not matched are initialized as new trajectories and remaining trajectories.

[0016] Then, in the second data association phase, a second IoU matching is performed on the low-resolution detection box targets output by the detector and the remaining tracks that are temporarily retained after the first data association phase. If the match is successful, the lost track is restored. Otherwise, the target with the unmatched track is considered a new target and temporarily stored.

[0017] Finally, lifecycle management is performed on continuously unmatched trajectories. If the target does not appear after the number of lost frames exceeds the set number of frames, the target is considered lost and the trajectory is deleted; the unique ID and position coordinates of each target are output.

[0018] Furthermore, the width, depth and resolution of the unified scaling network are:

[0019] Adjust the network depth to: d = α φ ;

[0020] Adjust the network width to: w = β φ ;

[0021] Adjust the network resolution to: r = γ φ ;

[0022] Among them, α is the depth scaling factor, β is the width scaling factor, γ is the resolution scaling factor, and φ is the global compounding factor.

[0023] Furthermore, in the Inner_Wise_WIoU loss function, when the degree of sample abnormality is high due to severe occlusion, the gradient suppression strength is increased by adjusting r to reduce the interference of low-quality predictions on the loss calculation.

[0024] Furthermore, the target detector YOLOV10-EficientCCFM model is trained as follows:

[0025] Phase 1: Replace the CSPDarknet backbone network of YOLOv10n with EfficientNetV1, retaining the original detection head and neck structure. During training, use the initial learning rate for multi-batch training. The training model EfficientNetV1 uses a compound scaling mechanism to enhance the multi-scale feature extraction capability.

[0026] Phase II: While retaining the original CSPDarknet backbone network of YOLOv10n, the neck module is replaced with a lightweight cross-scale feature fusion module (CCFM). This module uses a channel attention mechanism to weightedly fuse multi-scale features and train the model to detect small objects at a distance.

[0027] Phase 3: Integrating the improvements from the first two phases, we built a joint model combining the EfficientNetV1 backbone network and the CCFM neck module. We also trained the collaboration between EfficientNetV1 and CCFM to improve the model's adaptability to complex scenarios.

[0028] The fourth stage: Based on the third stage, the original IoU loss function is replaced by the Inner_Wise_WIoU loss function; the training combines the effectiveness of internal optimization and contextual information.

[0029] Furthermore, if the confidence score of the target bounding box output by the detector is greater than 0.5, the detected target box is a high-score detection box;

[0030] If the confidence score of the target bounding box output by the detector is >0.1 and <0.5, the detected target box is a low-score detection box.

[0031] Furthermore, the dynamic adjustment of the matching threshold includes:

[0032] ① When the occlusion ratio is greater than 0.5, the occlusion is severe and the threshold is reduced to 90% of the initial threshold.

[0033] ② When the occlusion ratio is greater than 0.3, it is considered moderate occlusion, and the threshold is lowered to 95% of the initial threshold.

[0034] ③ When there is no occlusion or the occlusion ratio is ≤0.3, restore the initial threshold.

[0035] Compared with the prior art, the present invention has the following beneficial technical effects:

[0036] The pedestrian multi-target detection and tracking algorithm based on improved YOLOv10 proposed in the present invention is based on the YOLOv10-EfficientCCFM model structure proposed in the present invention. According to the compound scaling mechanism, a compound coefficient is used to uniformly balance the network depth, width and resolution, and the convolutional network is extended to any target resource constraint while maintaining model efficiency. And according to the lightweight cross-scale feature fusion module, features of different scales are integrated through fusion operations, effectively integrating detailed features and contextual information, thereby improving the overall performance of the model. By fusing the compound scaling mechanism with the cross-scale feature fusion module, the network's detection capability for targets in dense and complex scenes is enhanced.

[0037] The present invention adopts a new loss function: Based on WeightedIoU, InnerIoU and WiseWIoU, the present invention proposes the Inner_Wise_WIoU loss function to achieve high-precision bounding box regression in occlusion scenarios.

[0038] Based on target detection, the trained detector is applied to the tracking task; the present invention further proposes a threshold adjustment strategy based on the degree of pedestrian occlusion, which effectively reduces the problem of target occlusion and loss by dynamically adjusting the detection frame matching threshold, thereby improving the accuracy and robustness of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Schematic diagram of the YOLOV10-EfficientCCFM model structure of the present invention.

[0040] Figure 2 Schematic diagram of the tracking algorithm of the present invention. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below with reference to the embodiments, which are intended to explain the present invention rather than to limit it.

[0042] This invention aims to further improve the performance of multi-target pedestrian detection and tracking. ByteTrack is an efficient detection-based tracking algorithm that improves tracking performance by fully utilizing score detection boxes. During the data association phase, this algorithm not only uses high-score detection boxes but also dynamically adjusts the matching threshold to reduce target loss and ID switching when occlusion occurs between pedestrian targets.

[0043] However, ByteTrack still has certain limitations when dealing with complex scenes (such as target occlusion, scale changes, and background interference). For example, in dense crowd scenes, mutual occlusion between pedestrians will lead to a decrease in the accuracy of detection and tracking; and in dynamic scenes, the rapid movement and multi-scale changes of pedestrians also put higher demands on the robustness of the algorithm. To this end, based on ByteTrack, the present invention introduces EfficientNetV1 as a feature extraction network in the detection part, and uses its compound scaling mechanism to improve the performance of the convolutional neural network; it integrates the lightweight cross-scale feature fusion module CCFM (Cross-Scale Feature Fusion Module) to enhance the model's adaptability to scale changes and its ability to detect small-scale targets;

[0044] To address the problem of over-parameterization of deep convolutional neural networks in some detection backbone networks, this paper introduces a composite scaling mechanism to reduce the number of parameters and improve the network's feature extraction capabilities. Furthermore, considering the presence of targets of different scales in the dataset, a lightweight cross-scale feature fusion module is integrated at the neck of the network to enhance the model's adaptability to scale changes. Furthermore, to address the issues of small and occluded targets, a new loss function (Inner_Wise_WIoU) is designed to achieve high-precision bounding box regression in occluded scenarios. This paper applies the new model to pedestrian tracking, and to address the problem that fixed confidence thresholds in tracking tasks are not suitable for concentrated and rapidly changing targets, the paper calculates the degree of occlusion between pedestrians and dynamically adjusts the matching threshold, thereby effectively improving tracking performance and adaptability in complex scenarios.

[0045] The pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment provided by the present invention includes the following operations:

[0046] 1) Build the YOLOV10-EfficientCCFM model for the target detector;

[0047] And replace the loss function with the Inner_Wise_WIoU loss function. The closer the calculation result of the loss function is to 0, the better the match between the predicted area and the true area.

[0048] 2) Train the target detector YOLOV10-EficientCCFM model based on the dataset;

[0049] 3) ByteTrack is selected as the multi-target tracking algorithm. The tracker first detects the target bounding box and confidence score from the video frame through the target detector. Based on the confidence score, the detected target boxes are divided into high-score detection boxes and low-score detection boxes. Then, the matching threshold of high-score detection boxes and low-score detection boxes is dynamically adjusted according to the occlusion ratio between the targets. Then, IoU matching is performed, and finally the unique ID and position coordinates of each target are output.

[0050] The following is a detailed description of each step.

[0051] 1) Build a pedestrian multi-target detection network model YOLOV10-EficientCCFM

[0052] Network structure: The network model consists of three parts: backbone network, neck network, and detection head.

[0053] The backbone network is the main feature extraction component of the model, used to balance the network's depth, width, and resolution, improving its feature extraction capabilities. The neck network, an intermediate layer between the backbone network and the head, is responsible for fusing feature maps of different scales, enhancing the model's adaptability to scale changes and its ability to detect small-scale objects. The detection head is the output component of the model and is responsible for the final task prediction, generating the object's bounding box and category prediction.

[0054] The network uses the Multi-Object Tracking16 (MOT16) dataset, a widely used dataset for multi-object tracking of pedestrians. This dataset consists of multiple videos, each containing multiple consecutive frames, and is used to train and evaluate the performance of multi-object tracking algorithms. The dataset incorporates diverse shooting conditions, such as weather conditions, time of day (daytime or nighttime), and shooting angles (low, medium, and high). The videos also feature a high density of pedestrians, as well as issues such as occlusion, similar appearance, and complex motion, all of which complicate tracking.

[0055] Backbone Network Design: In the multi-target pedestrian detection task, the backbone network's feature extraction capabilities directly impact model performance. Model compression is a common approach that reduces model size by trading accuracy for efficiency. ConvNets are typically scaled up in one of three dimensions: depth, width, or image size. While increasing network width, depth, or resolution improves accuracy, this improvement diminishes for larger models. Arbitrary scaling requires tedious manual tuning and may result in suboptimal accuracy and efficiency.

[0056] In pursuit of better accuracy and efficiency, the present invention improves yolov10 by balancing all dimensions of network width, depth and resolution during continuous network scaling: the yolov10 backbone network is replaced by EfficientNetV1, and a composite scaling mechanism, that is, a set of fixed scaling factors, is used to uniformly scale the width, depth and resolution of the network, which can achieve state-of-the-art accuracy while minimizing the number of parameters.

[0057] Specifically, the composite scaling mechanism of EfficientNetV1 uniformly adjusts the network dimensions to:

[0058] Adjust network depth: d = α φ ;

[0059] Adjust network width: w = β φ ;

[0060] Adjust network resolution: r = γ φ ;

[0061] Among them, α is the depth scaling factor, β is the width scaling factor, and γ is the resolution scaling factor, which are determined by grid search; φ is the global composite coefficient, which controls the overall computational complexity. The larger φ is, the more complex the model is.

[0062] For example, the original YOLOv10 backbone network has a depth of d0 = 3 (the number of CSP module stacking layers), a width of w0 = 64 (the number of initial channels), and a resolution of r0 = 640 (the input size). When the base model φ = 1, the depth d = α after replacing it with EfficientNetV1 φ d0 = 1.2 × 3 ≈ 4 (rounded up); width w = β φ w0 = 1.11 × 64 ≈ 72 (aligned by multiples of 8); resolution r = γ φ r0 = 1.151 × 640 ≈ 736 (scaling to maintain aspect ratio).

[0063] Selection and design of the neck network: The role of the neck network is to further perform operations such as feature fusion based on the feature maps extracted by the backbone network. Feature fusion refers to integrating the feature maps between different layers into the same size and fusing them into a feature map through upsampling or downsampling methods. Based on replacing the backbone network of YOLOv10 with EfficientNetV1, the present invention further introduces a lightweight cross-scale feature fusion module (CCFM) and proposes the YOLOv10-EficientCCFM model structure. EfficientNetV1 is able to extract multi-level features through balanced scaling and efficient structural design, but it has certain limitations in feature fusion and contextual information utilization. The CCFM module is introduced to reconstruct the feature pyramid, and the features extracted by the replaced backbone network are sent to the neck network to achieve efficient fusion of multi-scale detail features.

[0064] CCFM addresses the shortcomings of EfficientNetV1 through cross-scale feature fusion and contextual information enhancement. Specifically, CCFM can effectively fuse feature maps of different scales, enhancing the model's ability to detect multi-scale objects. At the same time, through the attention mechanism and multi-scale feature aggregation, it improves the ability of feature maps to express contextual information.

[0065] Experiments show that the deep fusion of EfficientNetV1 and CCFM significantly improves the mAP of the model in pedestrian detection tasks, especially in small target detection and occlusion scenarios.

[0066] like Figure 1 The model structure of YOLOV10-EficientCCFM is shown in Figure 1. Table 1 is the description of each module in the model structure. The input image size is 640×640×3. The YOLOV10-EficientCCFM network first extracts features in the backbone network, performs feature fusion in the neck, and finally performs target position prediction and position regression.

[0067] Table 1 Description of each module in the model structure

[0068]

[0069] Loss function design: In object detection tasks, the loss function is a core component of model training. It is used to measure the difference between the predicted results and the true labels. The design of the loss function directly affects the model training effect and detection accuracy.

[0070] This paper improves the loss function of YOLOv10 and proposes the Inner_Wise_WIoU loss function based on the Weighted IoU, InnerIoU and WiseWIoU loss functions to better optimize the bounding box regression task.

[0071] Specifically, this loss function combines dynamic weight allocation, interior region enhancement, and gradient focusing mechanisms to address complex situations such as pedestrian occlusion. By introducing multi-scale interior region weights, a dynamic boundary-interior balance factor, and outlier-aware gradient modulation, the model significantly improves its ability to handle occluded regions and enhances the accuracy of fitting the target's interior region.

[0072] By adaptively adjusting the auxiliary frame size and dynamically adjusting the weight according to the target occlusion ratio, we can effectively deal with multi-scale targets and different degrees of occlusion, avoiding the disadvantages of fixed auxiliary frame size. Formula (1) is used to calculate the size weight of the adaptive auxiliary frame, and its value is determined by the area ratio of the predicted frame to the real frame and the occlusion ratio. Where w and h are the width and height of the predicted frame; w gt 、h gt is the width and height of the real frame. The calculated value is the ratio of the predicted box area to the real box area, which reflects the relative relationship between the predicted box and the real box in size.

[0073] For example, if the predicted box and the real box are the same size, the ratio is 1; if the predicted box is smaller than the real box, the ratio is less than 1; Occ_Ratio is the occlusion ratio; γ∈[0,1] is the inverse proportional factor of the occlusion ratio. The more the target is occluded, the larger the Occ_Ratio is, and the smaller γ is. Its function is to dynamically adjust the weight according to the occlusion situation. inner The final adaptive auxiliary box size weight is calculated by taking into account the area ratio of the predicted box to the ground-truth box, as well as the object's occlusion. This calculation allows for a more reasonable determination of the auxiliary box size weight when the object is occluded or exhibits multi-scale variations, thereby focusing on the main object area.

[0074]

[0075] The dynamic boundary-interior balance factor further enhances the adaptability of the loss function in complex scenarios, combining the boundary weight of WIoU and W inner Internal region optimization. As training progresses, the weight ratio of the boundary and internal regions gradually transitions from boundary alignment in the early stage to internal region fitting in the late stage, thus ensuring the optimization focus on different regions at different training stages and greatly improving the accuracy of the model. Formula (2) is the total weight matrix, which is represented by W boundary is the boundary weight matrix (inherited from Weighted Intersection over Union) and the internal weight W in formula (1) inner A linear combination of λ and λ is a balance factor, ranging from 0 to 1.

[0076] When λ is close to 1, W total More dependent on W boundary , which means that more emphasis is placed on the boundary area; when λ is close to 0, W total More dependent on W inner , that is, more attention is paid to the internal area. λ dynamically decays with the training stage. In the early stage of training, λ is set larger, focusing on boundary alignment. In the later stage of training, λ is reduced to enhance the fitting of the internal area and improve occlusion robustness. T is the total number of training rounds, which serves as the time scale parameter in the decay formula of the dynamic balance factor λ, controlling λ to decay from the initial value λ0 to the minimum value λ min The speed of k is based on λ0, λ min , a coefficient calculated by T, which is used to determine the rate of exponential decay. t is the current training round. As t increases from 1 to T, λ decays exponentially from λ0 to λ min .

[0077]

[0078] In addition, a dynamic non-monotonic focusing mechanism is introduced in the loss function of Wise-IoU-v3, as shown in formula (3). The gradient gain is adjusted by the abnormality β to suppress the negative impact of low-quality samples on bounding box regression. β is the abnormality degree, which is the ratio of the current sample IoU loss to the average loss, reflecting the degree of difference between the current sample loss and the average loss. The larger the value, the more abnormal the current sample is (the sample quality is low). α and δ are hyperparameters in the formula, which jointly affect the calculation result of the gradient gain factor r. When the sample quality is high (β≈1), the gradient gain r≈1, and the optimization is normal; when the sample quality is low (β>>1), the numerator β increases linearly and the denominator δ·α β-δ As β grows exponentially (because α>1), the gradient gain r↓ suppresses noise propagation and enables learning with different sample qualities.

[0079]

[0080] The final formula of the Inner_Wise_WIoU loss function is shown in formula (4). It represents the calculation result of the loss function. The closer the value is to 0, the better the match between the predicted area and the real area. In the formula, B is the predicted bounding box area, and B gt is the area of ​​the true bounding box, W total is the dynamic weight matrix. W total ·(B∩B gt ) takes into account the importance of different locations and weights the contribution of the intersection area; ∑W total ·(B∪B gt) performs a doubly weighted summation of the unioned regions, comprehensively considering the weights of the unioned regions at different locations. r is a dynamic gradient gain factor, which dynamically adjusts the gradient propagation strength of the loss function based on the degree of sample anomaly. When the sample is highly anomaly (such as severe occlusion), r is adjusted to increase the gradient suppression strength and reduce the interference of low-quality predictions on the loss calculation.

[0081]

[0082] Compared with the existing WIoU, Inner WIoU and WiseWIoU, the Inner_Wise_WIoU loss function successfully solves the shortcomings of existing loss functions in multi-target tracking tasks, especially in complex occlusion and multi-scale problems. Inner_Wise_WIoU demonstrates excellent performance, can more accurately fit the internal area of ​​the target and optimize the bounding box regression, thereby improving the robustness and accuracy of the model in complex environments.

[0083] Detection head selection: Select the YOLO detection head for multi-target pedestrian detection and positioning, and output the pedestrian position and confidence results.

[0084] 2) Training of multi-target pedestrian detection network

[0085] The training process of the improved YOLOv10 detector proposed in this invention is divided into the following steps:

[0086] 1. Phase 1: First, the CSPDarknet backbone network of the original YOLOv10n was replaced with EfficientNetV1, preserving the original detection head and neck structure. Training was performed with an initial learning rate of 0.001 and a batch size of 4. After 300 epochs of training, the model's mAP50 on the validation set improved from 79.31% to 80.17%. Experiments demonstrated that EfficientNetV1 significantly enhanced its multi-scale feature extraction capabilities through a compound scaling mechanism.

[0087] 2. Phase II: While retaining the original CSPDarknet backbone network of YOLOv10n, the neck module was replaced with a lightweight Cross-Scale Feature Fusion Module (CCFM), which weightedly fuses multi-scale features using a channel-wise attention mechanism. Experimental results show that the model's mAP50 score improved to 80.37%. The CCFM module effectively alleviates the problem of neglecting channel differences in traditional feature fusion, enhancing the detection of small objects at a distance.

[0088] 3. The third stage: Integrating the improvements from the first two stages, we constructed a joint model combining the EfficientNetV1 backbone network and the CCFM neck module. Ablation experiments showed that the synergy between EfficientNetV1 and CCFM significantly improved the model's adaptability to complex scenes, particularly in frames with densely occluded objects, improving detection box positioning accuracy by approximately 1.16%.

[0089] 4. Phase 4: Building on the foundation of Phase 3, the original IoU loss function was further replaced with an innovative Inner_Wise_WIoU loss function. This loss function optimizes the bounding box regression process for occluded objects by shrinking the inner region of the ground-truth bounding box and introducing context-aware weights. Experimental results show that the model's mAP50 score further improves to 80.47%, validating the effectiveness of combining internal optimization with contextual information.

[0090] In order to comprehensively and accurately evaluate the performance of the detector, the present invention adopts the following evaluation indicators: mAP50, mAP50-95, Precision (%), Recall (%) and GFLOPS.

[0091] mAP50 is the mean average precision at an IoU threshold of 0.5, reflecting the model's detection capabilities in general scenarios. mAP50-95 is the mean average precision at IoU thresholds ranging from 0.5 to 0.95, providing a more rigorous assessment of the model's performance under high-precision requirements. Precision (%) measures the proportion of true positive samples within bounding boxes predicted by the model as positive, reflecting the model's false positive rate. Recall (%) measures the proportion of true positive samples within bounding boxes correctly predicted by the model as positive, reflecting the model's missed detection rate. GFLOPS assesses the model's computational complexity and measures its inference efficiency. These metrics reflect the model's detection accuracy, recall, and computational efficiency from different perspectives, meeting the requirements of practical applications. By combining these evaluation metrics, we can ensure the model's robustness and real-time performance in complex scenarios, providing a reliable performance evaluation basis for pedestrian multi-target detection tasks.

[0092] Ablation experiment: EfficientNetV1, CCFM and Inner_Wise_WIoU loss functions are gradually introduced into the network for training to observe whether they have any improvement effect on pedestrian multi-target detection and record the experimental results (see Table 4 for experimental results).

[0093] 3) Multi-target tracking algorithm based on dynamic threshold optimization of pedestrian occlusion degree

[0094] Selection of multi-target tracking algorithms: The quality of the target tracking algorithm directly affects tracking performance. The Sort algorithm estimates and matches target states in consecutive frames, but targets are easily lost when occluded. DeepSort improves real-time performance and accuracy based on Sort, but tracking performance deteriorates in complex scenes or densely populated targets, and the pedestrian re-identification step consumes a lot of computing time.

[0095] This paper selects ByteTrack as a multi-target tracking algorithm. Compared to SORT and DeepSORT, ByteTrack significantly reduces target loss and ID switching issues by fully utilizing low-scoring detection boxes and a dynamic thresholding strategy. Furthermore, ByteTrack does not require the extraction of appearance features, resulting in high computational efficiency and suitability for real-time applications. Experiments show that ByteTrack achieves significantly higher HOTA and MOTA than SORT and DeepSORT in the multi-target pedestrian tracking task, particularly in complex scenarios.

[0096] Design of a Threshold-Optimized Multi-Target Tracking Algorithm Based on Pedestrian Occlusion Degree: This paper proposes a threshold-optimized multi-target tracking algorithm based on pedestrian occlusion degree, aiming to address the problem of degraded tracking performance caused by target occlusion in complex scenes. The algorithm evaluates the degree of occlusion by calculating the intersection over union (IoU) between targets and dynamically adjusts the tracking thresholds (including a high threshold track_high_thresh and a low threshold track_low_thresh) based on the occlusion degree.

[0097] See also Figure 2 The tracker first extracts the target bounding box and confidence score in the video frame through the YOLOv10-EfficientCCFM detector, and divides the detected target box into high-score detection boxes (confidence score > 0.5) and low-score detection boxes (confidence score > 0.1). Then, based on the occlusion ratio between the targets calculated in real time, the matching threshold of the high-score detection box and the low-score detection box is dynamically adjusted:

[0098] If the occlusion ratio is greater than 50%, reduce the matching threshold of both the high-score detection frame and the low-score detection frame by 90%;

[0099] If the occlusion ratio is greater than 30%, reduce the matching threshold of both the high-score detection frame and the low-score detection frame by 95%;

[0100] Then, the first data association is performed, and the high-scoring detection box target (high-confidence target) is matched with the existing trajectory predicted by the Kalman filter through the Hungarian algorithm for IoU matching (the existing trajectory predicts the target state of the current frame through the Kalman filter, and is obtained based on the Hungarian matching association of the historical trajectory and the detection result), and the trajectory position and motion state are updated; for the unmatched high-confidence target, it will be initialized as a new trajectory and a remaining trajectory (the trajectory of the previous frame was successfully matched, but the trajectory in the current frame did not match the target).

[0101] The second data association phase is performed on two types of targets: low-confidence targets output by the detector and residual tracks that are temporarily retained after the first data association phase (the track was successfully matched before but was temporarily lost and unmatched, and the number of lost frames is less than a manually set threshold). If the match is successful, the lost track is restored; otherwise, the target with the unmatched track is considered a new target and temporarily stored.

[0102] Finally, lifecycle management is performed on continuously unmatched trajectories. If the target does not appear after the number of lost frames exceeds the set number of frames, the target is considered lost and the trajectory is deleted; the unique ID and position coordinates of each target are output.

[0103] Both data association passes use the same matching threshold adjustment method for occlusion between pedestrians; both data association stages are IoU matching; the difference is that the first data association stage matches high-confidence targets, and the second data association stage matches the targets remaining after the first association and the low-confidence targets output by the detector.

[0104] The present invention also calculates the degree of occlusion. When the occlusion is severe, the algorithm lowers the matching threshold to retain more low-scoring detection frames, thereby reducing target loss; when the occlusion is light or there is no occlusion, the algorithm restores the initial matching threshold to reduce mismatching.

[0105] Furthermore, the following schemes can be used to dynamically adjust the matching threshold:

[0106] ① When the occlusion is severe (occlusion ratio > 0.5), the matching threshold is reduced to 90% of the initial threshold.

[0107] ② When there is moderate occlusion (occlusion ratio > 0.3), the matching threshold is lowered to 95% of the initial threshold.

[0108] ③ When there is no occlusion or the occlusion is small, restore the initial matching threshold.

[0109] Table 2 is an embodiment of a method for calculating the degree of occlusion between pedestrians.

[0110] Table 2 Calculation method of occlusion degree between pedestrians

[0111]

[0112] 4) Evaluation of the effect of multi-target pedestrian tracking

[0113] In the multi-target tracking task, in order to comprehensively evaluate the performance of the model, this paper uses HOTA (Higher Order Tracking Accuracy), MOTA (Multiple Object Tracking Accuracy), IDF1 (Identity F1 Score) and IDS (Identity Switches) as evaluation indicators.

[0114] HOTA provides a comprehensive evaluation of tracking performance by comprehensively considering detection accuracy, association accuracy, and identity consistency; MOTA combines the false detection rate, missed detection rate, and ID switching rate, and is a classic indicator for measuring tracking accuracy; IDF1 reflects the model's ability to maintain target identity consistency by calculating the F1 score of the target identity match; IDS directly counts the number of target identity switches and is used to evaluate the robustness of the model in complex scenarios. By comprehensively using these indicators, the present invention can comprehensively evaluate the performance of multi-target tracking models from multiple dimensions such as detection accuracy, association accuracy, identity consistency, and robustness, especially in scenes with dense pedestrians and severe occlusion.

[0115] (1) The experimental environment is shown in Table 3:

[0116] Table 3 Experimental environment

[0117]

[0118] (2) Improve the ablation experiment of yolov10:

[0119] Regarding the optimization of the detector, in order to verify the effectiveness of these improvements, the present invention designed the following ablation test:

[0120] First, the original YOLOv10 model is used as the baseline model, and its evaluation indicators such as mAP50, mAP50-95, Precision, Recall, and GFLOPS are tested on the validation set.

[0121] The EfficientNetV1, CCFM, and Inner_Wise_WIoU loss functions are gradually introduced into the baseline model, and their performance improvements are tested respectively;

[0122] Finally, the three improvements were combined to test the performance of the overall model. By comparing the experimental results of the baseline model, the single-improvement model, and the fused-improvement model, we can clearly demonstrate the contribution of each improvement to detection performance, especially the performance improvement in complex scenes (such as occlusion and scale changes).

[0123] Experimental results show that the model integrating the three improvements significantly outperforms the baseline model and the single-improvement model in terms of mAP50, mAP50-95, and Recall (%), while maintaining a low computational complexity (GFLOPS), verifying the effectiveness and practicality of the proposed improvements. Table 4 shows the performance comparison results of the detector module.

[0124] Table 4 Detector module performance comparison results

[0125] Methods baseline CCFM √ √ √ EfficientNetV1 √ √ √ Inner_Wise_WIoU √ map50(%) 79.313 80.366 80.174 80.47 81.178 map50-95(%) 52.978 53.429 53.807 53.561 54.019 precision(%) 87.864 89.478 88.085 90.027 90.914 recall(%) 68.23 69.69 70.2 69.39 71.227 GFLOPS 4.19 3.33 2.99 2.13 2.13

[0126] As shown in Table 4, the introduction of EfficientNetV1 and CCFM can improve the performance of Yolov10. After fusing EfficientNetV1 and CCFM, the performance is further improved and GFLPOS is reduced to half of Yolov10. After introducing the Inner_Wise_WIoU loss function into the Yolov10-EficientCCFM model proposed in this paper, compared with Yolov10, Map50 is improved by 1.86%, Map50-95 is improved by 1.041%, precision is improved by 3.05%, and recall is improved by 2.997%.

[0127] (3) Ablation test to improve the tracker:

[0128] We added the optimized modules to the ByteTrack algorithm, analyzing the performance improvements of each module. We also compared the tracking performance of the ByteTrack + CCFM + EfficientNetV1 + Inner_Wise_WIoU module with a threshold-optimized tracking algorithm based on pedestrian occlusion. Table 5 shows a comparison of the modular tracking performance on the MOT16 dataset, and Table 6 shows a performance comparison between the ByteTrack algorithm and the proposed algorithm.

[0129] Table 5. Performance comparison of modules on the MOT16 dataset

[0130]

[0131] Table 6 Performance comparison of ByteTrack algorithm and proposed algorithm

[0132] Tracking algorithm HOTA% MOTA% IDF1% IDS ByteTrack 43.301 39.537 49.701 484 The present invention 46.195 44.35 53.285 490

[0133] Tables 5 and 6 show that replacing the loss function Inner_Wise_WIoU improves localization accuracy and small target detection, resulting in improvements of 0.584% in HOTA, 1.972% in MOTA, and 0.319% in IDF1. Integrating EfficientNetV1 with a balanced scaling of network depth, width, and resolution improves convolutional neural network performance, resulting in improvements of 1.065% in HOTA, 0.216% in MOTA, and 1.191% in IDF1. To enhance the accuracy of the tracking algorithm in occluded scenarios for multi-target pedestrian tracking, an occlusion ratio is introduced. When occlusion is severe (occlusion ratio > 0.5), the threshold is lowered to 90% of the initial threshold; when occlusion is moderate (occlusion ratio > 0.3), the threshold is lowered to 95% of the initial threshold; when there is no occlusion or the occlusion is minimal, the initial threshold is restored. Therefore, compared with the ByteTrack algorithm, the proposed algorithm improves HOTA by 2.897%, MOTA by 4.813%, and IDF1 by 3.584% while keeping the ID switching basically unchanged.

[0134] The above embodiments are preferred examples for implementing the present invention, and the present invention is not limited to the above embodiments. Any non-essential additions or replacements made by those skilled in the art based on the technical features of the present invention fall within the scope of protection of the present invention.

Claims

1. Pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment, characterized by: The following operations are included: 1) Build the YOLOV10-EfficientCCFM model of the object detector, which includes the backbone network, neck network and detection head; The backbone network replaces the YOLOV10 backbone network with EfficientNetV1, introduces a composite scaling mechanism, and uses a fixed scaling factor to uniformly scale the network width, depth, and resolution. The neck network integrates the lightweight cross-scale feature fusion module (CCFM) into the YOLOV10 neck network to fuse feature maps of different scales. At the same time, the attention mechanism and multi-scale feature aggregation are used to improve the contextual information expression capability of the feature maps. The detection head uses the YOLO detection head to detect and locate multi-target pedestrians, and outputs the pedestrian position and confidence results. The loss function is replaced by the Inner_Wise_WIoU loss function shown in formula (4). The closer the calculation result of the loss function is to 0, the better the match between the predicted area and the true area. Where B is the predicted bounding box area, B gt is the area of ​​the true bounding box, W total is the dynamic weight matrix, W total ·(B∩B gt ) takes into account the importance of different locations and weights the contribution of the intersection area; ΣW total ·(B∪B gt ) is a double-weighted summation of the union area, taking into account the weights of the union areas at different positions, and r is the dynamic gradient gain factor; 2) Train the target detector YOLOV10-EficientCCFM model based on the dataset; 3) ByteTrack is selected as the multi-target tracking algorithm. The tracker first detects the target bounding box and confidence score from the video frame using an object detector. Based on the confidence score, the detected target boxes are divided into high-scoring detection boxes and low-scoring detection boxes. Then, the matching threshold of the high-scoring detection boxes and the low-scoring detection boxes is dynamically adjusted according to the occlusion ratio between the targets. Then, the first data association is performed, and the high-scoring detection box targets are matched with the existing trajectories predicted by the Kalman filter using the Hungarian algorithm to perform IoU matching, and the trajectory position and motion state are updated. High-confidence targets that are not matched are initialized as new trajectories and remaining trajectories. Then, in the second data association phase, a second IoU matching is performed on the low-resolution detection box targets output by the detector and the remaining tracks that are temporarily retained after the first data association phase. If the match is successful, the lost track is restored. Otherwise, the target with the unmatched track is considered a new target and temporarily stored. Finally, lifecycle management is performed on continuously unmatched trajectories. If the target does not appear after the number of lost frames exceeds the set number of frames, the target is considered lost and the trajectory is deleted; the unique ID and position coordinates of each target are output.

2. The pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment according to claim 1 is characterized in that: The width, depth and resolution of the uniformly scaled network are: Adjust the network depth to: d = α φ ; Adjust the network width to: w = β φ ; Adjust the network resolution to: r = γ φ ; Among them, α is the depth scaling factor, β is the width scaling factor, γ is the resolution scaling factor, and φ is the global compounding factor.

3. The pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment according to claim 1 is characterized in that: In the Inner_Wise_WIoU loss function, when the sample abnormality is high due to severe occlusion, the gradient suppression strength is increased by adjusting r to reduce the interference of low-quality predictions on the loss calculation.

4. The pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment according to claim 1 is characterized in that: The target detector YOLOV10-EficientCCFM model is trained as follows: Phase 1: Replace the CSPDarknet backbone network of YOLOv10n with EfficientNetV1, retaining the original detection head and neck structure; During the training process, the initial learning rate is used for multi-batch training, and the training model EfficientNetV1 enhances the multi-scale feature extraction capability through a compound scaling mechanism; The second stage: Based on the original CSPDarknet backbone network of YOLOv10n, the neck module is replaced with a lightweight cross-scale feature fusion module CCFM, which fuses multi-scale features through a weighted channel attention mechanism; Training the model’s ability to detect small targets at long distances; Phase 3: Integrating the improvements from the first two phases, we built a joint model combining the EfficientNetV1 backbone network and the CCFM neck module. We also trained the collaboration between EfficientNetV1 and CCFM to improve the model's adaptability to complex scenarios. The fourth stage: Based on the third stage, the original IoU loss function is replaced by the Inner_Wise_WIoU loss function; the training combines the effectiveness of internal optimization and contextual information.

5. The pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment according to claim 1 is characterized in that: If the confidence score of the target bounding box output by the detector is greater than 0.5, the detected target box is a high-score detection box; If the confidence score of the target bounding box output by the detector is >0.1 and <0.5, the detected target box is a low-score detection box.

6. The pedestrian multi-target detection and tracking algorithm based on cross-layer fusion and dynamic adjustment according to claim 1 is characterized in that: Dynamic adjustment of the matching threshold includes: ① When the occlusion ratio is greater than 0.5, the occlusion is severe and the threshold is reduced to 90% of the initial threshold. ② When the occlusion ratio is greater than 0.3, it is considered moderate occlusion, and the threshold is lowered to 95% of the initial threshold. ③ When there is no occlusion or the occlusion ratio is ≤0.3, restore the initial threshold.

Images