Online multi-target tracking method and system based on feature fusion and iterative association

Abstract

The present disclosure relates to an online multi-target tracking method and system based on feature fusion and iterative association, and relates to the technical field of target tracking. The online multi-target tracking method comprises: fusing and updating confidence degrees of possible targets in each image in a given video sequence by using position information corresponding to the possible targets and the confidence degrees, to obtain fused and updated confidence degrees; determining a plurality of threshold values corresponding to the fused and updated confidence degrees by using an iterative algorithm or an adaptive threshold method; dividing the fused and updated confidence degrees into a plurality of target detection frame intervals based on the plurality of threshold values; and tracking the possible targets in each image in the given video sequence by using position information corresponding to the plurality of target detection frame intervals. The embodiment of the present disclosure can realize online multi-target tracking.

Description

Technical Field

[0001] This disclosure relates to the field of target tracking technology, and in particular to an online multi-target tracking method and system based on feature fusion and iterative correlation. Background Technology

[0002] Autonomous driving demands high levels of safety and efficiency. Pedestrian tracking technology can detect pedestrians and other vehicles around a vehicle in a timely manner, preventing potential traffic accidents, especially in unexpected situations. Therefore, the development of high-safety and high-efficiency pedestrian tracking technology plays a crucial role in autonomous driving technology, particularly in addressing issues such as severe occlusion, clustering, and ambiguity.

[0003] Video object tracking has a relatively short history, and early algorithms such as Mean Shift and Particle Filter had low accuracy and were unsuitable for practical applications. In recent years, with the development of object tracking, mainstream R-CNN and YOLO series object detection algorithms have achieved extremely high accuracy. Object detection-based Multi-Object Tracking (MOT) has rapidly developed and gradually become the mainstream framework, greatly promoting the development of multi-object tracking. With the increasing application of attention mechanisms such as Transformer in computer vision, researchers have recently proposed attention-based multi-object tracking frameworks. In this paper, because it specifically processes sequence information, it can perfectly perform frame-by-frame operations on video sequences. In addition, there is also a method using Graph Convolutional Networks (GCNs) to solve the MOT problem. This method treats a group of detected objects in consecutive frames as a node and completes the matching using the Hungarian algorithm, which is an offline tracking method.

[0004] Specifically, Tracking-by-detection (TBD) frameworks have made significant progress in multi-object tracking. Existing methods typically perform frame-by-frame object detection on the video sequence, using bounding boxes to identify the objects, and then transform the problem into a correlation problem between consecutive frames. However, when the target object faces occlusion, especially occlusion by similar objects, the reliability of the matching decreases, particularly in cases of severe occlusion.

[0005] During tracking, techniques such as Kalman filters or deep learning models can be used, allowing for accurate prediction of the object's location in the next frame. For example, in 2016, Alex Bewley et al. proposed SORT, an online real-time multi-object tracking method that uses convolutional neural networks to detect pedestrians, making it one of the first multi-object tracking algorithms. Furthermore, the SORT algorithm uses Faster-RCNN for object detection to obtain corresponding location coordinates (location information), classification, and confidence level, and feeds the object's state (e.g., location coordinates, classification, confidence level) back to the Kalman filter for object association. However, it ignores the appearance features of the detected object, losing its tracking value in practical applications. DeepSORT is an improvement on SORT, introducing appearance features into the data association to improve tracking robustness.

[0006] Similar to DeepSORT, MOTDT's main purpose is to enhance the correlation efficiency of SORT and DeepSORT. Various data correlation methods such as SORT, DeepSORT, FairMOT, and MeMOT utilize positional and motion cues or appearance features in different ways to achieve robust tracking results.

[0007] Bytetrack proposes a general association method that associates almost every detection box, not just high-resolution ones. OC-SORT designs a tracker based on a pure motion model to improve tracking accuracy in crowded scenes. Some short-term trackers use independent re-identification (reID) mechanisms for long-term association.

[0008] In practical applications, efficiently and quickly resolving MOT (Moment of Target Loss) issues is a challenging task due to target occlusion, loss, clustering, and complexity changes in crowded scenarios. Among these influencing factors, target loss caused by high-cluster occlusion has a significant impact on the performance of MOT trackers. Summary of the Invention

[0009] This disclosure proposes a technical solution for an online multi-target tracking method and system based on feature fusion and iterative correlation.

[0010] According to one aspect of this disclosure, an online multi-target tracking method based on feature fusion and iterative association is provided, comprising: fusing and updating the confidence level using the location information and confidence level corresponding to possible targets within each frame of a given video sequence to obtain a fused updated confidence level;

[0011] Multiple thresholds corresponding to the fusion update confidence level are determined using iterative algorithms or adaptive thresholding methods.

[0012] Based on the multiple thresholds, the fusion update confidence is divided into multiple target detection box intervals;

[0013] Using the location information corresponding to the multiple target detection box intervals, the tracking of possible targets within each frame of the given video sequence is performed.

[0014] Preferably, the method for fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence to obtain a fused updated confidence score includes: determining the corresponding target bounding box or target detection box based on the location information of possible targets within each frame; calculating the height of the target bounding box or target detection box corresponding to the possible target and the height of each frame, and determining the corresponding height difference; fusing and optimizing the confidence score based on the height difference to obtain a fused updated confidence score; and updating the fused and optimized confidence score using the fused updated confidence score.

[0015] Preferably, the method for calculating the target bounding box or target detection box corresponding to the possible target and the height corresponding to each frame of the image, and determining the corresponding height difference, includes: determining the ordinate of the lowest point or the highest point of the target bounding box or target detection box corresponding to the possible target; calculating a first difference between the height corresponding to each frame of the image and the ordinate, and determining the corresponding height difference.

[0016] Preferably, the method for fusing and optimizing the confidence score based on the height difference to obtain a fused updated confidence score includes: obtaining the information range level of the target bounding box or target detection box corresponding to the possible target; multiplying the information range level by the height difference to obtain the height difference corresponding to the information range level; calculating a second difference between I and the height difference corresponding to the information range level; and fusing and optimizing the confidence score based on the second difference to obtain a fused updated confidence score.

[0017] Preferably, the method for fusing and optimizing the confidence score based on the second difference to obtain a fused updated confidence score includes: obtaining adjustable weight parameters; adjusting the confidence score using the adjustable weight parameters to obtain an adjusted confidence score; and adding the second difference to the adjusted confidence score to obtain a fused updated confidence score.

[0018] Preferably, the method for adjusting the confidence level using the adjustable weight parameter to obtain the adjusted confidence level includes: multiplying the adjustable weight parameter by the confidence level to obtain the adjusted confidence level.

[0019] Preferably, the method for determining multiple thresholds corresponding to the fusion update confidence using an iterative algorithm includes: establishing an objective function containing the inter-class variances corresponding to the multiple thresholds to be determined. Iteratively calculate the average fusion update confidence U corresponding to each changing threshold among the multiple thresholds to be determined. i or μ i The average confidence level U corresponding to the fusion update confidence level T or μ T The sum of squares corresponding to multiple third differences between them; during the iterative calculation, the objective function of the inter-class variance is... The multiple thresholds corresponding to the maximum value are determined as multiple thresholds; wherein, the sum of the multiple thresholds is configured to be 1.

[0020] Preferably, the step of establishing an objective function for the inter-class variance corresponding to multiple thresholds to be determined is... The process also includes: obtaining a preset threshold and establishing an inter-class variance objective function that includes the preset threshold and multiple thresholds to be determined; wherein, the preset threshold is configured as the threshold corresponding to the closest non-possible target to the background in each frame of the given video sequence; wherein, the preset threshold and the sum of the multiple thresholds are configured to be 1.

[0021] Preferably, the iterative calculation of the average fusion update confidence level U corresponding to each changing threshold among the multiple thresholds to be determined i or μ i The average confidence level U corresponding to the fusion update confidence level T or μ T The method for determining the sum of squares corresponding to multiple third differences includes: obtaining a set number of iterations, multiple initial change thresholds corresponding to the multiple thresholds to be determined, and an initial change average fusion update confidence level U. i or μ i and its corresponding first set change rate, and the second set change rate corresponding to the plurality of initial thresholds; calculate the average confidence level U corresponding to the fusion update confidence level. T or μ T Based on the multiple initial change thresholds corresponding to the multiple thresholds to be determined, and the initial change average fusion update confidence level U, i or μ i The first set change rate and the second set change rate corresponding to the plurality of initial thresholds, and the average fusion update confidence U of the change of each change threshold among the plurality of thresholds to be determined within the set number of iterations. i or μ i The average confidence level U corresponding to the fusion update confidence level T or μ TThe sum of squares corresponding to multiple third differences between them.

[0022] Preferably, when the number of iterations is greater than 1, the average fusion update confidence level U corresponding to each change threshold among the multiple thresholds to be determined is... i or μ i The configuration is to multiply the average fusion update confidence of the changes in the previous iteration by the first set change rate; each of the multiple thresholds to be determined is configured as the change threshold in the previous iteration multiplied by the second set change rate.

[0023] Preferably, the method for determining multiple thresholds corresponding to the fusion update confidence using the adaptive threshold method includes: sorting the fusion update confidence by size to obtain sorted fusion update confidence; determining the clustering interval corresponding to the set clustering degree based on the sorted fusion update confidence and a set clustering degree; and determining multiple thresholds corresponding to the fusion update confidence using the clustering interval and the number corresponding to the multiple thresholds.

[0024] Preferably, the method for determining the multiple thresholds corresponding to the fusion update confidence level using the number of clustering intervals and the multiple thresholds includes: determining multiple clustering interval coefficients corresponding to the multiple thresholds based on the minimum and / or maximum fusion update confidence levels corresponding to the sorted fusion update confidence levels, the number of clustering intervals and the multiple thresholds; determining the multiple thresholds on the curve corresponding to the sorted fusion update confidence levels based on the multiple clustering interval coefficients; wherein, the interval range corresponding to each clustering interval coefficient includes the clustering interval.

[0025] Preferably, the method for determining the multiple clustering interval coefficients corresponding to the multiple thresholds based on the minimum and / or maximum fusion update confidence corresponding to the sorted fusion update confidence, the clustering intervals, and the number of the multiple thresholds includes: taking the minimum and / or maximum fusion update confidence as starting points, respectively determining the multiple clustering interval coefficients corresponding to the multiple thresholds, including the clustering intervals.

[0026] Preferably, based on the multiple thresholds corresponding to the fusion update confidence determined by the iterative algorithm, the multiple thresholds corresponding to the fusion update confidence determined by the adaptive thresholding method are verified; and / or, the number of the multiple thresholds is configured to be 2, and the number of the multiple target detection box intervals into which the fusion update confidence is divided is 3.

[0027] Preferably, when the number of target detection box intervals corresponding to the fusion update confidence is 3 or the number of thresholds is configured to be 2, the minimum fusion update confidence and the maximum fusion update confidence are used as starting points to determine two clustering interval coefficients corresponding to the two thresholds, including the clustering interval; wherein, the values ​​corresponding to the two clustering interval coefficients are the same, and the values ​​corresponding to the two clustering interval coefficients are configured to be 2 / 3.

[0028] Preferably, the method for dividing the fusion update confidence into multiple target detection box intervals based on the multiple thresholds includes: configuring the interval from the minimum fusion update confidence to the minimum threshold among the multiple thresholds as a first detection box interval; configuring the intervals corresponding to adjacent thresholds among the multiple thresholds as multiple second detection box intervals; and configuring the interval from the maximum threshold among the multiple thresholds to the maximum fusion update confidence as a third detection box interval.

[0029] Preferably, the number of the multiple thresholds is configured to be 2, and the two thresholds are respectively configured as a first threshold and a second threshold; the fusion update confidence is divided into 3 target detection box intervals using the first threshold and the second threshold, and the 3 target detection box intervals are respectively configured as a high-scoring detection box interval (third detection box interval), a medium-scoring detection box interval (second detection box interval) and a low-scoring detection box interval (first detection box interval).

[0030] Preferably, the method for tracking possible targets within each frame of a given video sequence using the position information corresponding to the plurality of target detection box intervals includes: determining position information corresponding to trajectory marker detection box intervals for initializing the trajectory from the position information corresponding to the plurality of target detection box intervals; initializing a list of tracks being tracked using the position information corresponding to the trajectory marker detection box intervals; and tracking possible targets within each frame of the given video sequence based on the initialized list of tracks being tracked and the position information corresponding to the plurality of target detection box intervals.

[0031] Preferably, the method for determining the location information corresponding to the trajectory marker detection box interval for trajectory initialization from the location information corresponding to the plurality of target detection box intervals includes: configuring the intervals corresponding to adjacent thresholds among the plurality of thresholds as a plurality of second detection box intervals; configuring the interval corresponding to the maximum threshold among the plurality of thresholds to the maximum fusion update confidence among the fusion update confidence as a third detection box interval; and determining the location information corresponding to the trajectory marker detection box interval for trajectory initialization from the location information corresponding to the third detection box interval and / or any one or more detection box intervals among the plurality of second detection box intervals.

[0032] Preferably, the method for tracking possible targets within each frame of a given video sequence based on the initialized list of tracked trajectories and the position information corresponding to the plurality of target detection box intervals includes: predicting the trajectories in the initialized list of tracked trajectories to obtain corresponding predicted position information; and tracking possible targets within each frame of the given video sequence based on the predicted position information and the position information corresponding to the plurality of target detection box intervals.

[0033] Preferably, the method for tracking possible targets within each frame of a given video sequence based on predicted location information and location information corresponding to the plurality of target detection box intervals includes: constructing a corresponding first cost matrix using the predicted location information and location information corresponding to a third detection box interval from the maximum threshold among multiple thresholds in the plurality of target detection box intervals to the maximum fusion update confidence among the fusion update confidence; matching the predicted location information in the first cost matrix with the location information to obtain a first matching trajectory corresponding to a first optimal matching combination, a first unmatched detection value, and a first unmatched trajectory corresponding to the predicted location information or the tracked trajectory list; wherein the first unmatched detection value is configured as the location information corresponding to the third detection box interval; constructing a plurality of second cost matrices using the location information corresponding to multiple second detection box intervals corresponding to adjacent thresholds among the plurality of thresholds, the first unmatched detection value, and the first unmatched trajectory; and matching the first unmatched trajectory in the plurality of second cost matrices with the location information. The process involves obtaining the second matching trajectory, the second unmatched detection value, and the second unmatched trajectory corresponding to the first unmatched trajectory, all corresponding to the second optimal matching combination. The second unmatched detection value is configured as the location information corresponding to the second detection box interval. A third cost matrix is ​​constructed using the location information corresponding to the first detection box interval and the second unmatched trajectory, based on the minimum fusion update confidence in the fusion update confidence to the minimum threshold among the multiple thresholds. The second unmatched trajectory in the third cost matrix is ​​then matched with the location information to obtain the third matching trajectory, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory, all corresponding to the third optimal matching combination. The first unmatched detection value is configured as the location information corresponding to the third detection box interval. Based on the first matching trajectory, the second matching trajectory, and the third matching trajectory, possible targets within each frame of the given video sequence are tracked. And / or, the trajectories in the tracked trajectory list are updated using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

[0034] Preferably, the method for constructing multiple second cost matrices using the position information corresponding to multiple second detection box intervals corresponding to adjacent threshold intervals among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory includes: configuring the multiple second detection box intervals as second detection box intervals with sequentially decreasing fusion update confidence and cascaded; constructing a first sub-cost matrix among the multiple second cost matrices using the position information corresponding to the second sub-detection box interval with the highest fusion update confidence among the intervals corresponding to adjacent thresholds among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory; constructing a second sub-cost matrix among the multiple second cost matrices using the position information corresponding to the second sub-unmatched trajectory and the second sub-unmatched detection value of the first sub-cost matrix, and the position information corresponding to the detection box interval cascaded with the second sub-detection box interval with the highest fusion update confidence; and constructing the sub-cost matrix to be constructed this time by using the position information corresponding to the unmatched trajectory and the unmatched detection value of the previously constructed sub-cost matrix, and the position information corresponding to the detection box interval of the sub-cost matrix to be constructed this time.

[0035] Preferably, the method for matching the first unmatched trajectory with the location information in the plurality of second cost matrices includes: determining the coordinates of a first bounding box describing the first unmatched trajectory and the coordinates of a second bounding box describing the location information in the plurality of second cost matrices; determining the corresponding intersection-union ratio (IU) and width IU based on the first bounding box coordinates and the second bounding box coordinates; determining the corresponding width modulation IU based on the IU and the width IU; and performing width modulation IU matching on the predicted location information and the location information in the second cost matrices.

[0036] Preferably, the method for determining the corresponding width intersection-union ratio based on the coordinates of the first rectangular frame and the coordinates of the second rectangular frame includes: determining the coordinates of a first point corresponding to the coordinates of the first rectangular frame and the coordinates of a second point corresponding to the coordinates of the second rectangular frame; wherein the coordinates of the first point and the coordinates of the second point are diagonally related; determining the maximum and minimum horizontal coordinates of the first horizontal coordinate and the second horizontal coordinate of the second point; and determining the corresponding width intersection-union ratio based on the maximum and minimum horizontal coordinates.

[0037] Preferably, the method for determining the corresponding crossover ratio and width crossover ratio based on the maximum abscissa and the minimum abscissa respectively includes: calculating the difference between the first abscissa corresponding to the minimum abscissa and the maximum abscissa respectively; calculating the difference between the second abscissa corresponding to the maximum abscissa and the minimum abscissa respectively; and calculating the ratio of the first abscissa difference to the second abscissa difference respectively to determine the corresponding width crossover ratio.

[0038] Preferably, before performing width modulation cross-connection ratio (WCR) matching on the predicted location information and the location information in the second cost matrix, the method includes: adjusting the WCR using preset adjustment parameters to obtain an adjusted WCR; and matching the predicted location information and the adjusted WCR on the predicted location information and the location information in the second cost matrix.

[0039] Preferably, the method for performing width modulation intersection-union ratio (WMCI) matching on the predicted location information and the adjusted location information in the second cost matrix includes: obtaining a Hungarian matching algorithm and performing WMCI matching on the predicted location information and the adjusted location information in the second cost matrix using the Hungarian matching algorithm.

[0040] Preferably, the method of adjusting the width modulation crossover ratio using a preset adjustment parameter to obtain the adjusted width modulation crossover ratio includes: calculating the preset adjustment parameter power corresponding to the width modulation crossover ratio to obtain the adjusted width modulation crossover ratio.

[0041] Preferably, the method for determining the preset adjustment parameter includes: obtaining adjacent thresholds among the multiple thresholds corresponding to each of the multiple second detection frame intervals; calculating the difference between the adjacent thresholds; and determining the preset adjustment parameter.

[0042] Preferably, the method for calculating the difference between adjacent thresholds is configured to be a value greater than or equal to 0; or, the method for calculating the difference between adjacent thresholds includes: determining the maximum threshold and the minimum threshold corresponding to the adjacent thresholds; and calculating the difference between the maximum threshold and the minimum threshold.

[0043] Preferably, the method for determining the preset adjustment parameter includes: determining a set order-of-magnitude parameter for the intersection-union ratio or width intersection-union ratio corresponding to a plurality of second cost matrices; and determining the preset adjustment parameter based on the difference between adjacent thresholds and the set order-of-magnitude parameter.

[0044] Preferably, the method for determining the preset adjustment parameter based on the difference between adjacent thresholds and the set order of magnitude parameter includes: calculating the ratio of the difference between adjacent thresholds to the set order of magnitude parameter; and configuring or determining the ratio as the corresponding preset adjustment parameter.

[0045] Preferably, the method for determining the corresponding width modulation cross-connect ratio based on the cross-connect ratio and the width cross-connect ratio respectively includes: multiplying the cross-connect ratio by the corresponding width cross-connect ratio to determine the corresponding width modulation cross-connect ratio.

[0046] Preferably, the coordinates of the first point and the second point, which are diagonally related, are configured as the upper left corner of the first rectangle and the lower right corner of the second rectangle; or, the coordinates of the first point and the second point, which are diagonally related, are configured as the lower left corner of the first rectangle and the upper right corner of the second rectangle; or, the coordinates of the first point and the second point, which are diagonally related, are configured as the upper right corner of the first rectangle and the lower left corner of the second rectangle; or, the coordinates of the first point and the second point, which are diagonally related, are configured as the lower right corner of the first rectangle and the upper left corner of the second rectangle.

[0047] Preferably, the method for matching the predicted location information in the first cost matrix with the location information includes: obtaining a Hungarian matching algorithm, and using the Hungarian matching algorithm to perform intersection-union-ratio matching on the predicted location information in the first cost matrix with the location information.

[0048] Preferably, the method for matching the second unmatched trajectory with the location information in the third cost matrix includes: obtaining a Hungarian matching algorithm, and using the Hungarian matching algorithm to match the second unmatched trajectory with the location information in the third cost matrix to perform intersection-union matching.

[0049] Preferably, before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of the given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of the given video sequence.

[0050] Preferably, the method for detecting the location information and confidence level of a possible target in each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of a possible target in each frame of a given video sequence.

[0051] According to one aspect of this disclosure, an online multi-target tracking device / system based on feature fusion and iterative correlation is provided, comprising:

[0052] The fusion update unit is used to fuse and update the confidence level by using the location information and confidence level of possible targets in each frame of a given video sequence to obtain the fused updated confidence level.

[0053] The threshold determination unit is used to determine multiple thresholds corresponding to the fusion update confidence determination using an iterative algorithm;

[0054] The target detection box interval division unit is used to divide the fused update confidence into multiple target detection box intervals based on the multiple thresholds.

[0055] The target tracking unit is used to track possible targets within each frame of the given video sequence by utilizing the position information corresponding to the multiple target detection box intervals.

[0056] According to one aspect of this disclosure, an online multi-target tracking system based on feature fusion and iterative correlation is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described online multi-target tracking method.

[0057] According to one aspect of this disclosure, an online multi-target tracking system based on feature fusion and iterative correlation is provided, comprising: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above-described online multi-target tracking method.

[0058] According to one aspect of this disclosure, an online multi-target tracking system based on feature fusion and iterative correlation is provided, comprising: a computer program product including a computer program / instructions that, when executed by a processor, implement the above-described online multi-target tracking method.

[0059] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the above-described online multi-target tracking method.

[0060] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described online multi-target tracking method.

[0061] In the embodiments of this disclosure, a technical solution for an online multi-target tracking method and system based on feature fusion and iterative association is proposed to solve at least one technical problem, such as the tendency for incorrect matching to occur in existing challenging scenarios and the limited differentiation provided by appearance information when two adjacent targets highly overlap.

[0062] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0063] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0064] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0065] Figure 1 A flowchart illustrating an online multi-target tracking method based on feature fusion and iterative correlation according to an embodiment of the present disclosure is shown.

[0066] Figure 2 A block diagram is shown corresponding to the online multi-target tracking method based on feature fusion and iterative correlation according to an embodiment of the present disclosure;

[0067] Figure 3 A schematic diagram illustrating depth information in each frame of a given video sequence according to an embodiment of the present disclosure is shown.

[0068] Figure 4 This diagram illustrates the threshold determination process for an example of an adaptive threshold according to an embodiment of the present disclosure.

[0069] Figure 5 This diagram illustrates the principle of width modulation IOU according to an embodiment of the present disclosure.

[0070] Figure 6 A schematic diagram illustrating the positional relationship between the detection frame and the trajectory under different extreme cases according to embodiments of the present disclosure;

[0071] Figure 7 A schematic diagram showing the MOTA comparison on each subset of the validation set of two datasets according to an embodiment of the present disclosure;

[0072] Figure 8 This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment;

[0073] Figure 9 This is a block diagram illustrating an electronic device 1900 according to an exemplary embodiment. Detailed Implementation

[0074] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0075] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0076] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0077] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0078] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further.

[0079] In addition, this disclosure also provides an online multi-target tracking device / system, electronic device, computer-readable storage medium, and program based on feature fusion and iterative correlation. All of the above can be used to implement any of the online multi-target tracking methods based on feature fusion and iterative correlation provided in this disclosure. The corresponding technical solutions and descriptions are described in the relevant section on online multi-target tracking methods based on feature fusion and iterative correlation, and will not be repeated here.

[0080] Figure 1 A flowchart illustrating an online multi-target tracking method based on feature fusion and iterative correlation according to an embodiment of the present disclosure is shown, such as... Figure 1As shown, the online multi-target tracking method includes: Step S101: fusing and updating the confidence level using the location information and confidence level of possible targets within each frame of a given video sequence to obtain a fused updated confidence level; Step S102: determining multiple thresholds corresponding to the fused updated confidence level using an iterative algorithm or an adaptive thresholding method; Step S103: dividing the fused updated confidence level into multiple target detection box intervals based on the multiple thresholds; Step S104: tracking possible targets within each frame of the given video sequence using the location information corresponding to the multiple target detection box intervals. This method aims to solve at least one technical problem in existing challenging scenarios, such as the tendency for incorrect matching in associations and the limited distinguishing power of appearance information when two adjacent targets highly overlap.

[0081] Figure 2 A block diagram is shown corresponding to the online multi-target tracking method based on feature fusion and iterative correlation according to an embodiment of the present disclosure. Figure 2 As shown, this invention uses the bytetrack algorithm as a benchmark, enhances the feature representation of the detection results, and makes adaptive, detailed classifications. Similar to bytetrack, it assumes that the detection results are near the tracking values, therefore using IoU to calculate the distance between the trajectory and the detection values. It adds three data association stages to bytetrack: first, it matches detection values ​​that are identified as targets and are easily matched (high confidence) with the trajectory; second, it matches detection values ​​that are identified as targets but are easily confused (medium confidence) with the trajectory; and third, it associates the remaining low-confidence detection values ​​with the trajectory. The method proposed in this invention is highlighted in the dashed box.

[0082] Among them, the real-time MOT tracker / tracking method is based on the TBD framework. The video (given video sequence) is in units of frames. First, the target detector (hereinafter referred to as the detector) detects all possible targets in all video frames and marks the corresponding probability values. Then, based on the probability values ​​corresponding to the possible targets, the existing or updated trajectory is matched, and the matched trajectory is updated to achieve real-time multi-target tracking.

[0083] like Figure 2 As shown, this is illustrated using a set of 3 target detection bounding boxes. The basic framework of this algorithm or method is Bytetrack, with several improvements made (such as...). Figure 1The three dashed boxes correspond to the depth information fusion, the adaptive thresholding based on confidence, and the cost matrix (WMIoU distance). Given a video sequence (hereinafter referred to as video frames), each frame is first detected by the object detector, which outputs the location information and confidence of possible targets within each frame. The location information and confidence of possible targets within each frame are then fused and updated by the depth information fusion module to obtain the corresponding fused updated confidence.

[0084] like Figure 2 As shown, an adaptive thresholding module divides the fusion update confidence of possible targets in each frame of the image into three confidence intervals (high-scoring detection box, medium-scoring detection box, and low-scoring detection box) according to the confidence adaptive thresholding method. The trajectories in the tracked trajectory list are predicted by the Kalman filter algorithm to obtain the corresponding predicted position information. The predicted position information and the position information output by the target detector are used to construct the corresponding cost matrix. The predicted position information in the cost matrix is ​​matched with the position information output by the target detector using the Hungarian matching algorithm to obtain the matched trajectory (matched trajectory), the unmatched detection value (position information output by the target detector), and the unmatched trajectory (trajectories in the tracked trajectory list) corresponding to the optimal matching combination.

[0085] like Figure 2 As shown, the parameters corresponding to the position information of the Kalman filter algorithm are updated by using the corresponding cost matrix and the optimal matching combination (matched trajectory) obtained by the Hungarian matching algorithm for the three confidence intervals (high-scoring detection box, medium-scoring detection box, and low-scoring detection box); and the trajectory in the tracked trajectory list is updated by using the optimal matching combination (matched trajectory) corresponding to the three confidence intervals (high-scoring detection box, medium-scoring detection box, and low-scoring detection box).

[0086] like Figure 2 As shown, the second data association matching uses width-based WMIoU matching. Detection values ​​that did not match in the first and second data association matching are added to the third data association matching, ultimately completing the trajectory update for one frame. Trajectories that did not match in the third data association are marked as lost trajectories and are saved for a set number of frames (e.g., 30 frames). During the saving period, they participate in subsequent target matching; if no match is found after the time limit, they are discarded.

[0087] Step S101: Use the location information and confidence level of possible targets in each frame of the given video sequence to fuse and update the confidence level, and obtain the fused updated confidence level.

[0088] In embodiments of this disclosure, before fusing and updating the confidence score using the location information and confidence score corresponding to possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score corresponding to possible targets within each frame of a given video sequence.

[0089] In embodiments of this disclosure, the given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians. For example, a camera or video camera can be used to capture images of vehicles and / or pedestrians to obtain the video sequence corresponding to the vehicles and / or pedestrians.

[0090] In embodiments of this disclosure, the method for detecting the location information and confidence level of a possible target within each frame of a given video sequence includes: acquiring a target detector (setting a target detector or detector); and using the target detector to detect the location information and confidence level of a possible target within each frame of a given video sequence.

[0091] In the embodiments of this disclosure and other possible embodiments, the target detector (set target detector or detector) corresponding to the target detection based on the TBD (tracking-by-detection) paradigm / framework detects the target of interest in each frame of the given video sequence and obtains the corresponding location coordinates (location information), classification, and confidence level.

[0092] Specifically, in the embodiments of this disclosure and other possible embodiments, based on the TBD (tracking-by-detection) paradigm / framework for target tracking, target detection is divided into two parts. First, an efficient detector is used to detect the target of interest in each frame, obtaining the corresponding location coordinates (location information), classification, and confidence level. Second, the same target in two adjacent frames is associated. A common association method is to calculate the similarity between the objects framed by each localization box between consecutive frames, and give the identity ID corresponding to each detection box in the current frame based on the similarity matrix, thus achieving tracking.

[0093] However, in challenging scenarios, such as occlusion and clustering, associations are prone to mismatches, and the distinction provided by appearance information is limited when two adjacent targets highly overlap. To overcome these shortcomings, this invention aims to design a new model, method, device, or system within a TBD-based framework, based on advanced detectors. This model can effectively promote the integration of detection and tracking and efficiently handle target matching in complex scenarios.

[0094] In embodiments of this disclosure and other possible embodiments, based on detection-by-detection, object detection (position information and confidence level corresponding to possible targets) obtained from the detector is used to track the same object in subsequent frames.

[0095] In embodiments of this disclosure, the method for fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence to obtain a fused updated confidence score includes: determining a corresponding target bounding box or target detection box based on the location information of possible targets within each frame; calculating the height of the target bounding box or target detection box corresponding to the possible target and the height of each frame, and determining the corresponding height difference (H-y0); fusing and optimizing the confidence score S based on the height difference (H-y0) to obtain a fused updated confidence score S0; and updating the fused updated confidence score using the fused updated confidence score.

[0096] In embodiments of this disclosure, the method for calculating the target bounding box or target detection box corresponding to the possible target and the height corresponding to each frame of the image, and determining the corresponding height difference, includes: determining the ordinate of the lowest or highest point of the target bounding box or target detection box corresponding to the possible target; calculating a first difference between the height corresponding to each frame of the image and the ordinate, and determining the corresponding height difference.

[0097] In embodiments of this disclosure, the method for fusing and optimizing the confidence score based on the height difference to obtain a fused and updated confidence score includes: obtaining the information range level 10 of the target bounding box or target detection box corresponding to the possible target. -3 The information range level 10 -3 Multiplying by the height difference (H-y0) yields the height difference (H-y0) corresponding to the information range level. * 10 -3 Calculate the second difference between 1 and the height difference corresponding to the information range level: 1 - (H - y0). * 10 -3 Based on the second difference 1-(H-y0) * 10 -3 The confidence scores are then fused and optimized to obtain the fused updated confidence score S0.

[0098] In embodiments of this disclosure, the method for fusing and optimizing the confidence score based on the second difference to obtain a fused updated confidence score includes: obtaining an adjustable weight parameter α; adjusting the confidence score S using the adjustable weight parameter α to obtain an adjusted confidence score; and adding the second difference 1-(H-y0) to the adjusted confidence score. *10 -3 The fusion update confidence level S0 is obtained.

[0099] In embodiments of this disclosure, the method for adjusting the confidence level using the adjustable weight parameter to obtain an adjusted confidence level includes: multiplying the adjustable weight parameter α by the confidence level S to obtain the adjusted confidence level S. * α.

[0100] In the embodiments of this disclosure and other possible embodiments, the multi-target tracker under the TBD paradigm is generally composed of two parts: a target detector and a trajectory association discriminator. The overall tracking performance largely depends on the performance of the detector. Target detectors such as YOLOX have shown strong performance in detection and also perform well in complex scenes. However, a significant drawback of this paradigm is that the association between the two parts is not obvious, which makes it impossible to reflect the high efficiency and performance of the detector in the target tracking problem. This phenomenon is further aggravated by the fluctuation of matching efficiency in crowded or highly clustered scenes. Therefore, making more use of the high-confidence information output by the detector in the association part can greatly improve the overall tracking effect.

[0101] In embodiments of this disclosure and other possible embodiments, taking the YOLOX algorithm as an example, the output information when used as a detector is the detected target bounding box and confidence score. This can be considered as reliable information throughout the tracking process, combined with measures to address the problem of track performance degradation in complex scenes. This invention also finds that the location information of targets in an image can reflect the priority during association matching.

[0102] Figure 3 This diagram illustrates depth information in each frame of a given video sequence according to embodiments of the present disclosure. Figure 3 As shown, the vertical distance between the bottom edge of the pedestrian bounding box after target detection and the bottom edge of the image is used as the target's depth position information. Assuming the camera sensor position is fixed and all detected targets are on the same horizontal line, the Euclidean distance between the target and the next fixed point of the camera is positively correlated with the depth position information. Although it cannot accurately represent the actual position of the target in the 3D scene, it can promote target segmentation in complex scenes. Under the same confidence level, the depth position information is negatively correlated with the target's own feature differentiation, which is reflected in the tracking process as a higher matching degree. After combining the depth information with the confidence level, the new confidence level replaces the original confidence level (hereinafter referred to as confidence level or fusion updated confidence level). In subsequent segmentation thresholding, it is easier to perform clustering, reduce mutual interference during cascade matching, and improve the overall tracking robustness.

[0103] In the embodiments of this disclosure and other possible embodiments, the confidence level in the target detector output information ranges from 0 to 1, the target bounding box information ranges from 10⁻³, assuming the confidence level is S, the lower right corner coordinates of the bounding box (target bounding box) corresponding to the location information are assumed to be (x0, y0), and the height of each frame image is H. The confidence level S0 of the fusion update is calculated as follows:

[0104] S0 = S * α+(1-(H-y0) * 10 -3 )

[0105] Where α is an adjustable weight parameter, y0 is the height of the target bounding box, and H is the height of the entire image corresponding to each frame. This invention considers depth information to have high reliability, therefore reducing the depth information to the order of 10. 0 After adjusting the level, the correlation between the depth location information and the target's own features is adjusted. Finally, it is combined with the original confidence score, and S0 replaces S as the new confidence score, which participates in the subsequent matching and association steps.

[0106] Step S102: Using an iterative algorithm or an adaptive thresholding method, determine the multiple thresholds corresponding to the fusion update confidence level.

[0107] In the embodiments and other possible embodiments disclosed herein, during the early image segmentation process, when performing operations such as binarization before image segmentation, the desired result is to preserve all information within the corresponding region of the image. In a laboratory environment, the corresponding materials are templated; however, when applying laboratory methods to a real-world environment, this invention finds that the lighting environment has a significant impact on the results. Processing under these conditions yields unsatisfactory results: patches of black and white, with features of the black areas failing to be extracted. In this case, adaptive thresholding algorithms become particularly important. Unlike global thresholding, it places greater emphasis on contextual relationships, segmenting the original image into smaller regions for judgment, greatly reducing the impact of shadows on the image itself. In essence, this algorithm is a local threshold, and a weighted average method is generally used to obtain the local threshold, resulting in a more accurate segmented image overall. Similarly, this invention aims to enable the algorithm to "see" more easily distinguishable targets during the association process of multi-target tracking algorithms. Therefore, this invention attempts to apply this idea to multi-target tracking to further improve the algorithm's performance.

[0108] In the embodiments and other possible embodiments disclosed herein, in the ByteTrack algorithm, the authors use preset thresholds to divide different confidence intervals. Setting these thresholds requires extensive repeated experiments. Depending on the complexity of the data, different datasets have different thresholds (e.g., the MOT17 dataset is relatively dispersed, so the threshold is set to 0.6, while the MOT20 dataset is denser than the MOT17 dataset, so the threshold is set to 0.3). For a new dataset, the density can be known beforehand, and a corresponding threshold can be set accordingly. To obtain better experimental data, different subsets of the same dataset may also have different threshold divisions. Although ByteTrack decouples low-score occlusions from the scene, low-score targets are still crowded. In this case, IoU-based data association is prone to matching errors, thus limiting the tracking ability to handle occlusions. Furthermore, in actual scene changes, the density at future moments is difficult to predict. This invention requires a method for adaptively classifying low-confidence and high-confidence detections output by deep learning detectors.

[0109] In the embodiments of this disclosure and other possible embodiments, in the field of image segmentation, segmenting the target region and background region from a grayscale image, for color-balanced images, a single threshold can be used to complete the thresholding process. However, sometimes the colors of an image are uneven. In this case, using only a single threshold cannot obtain a clear and effective thresholded image. Therefore, an adaptive thresholding method, also called a local thresholding method, has emerged. Its idea is not to calculate a threshold for the entire image, but to obtain a threshold by calculating the weighted average of the neighboring regions along the axis of each pixel, and then using this threshold to process the current pixel. It can adaptively calculate different thresholds. Inspired by this, applying this method to thresholding for target tracking is also effective.

[0110] First, the threshold should not be a "dividing line" for quantity proportions; second, the threshold should possess "mathematical properties" that describe data trends. When different video sequences are fed to the detector as a whole, each frame represents a local segment. The complexity changes in these local segments make it difficult to effectively segment confidence levels using a fixed threshold set for the whole. Therefore, it is necessary to calculate the threshold separately for each frame—the so-called adaptive threshold.

[0111] In embodiments of this disclosure, the method for determining multiple thresholds corresponding to the fusion update confidence using an iterative algorithm includes: establishing an objective function containing the inter-class variances corresponding to the multiple thresholds to be determined. Iteratively calculate the average fusion update confidence U corresponding to each changing threshold among the multiple thresholds to be determined. i or μ i The average confidence level U corresponding to the fusion update confidence level Tor μ T The sum of squares corresponding to multiple third differences between them; during the iterative calculation, the objective function of the inter-class variance is... The multiple thresholds corresponding to the maximum value are determined as multiple thresholds; wherein, the sum of the multiple thresholds is configured to be 1.

[0112] In embodiments of this disclosure, the establishment of an objective function comprising multiple thresholds to be determined includes the inter-class variance. The process also includes: obtaining a preset threshold and establishing an inter-class variance objective function that includes the preset threshold and multiple thresholds to be determined; wherein, the preset threshold is configured as the threshold corresponding to the closest non-possible target to the background in each frame of the given video sequence; wherein, the preset threshold and the sum of the multiple thresholds are configured to be 1.

[0113] In embodiments of this disclosure, the iterative calculation of the average fusion update confidence U corresponding to each changing threshold among the plurality of thresholds to be determined i or μ i The average confidence level U corresponding to the fusion update confidence level T or μ T The method for determining the sum of squares corresponding to multiple third differences includes: obtaining a set number of iterations, multiple initial change thresholds corresponding to the multiple thresholds to be determined, and an initial change average fusion update confidence level U. i or μ i and its corresponding first set change rate, and the second set change rate corresponding to the plurality of initial thresholds; calculate the average confidence level U corresponding to the fusion update confidence level. T or μ T Based on the multiple initial change thresholds corresponding to the multiple thresholds to be determined, and the initial change average fusion update confidence level U, i or μ i The first set change rate and the second set change rate corresponding to the plurality of initial thresholds, and the average fusion update confidence U of the change of each change threshold among the plurality of thresholds to be determined within the set number of iterations. i or μ i The average confidence level U corresponding to the fusion update confidence level T or μ T The sum of squares corresponding to multiple third differences between them.

[0114] In embodiments of this disclosure and other possible embodiments, when the number of iterations is greater than 1, the average fusion update confidence level U corresponding to each change threshold among the plurality of thresholds to be determined is... i or μ iThe configuration is to multiply the average fusion update confidence of the changes in the previous iteration by the first set change rate; each of the multiple thresholds to be determined is configured as the change threshold in the previous iteration multiplied by the second set change rate.

[0115] In the embodiments of this disclosure and other possible embodiments, the number of particles corresponding to the multiple target detection box intervals divided into multiple target detection box intervals is 3 or the number of particles corresponding to the multiple thresholds is configured to be 2.

[0116] In embodiments of this disclosure and other possible embodiments, the metric used is the maximum between-class variance (between-class variance objective function). Among them, differences between classes The larger the value, the smaller the intra-class distance, making the data easier to classify and reducing the probability of subsequent errors. The specific calculation of the maximum inter-class variance is as follows:

[0117] ω0+ω1=1

[0118]

[0119] Where ω = [ω0, ω1] represents the proportion of the two parts after the threshold is determined, and μ = [μ0, μ1] represents the average value of each part. T For the population average, when When the threshold is at its maximum, the thresholds for dividing ω0 and ω1 are the optimal thresholds, corresponding to the first and second thresholds among the multiple thresholds, respectively. Here, μ0 represents the change-average fusion update confidence corresponding to the first threshold; μ1 represents the change-average fusion update confidence corresponding to the second threshold.

[0120] In the embodiments of this disclosure and other possible embodiments, within the threshold range of [0,1], it is necessary to find two values ​​to separate the three layers (three confidence intervals, i.e., high-scoring detection boxes, medium-scoring detection boxes, and low-scoring detection boxes) corresponding to high, medium, and low scores. The inter-class variance is expressed as follows:

[0121]

[0122] After simplification, the objective function for the inter-class variance corresponding to the final inter-class variance formula is as follows:

[0123]

[0124] Where ω0 is the low-confidence layer in the image where the unlikely target is closest to the background, ω 1、 ω2 represents the first and second thresholds corresponding to the medium and high confidence layers (high-scoring detection boxes and medium-scoring detection boxes), and μ i To calculate the average value of each part, this invention requires two thresholds, so the number of layers / intervals is configured to 3, U iU represents the mean confidence level of the currently selected layer. T W is the population mean. i =[ω 0， ω 1， ω2], the proportion of the current layer / interval, let ω0=0, at this time when At its maximum, ω 1， ω2 represents the first and second thresholds in the optimal threshold correspondence.

[0125] In the embodiments of this disclosure and other possible embodiments, the previous method finds the optimal threshold by traversing all possible thresholds, which has a time complexity of O(n2). This method obviously cannot meet the requirements of real-time performance.

[0126] In embodiments of this disclosure, the method for determining multiple thresholds corresponding to the fusion update confidence using an adaptive thresholding method includes: sorting the fusion update confidence by size to obtain sorted fusion update confidence; determining a clustering interval corresponding to the set clustering degree based on the sorted fusion update confidence and a set clustering degree; and determining multiple thresholds corresponding to the fusion update confidence using the clustering interval and the number corresponding to the multiple thresholds.

[0127] In embodiments of this disclosure, the method for determining multiple thresholds corresponding to the fusion update confidence level using the number of clustering intervals and the multiple thresholds includes: determining multiple clustering interval coefficients corresponding to the multiple thresholds based on the minimum and / or maximum fusion update confidence levels corresponding to the sorted fusion update confidence levels, the number of clustering intervals and the multiple thresholds; determining the multiple thresholds on the curve corresponding to the sorted fusion update confidence levels based on the multiple clustering interval coefficients; wherein, the interval range corresponding to each clustering interval coefficient includes the clustering interval.

[0128] In embodiments of this disclosure, the method for determining multiple clustering interval coefficients corresponding to the multiple thresholds based on the minimum and / or maximum fusion update confidence corresponding to the sorted fusion update confidence, the clustering intervals, and the number of the multiple thresholds includes: taking the minimum and / or maximum fusion update confidence as starting points, determining multiple clustering interval coefficients corresponding to the multiple thresholds, including the clustering intervals.

[0129] In the embodiments of this disclosure, based on the multiple thresholds corresponding to the fusion update confidence determination determined by the iterative algorithm, the multiple thresholds corresponding to the fusion update confidence determination determined by the adaptive threshold method are verified.

[0130] In the embodiments of this disclosure, the number of the plurality of thresholds is configured to be 2, and the number of the fusion update confidence divided into the plurality of target detection box intervals is 3.

[0131] In the embodiments of this disclosure and other possible embodiments, when the number of the multiple target detection box intervals corresponding to the fusion update confidence is 3 or the number of the multiple thresholds is configured to be 2, the minimum fusion update confidence and the maximum fusion update confidence are respectively used as the starting point to determine two clustering interval coefficients corresponding to the two thresholds, including the clustering interval; wherein, the values ​​corresponding to the two clustering interval coefficients are the same, and the values ​​corresponding to the two clustering interval coefficients are configured to be 2 / 3.

[0132] Figure 4 A schematic diagram showing the threshold determination corresponding to an adaptive threshold instance according to an embodiment of the present disclosure is provided. Figure 4 As shown, taking the fusion update confidence division into multiple target detection box intervals with a number of 3 or the multiple thresholds with a number of 2 as an example, a simple and effective method is used to obtain two thresholds for each frame. That is, by simply averaging, it can be highly close to the optimal threshold interval, and at this time the threshold is at the steepest point of the steepest descent method, such as... Figure 4 As shown, after the confidence scores are sorted in order, a clustering interval coefficient α is set to determine the curve interval corresponding to the sorted fusion update confidence scores used for threshold division. The two intervals intersect at the middle confidence level, which aims to reconfirm the relatively clear confidence scores and reduce the error rate. This is then transformed into two individual threshold divisions. This method can also be flexibly applied to threshold division of other trackers, eliminating the tedious step of repeated trials to obtain a suitable threshold.

[0133] in, Figure 4 (a) is a subset of the test set of the MOT20 dataset, MOT20-06. Figure 4 (b) is a subset of the test set of the MOT17 dataset, MOT20-03. The α coefficient of both the higher complexity (MOT20) and lower complexity (MOT17) datasets is 2 / 3. The red circles show the first and second thresholds corresponding to the two thresholds. The higher confidence level is indicated by the red circles, which represent the high-scoring detection box intervals corresponding to high confidence. The area between the two circles is the medium confidence region (medium-scoring detection box intervals), and the rest is the low confidence region (low-scoring detection box intervals).

[0134] Step S103: Based on the multiple thresholds, divide the fusion update confidence into multiple target detection box intervals.

[0135] In embodiments of this disclosure, the method for dividing the fusion update confidence into multiple target detection box intervals based on the plurality of thresholds includes: configuring the interval from the minimum fusion update confidence to the minimum threshold among the plurality of thresholds as a first detection box interval; configuring the intervals corresponding to adjacent thresholds among the plurality of thresholds as multiple second detection box intervals; and configuring the interval from the maximum threshold among the plurality of thresholds to the maximum fusion update confidence as a third detection box interval.

[0136] In the embodiments of this disclosure and other possible embodiments, the number of the plurality of thresholds is configured to be 2, and the two thresholds are respectively configured as a first threshold and a second threshold; the fusion update confidence is divided into 3 target detection box intervals using the first threshold and the second threshold, and the 3 target detection box intervals are respectively configured as a high-scoring detection box interval (third detection box interval), a medium-scoring detection box interval (second detection box interval), and a low-scoring detection box interval (first detection box interval).

[0137] Step S104: Using the position information corresponding to the multiple target detection box intervals, track possible targets within each frame of the given video sequence.

[0138] In embodiments of this disclosure and other possible embodiments, target tracking typically employs Kalman filtering and the Hungarian algorithm to achieve accurate and efficient results. Due to measurement errors, the boundary frames obtained by the target detector often contain noise and inaccuracies. The Kalman filter addresses this issue by using data fusion between mathematical model predictions and measurements, while the Hungarian algorithm performs data fusion between estimates and observations.

[0139] In embodiments of this disclosure and other possible embodiments, ByteTrack is based on the YOLOX object detector. First, the detector acquires information such as the object's location, category, and confidence level, which is then sent as input to the tracker BYTE. Similar to the SORT algorithm, BYTE uses a Kalman filter to predict the location of existing trajectories in the current frame. Then, it calculates the Intersection over Union (IoU) between the detected and predicted frames, uses the Hungarian algorithm to match the detected frames to the optimal trajectory, and assigns an ID to each tracked trajectory.

[0140] In embodiments of this disclosure and other possible implementations, the main feature of BYTE is the use of low-resolution detection frames for double matching with existing trajectories, effectively optimizing pedestrian trajectory loss and identity switching issues caused by pedestrian occlusion and motion blur. To ensure real-time performance, BYTE utilizes only motion features and location information, rather than appearance features, for data association. Therefore, its tracking performance is highly dependent on the quality of the detection model.

[0141] Based on the above, in the embodiments of this disclosure, the method for tracking possible targets within each frame of a given video sequence using the position information corresponding to the plurality of target detection box intervals includes: determining position information corresponding to a trajectory marker detection box interval for initializing the trajectory from the position information corresponding to the plurality of target detection box intervals; initializing a list of tracks being tracked using the position information corresponding to the trajectory marker detection box intervals; and tracking possible targets within each frame of the given video sequence based on the initialized list of tracks being tracked and the position information corresponding to the plurality of target detection box intervals.

[0142] In embodiments of this disclosure, the method for determining the location information corresponding to the trajectory marker detection box interval for trajectory initialization from the location information corresponding to the plurality of target detection box intervals includes: configuring the intervals corresponding to adjacent thresholds among the plurality of thresholds as a plurality of second detection box intervals; configuring the interval corresponding to the maximum threshold among the plurality of thresholds to the maximum fusion update confidence among the fusion update confidence as a third detection box interval; and determining the location information corresponding to the trajectory marker detection box interval for trajectory initialization from the location information corresponding to the third detection box interval and / or any one or more detection box intervals among the plurality of second detection box intervals.

[0143] In embodiments of this disclosure, the method for tracking possible targets within each frame of a given video sequence based on an initialized list of tracked trajectories and the location information corresponding to the plurality of target detection box intervals includes: predicting the trajectories in the initialized list of tracked trajectories to obtain corresponding predicted location information; and tracking possible targets within each frame of the given video sequence based on the predicted location information and the location information corresponding to the plurality of target detection box intervals.

[0144] In embodiments of this disclosure and other possible embodiments, the method for predicting trajectories in the initialized tracked trajectory list to obtain corresponding predicted location information includes: obtaining a Kalman filter algorithm; and using the Kalman filter algorithm to predict trajectories in the initialized tracked trajectory list to obtain corresponding predicted location information.

[0145] In embodiments of this disclosure, the method for tracking possible targets within each frame of a given video sequence based on predicted location information and location information corresponding to the plurality of target detection box intervals includes: constructing a corresponding first cost matrix using the predicted location information and location information corresponding to a third detection box interval from the maximum threshold among multiple thresholds in the plurality of target detection box intervals to the maximum fusion update confidence among the fusion update confidence; matching the predicted location information in the first cost matrix with the location information to obtain a first matching trajectory corresponding to a first optimal matching combination, a first unmatched detection value, and a first unmatched trajectory corresponding to the predicted location information or the tracked trajectory list; wherein the first unmatched detection value is configured as the location information corresponding to the third detection box interval; constructing a plurality of second cost matrices using the location information corresponding to multiple second detection box intervals corresponding to adjacent thresholds among the plurality of thresholds, the first unmatched detection value, and the first unmatched trajectory; and matching the predicted location information in the first cost matrix with the location information corresponding to the third detection box interval. The first unmatched trajectories in multiple second cost matrices are matched with their location information to obtain the second matched trajectory, the second unmatched detection value, and the second unmatched trajectory corresponding to the first unmatched trajectory, corresponding to the second optimal matching combination. The second unmatched detection value is configured as the location information corresponding to the second detection box interval. A third cost matrix is ​​constructed using the location information corresponding to the first detection box interval and the second unmatched trajectory, obtained by matching the second unmatched trajectory with the location information in the third cost matrix. The third matched trajectory, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory are then matched with their location information to obtain the third matched trajectory, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory, corresponding to the third optimal matching combination. The first unmatched detection value is configured as the location information corresponding to the third detection box interval. Based on the first matched trajectory, the second matched trajectory, and the third matched trajectory, possible targets within each frame of the given video sequence are tracked.

[0146] In embodiments of this disclosure, the method further includes updating the trajectories in the tracked trajectory list using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

[0147] In embodiments of this disclosure and other possible embodiments, the method further includes updating the parameters corresponding to the position information in the Kalman filter algorithm using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

[0148] In embodiments of this disclosure and other possible embodiments, the method of constructing multiple second cost matrices by utilizing the position information corresponding to multiple second detection box intervals corresponding to adjacent threshold intervals among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory includes: configuring the multiple second detection box intervals as second detection box intervals with sequentially decreasing and cascaded fusion update confidence; constructing a first sub-cost matrix among the multiple second cost matrices by utilizing the position information corresponding to the second sub-detection box interval with the highest fusion update confidence among the intervals corresponding to adjacent thresholds among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory; constructing a second sub-cost matrix among the multiple second cost matrices by utilizing the position information corresponding to the second sub-unmatched trajectory and the second sub-unmatched detection value corresponding to the first sub-cost matrix, and the position information corresponding to the detection box interval cascaded with the second sub-detection box interval with the highest fusion update confidence; and constructing the sub-cost matrix to be constructed this time by using the position information corresponding to the unmatched trajectory and the unmatched detection value corresponding to the previously constructed sub-cost matrix, and the position information corresponding to the detection box interval to be constructed this time.

[0149] In the embodiments of this disclosure and other possible embodiments, the spatial stability of the target is also one of the key issues in multi-target tracking (MOT). In addition to the confidence state, the height state provides informative weak cues that help compensate for the shortcomings of strong cues.

[0150] Figure 5 This diagram illustrates the principle of width modulation (WMIOU) according to an embodiment of the present disclosure. Width modulation (WMIOU) is the main focus of this invention in this part, such as... Figure 5 As shown, WMIoU simply adds the width of the rectangle corresponding to the pedestrian detection value to enhance the calculated IoU. The center position, scale, and shape of the original detection and trajectory remain unchanged, but the length of the entire matching space is expanded.

[0151] like Figure 5 As shown, the IoU is used in camera space. In the method of this invention, the coordinates of the detected bounding boxes are transformed by projecting them onto the ground plane. The IoU is calculated using these transformed rectangles. This invention increases the granularity of the cost matrix and enables more effective matching of objects and occlusions.

[0152] In embodiments of this disclosure, the method for matching the first unmatched trajectory with location information in the plurality of second cost matrices includes: determining the coordinates A of a first bounding box describing the first unmatched trajectory and the coordinates B of a second bounding box describing the location information in the plurality of second cost matrices; determining the corresponding Intersection over Union (IoU) and Width Intersection over Union (WIoU) based on the first bounding box coordinates A and the second bounding box coordinates B; determining the corresponding Width Modulation Intersection over Union (WMIoU) based on the IoU and the WMIoU; and performing WMI-based width modulation intersection over union matching on the predicted location information and the location information in the second cost matrices.

[0153] In embodiments of this disclosure and other possible embodiments, the formula corresponding to the intersection-union ratio (IoU) is configured as follows:

[0154]

[0155] In embodiments of this disclosure, the method for determining the corresponding width intersection-union ratio based on the coordinates of the first rectangular frame and the coordinates of the second rectangular frame includes: determining the coordinates of a first point b1 corresponding to the coordinates of the first rectangular frame and the coordinates of a second point b2 corresponding to the coordinates of the second rectangular frame; wherein the coordinates of the first point b1 and the coordinates of the second point b2 are diagonally related; determining the maximum and minimum horizontal coordinates of the first horizontal coordinate and the second horizontal coordinate of the second point coordinate; and determining the corresponding width intersection-union ratio based on the maximum and minimum horizontal coordinates.

[0156] In the embodiments of this disclosure and other possible embodiments, the coordinates of the first point and the second point, which are diagonally related, are configured as the upper left corner and the lower right corner corresponding to the coordinates of the first rectangle; or, the coordinates of the first point and the second point, which are diagonally related, are configured as the lower left corner and the upper right corner corresponding to the coordinates of the first rectangle; or, the coordinates of the first point and the second point, which are diagonally related, are configured as the upper right corner and the lower left corner corresponding to the coordinates of the first rectangle; or, the coordinates of the first point and the second point, which are diagonally related, are configured as the lower right corner and the upper left corner corresponding to the coordinates of the second rectangle.

[0157] For example, this invention defines these two boxes as and in Represents the top left corner. It represents the bottom right corner.

[0158] In embodiments of this disclosure, the method for determining the corresponding intersection-over-union ratio and width intersection-over-union ratio based on the maximum abscissa and the minimum abscissa respectively includes: calculating the minimum abscissa respectively. With the maximum x-coordinate The corresponding first x-coordinate difference; calculate the maximum x-coordinate respectively. With the minimum x-coordinate The corresponding second horizontal coordinate difference; calculate the ratio of the first horizontal coordinate difference to the second horizontal coordinate difference, and determine the corresponding width intersection-union ratio.

[0159] In embodiments of this disclosure and other possible embodiments, the formula corresponding to the width intersection-union ratio is configured as follows:

[0160]

[0161] In embodiments of this disclosure and other possible embodiments, the method for determining the corresponding width modulation cross-connect ratio based on the cross-connect ratio and the width cross-connect ratio respectively includes: multiplying the cross-connect ratio by the corresponding width cross-connect ratio to determine the corresponding width modulation cross-connect ratio.

[0162] In embodiments of this disclosure and other possible embodiments, the formula corresponding to the width modulation cross-union ratio is configured as follows:

[0163] WMIoU=IoU·WIoU

[0164] In embodiments of this disclosure and other possible embodiments, to better utilize height states, the present invention combines width IoU (WIoU) with conventional IoU, introducing width-modulated IoU (WMIoU). Simultaneously, considering that WIoU represents width state, a weak cue, while IoU represents spatial information, a strong cue, the present invention uses WIoU to modulate IoU, achieving enhanced recognition of occluded or clustered objects through element-wise multiplication. Furthermore, the present invention finds that IoU is effective in matching targets with low confidence and severely occluded targets.

[0165] Figure 6 The diagram illustrates the positional relationship between the detection frame and the trajectory under different extreme conditions according to embodiments of the present disclosure. For example... Figure 6 As shown, Figure 6 In (a), the trajectory does not intersect with the target bounding box, and the IoU value is 0. Figure 6 When one party completely includes the other in (b) and (c), the WIoU cannot reflect the location information represented by the included party. This is because no matter which party is included, there will always be a WIoU value that is consistent, which is obviously inconsistent with the location information of the two parties.

[0166] Based on the above, in the embodiments of this disclosure, before performing width modulation cross-connection ratio (WCR) matching on the predicted location information and location information in the second cost matrix, the method includes: adjusting the WCR using preset adjustment parameters to obtain an adjusted WCR; and matching the predicted location information and location information in the second cost matrix with the adjusted WCR.

[0167] In embodiments of this disclosure and other possible embodiments, the method for performing width modulation intersection-union ratio (WMCU) matching on the predicted location information and the adjusted location information in the second cost matrix includes: obtaining a Hungarian matching algorithm and using the Hungarian matching algorithm to perform WMCU matching on the predicted location information and the adjusted location information in the second cost matrix.

[0168] In embodiments of this disclosure and other possible embodiments, the method of adjusting the width modulation crossover ratio using a preset adjustment parameter to obtain the adjusted width modulation crossover ratio includes: calculating the preset adjustment parameter power corresponding to the width modulation crossover ratio to obtain the adjusted width modulation crossover ratio.

[0169] In embodiments of this disclosure, the method for determining the preset adjustment parameter includes: obtaining adjacent thresholds [n, m] among the multiple thresholds corresponding to each of the multiple second detection frame intervals; calculating the difference mn between the adjacent thresholds; and determining the preset adjustment parameter.

[0170] In embodiments of this disclosure and other possible embodiments, the calculation of the difference between adjacent thresholds is configured as a value greater than or equal to 0; or, the method for calculating the difference between adjacent thresholds includes: determining the maximum threshold m and the minimum threshold n corresponding to the adjacent thresholds [n,m]; and calculating the difference mn between the maximum threshold m and the minimum threshold n.

[0171] In embodiments of this disclosure, the method for determining the preset adjustment parameter includes: determining a set order-of-magnitude parameter d for the intersection-union ratio or width intersection-union ratio corresponding to a plurality of second cost matrices; and determining the preset adjustment parameter based on the difference mn between adjacent thresholds and the set order-of-magnitude parameter d.

[0172] In embodiments of this disclosure, the method for determining the preset adjustment parameter based on the difference between adjacent thresholds and the set order-of-magnitude parameter includes: calculating the ratio of the difference mn between adjacent thresholds to the set order-of-magnitude parameter d. The ratios are respectively Configure or determine the corresponding preset adjustment parameters.

[0173] In the embodiments of this disclosure and other possible embodiments, this invention avoids such phenomena by dividing the interval based on the detected confidence value. To demonstrate effectiveness, the original IoU is not used within the interval; instead, adaptive adjustments are made based on the interval size. The final calculation formula is as follows:

[0174]

[0175] Where S is the fusion update confidence corresponding to the detection value in the current frame, m and n are the high and low thresholds (multiple thresholds, such as the first threshold and the second threshold) divided by the above adaptive method, and d is the set order of magnitude parameter for adjusting IoU and WIoU.

[0176] The above method can stabilize the target separated from the complex and ever-changing environment, reduce the possibility of confusion caused by it entering the next stage (low confidence matching), and reduce the complexity of error-prone situations such as severe occlusion and background from the generation path, thereby improving the overall tracking accuracy.

[0177] In embodiments of this disclosure and other possible embodiments, the method for matching the predicted location information in the first cost matrix with the location information includes: obtaining a Hungarian matching algorithm, and using the Hungarian matching algorithm to perform intersection-union-ratio (IU) matching on the predicted location information in the first cost matrix with the location information.

[0178] In embodiments of this disclosure and other possible embodiments, the method for matching the second unmatched trajectory with the location information in the third cost matrix includes: obtaining a Hungarian matching algorithm, and using the Hungarian matching algorithm to perform intersection-union matching on the second unmatched trajectory with the location information in the third cost matrix.

[0179] In summary, this paper, based on the bytetrack algorithm, enhances the feature representation of the detection results and makes adaptive, detailed segmentation, proposing a new tracking method and system based on bytetrack. By more accurately distinguishing different confidence scores and dividing intervals, it reduces interference between adjacent targets during matching and significantly filters the background. A generalized confidence output is introduced, which, combined with the original confidence score and target position information, forms a strong correlation between the detector and the tracker under the tracking-by-detection framework. An adaptive threshold segmentation method is adopted in the tracking model to remove the limitation of fixed thresholds on the algorithm under changing scenarios.

[0180] Specifically, to alleviate the technical problems proposed in this invention, based on the high reliability of target detection, an online multi-target tracking method based on feature fusion and iterative association is proposed by strengthening the inheritance between the target detection module and the association module. This method mainly includes: enhancing the output of the target detection module to provide more reliable confidence information; employing a new iterative association step, which significantly improves the performance of tracking targets with uniform appearance; and designing an adaptive parameter setting method, which reduces parameter debugging steps, making the algorithm simpler, and improves the robustness of the algorithm under changing scene complexity. Through these improvements to the algorithm, the method of this invention can solve the tracking interruption problem caused by occlusion in online tracking scenarios. Extensive experimental results show that the method of this invention achieves promising performance levels in both unoccluded and occluded tracking scenarios.

[0181] The execution entity of the online multi-target tracking method based on feature fusion and iterative correlation can be an image processing device. For example, the online multi-target tracking method based on feature fusion and iterative correlation can be executed by a terminal device, a server, or other processing devices. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this online multi-target tracking method based on feature fusion and iterative correlation can be implemented by a processor calling computer-readable instructions stored in memory.

[0182] Those skilled in the art will understand that in the above-described online multi-target tracking method based on feature fusion and iterative correlation in specific implementations, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0183] This disclosure also proposes an online multi-target tracking device / system based on feature fusion and iterative association, comprising: a fusion update unit, configured to fuse and update the confidence level using the location information and confidence level corresponding to possible targets within each frame of a given video sequence, to obtain a fused updated confidence level; a threshold determination unit, configured to determine multiple thresholds corresponding to the fused updated confidence level using an iterative algorithm; a target detection box interval division unit, configured to divide the fused updated confidence level into multiple target detection box intervals based on the multiple thresholds; and a target tracking unit, configured to track possible targets within each frame of the given video sequence using the location information corresponding to the multiple target detection box intervals.

[0184] This disclosure also proposes an online multi-target tracking device / system based on feature fusion and iterative correlation, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described online multi-target tracking method based on feature fusion and iterative correlation.

[0185] This disclosure also proposes an online multi-target tracking system based on feature fusion and iterative correlation, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described online multi-target tracking method.

[0186] This disclosure also proposes an online multi-target tracking system based on feature fusion and iterative correlation, comprising: a computer-readable storage medium storing computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described online multi-target tracking method.

[0187] This disclosure also proposes an online multi-target tracking system based on feature fusion and iterative correlation, comprising: a computer program product, including a computer program / instructions, which, when executed by a processor, implements the above-described online multi-target tracking method.

[0188] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to execute the online multi-target tracking method based on feature fusion and iterative correlation described in the above method embodiments. The specific implementation can be referred to the description of the online multi-target tracking method based on feature fusion and iterative correlation in the above embodiments. For the sake of brevity, it will not be repeated here.

[0189] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the aforementioned online multi-target tracking method based on feature fusion and iterative correlation. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0190] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to the above-described online multi-target tracking method based on feature fusion and iterative correlation. The electronic device may be provided as a terminal, a server, or other form of device.

[0191] This disclosure also proposes a computer program product, including a computer program / instructions that, when executed by a processor, implement the above-described online multi-target tracking method.

[0192] The experimental results of the proposed algorithm are described below, and they are compared with other state-of-the-art methods on the Crowdhuman and MOTChallenge (MOT17 and MOT20) datasets. Furthermore, sufficient ablation experiments were designed to test and validate the performance of the proposed module. As with most pioneers, HOTA and MOTA were used to evaluate the proposed algorithm, and the experimental results demonstrate its superiority over baseline methods.

[0193] Both the MOT17 and MOT20 datasets are public datasets for object tracking. The MOT17 dataset contains tracking of human objects, including seven sequences covering various scenarios and applications. The MOT20 dataset is an upgraded version of the MOT17 dataset, adding more sequences and more object categories. These datasets provide a benchmark for research on object tracking algorithms and can be used to compare and evaluate different algorithms. They also contribute to research in areas such as human behavior analysis, video analysis, and object detection. The CrowdHuman dataset is a large benchmark dataset designed specifically for object detection tasks in crowd scenes. The CrowdHuman dataset is divided into three parts: a training set, a validation set, and a test set. The training set contains 15,000 images, the validation set contains 4,370 images, and the test set contains 5,000 images. These images are all from real-world scenes, including various lighting conditions, viewpoints, and backgrounds.

[0194] The experimental platform was configured with an RTX 3060 GPU (10G). The operating system of the experimental platform was Ubuntu 18.04, the deep learning framework was PyTorch 1.9.0, the CUDA version was 11.1, and the Python version was 3.8.

[0195] The experiments employed a transfer learning strategy. The dataset was first pre-trained on the CrowedHuman dataset and a semi-MOT17 dataset, and then trained on the MOT17 dataset using the pre-trained weights. Hyperparameters were set to a batch size of 16, an epoch of 30, an initial learning rate of 0.001, and a weight decay of 0.0005.

[0196] The performance of the proposed algorithm is evaluated using High-Order Tracking Accuracy (HOTA) and CLEAR MOT metrics as standard protocols, which include Multi-Target Tracking Accuracy (MOTA), Identity F1 (IDF1) score, Main Tracking (MT), Main Loss (ML), and IDS.

[0197] HOTA is defined as the geometric mean of localization accuracy (LocA), detection accuracy (DetA), and correlation accuracy (AssA). As an advanced tracking metric, HOTA combines detection, localization, and correlation performance into a unified metric. All three components (localization, detection, and correlation) are important for tracking success, therefore HOTA is crucial for measuring overall tracking accuracy.

[0198] MOTA is the most widely used and officially recognized metric for evaluating the performance of object detection and tracking in Motion Detection (MOT). MOTA is defined as:

[0199]

[0200] Where t represents a frame, FN T FP T IDSW T and GT T These represent the number of false negatives, false positives, IDS, and true bounding boxes in frame t, respectively.

[0201] IDF1.IDF1 can be used to compute the bijection between the true and predicted trajectories and determine whether the model can robustly maintain the identity of objects throughout the sequence. IDF1 can be represented as:

[0202]

[0203] Wherein, IDTP, IDFN, and IDFP are the number of true positive IDs, false negative IDs, and false positive IDs in frame t, respectively.

[0204] Table 1 Results on the MOT17 test set

[0205]

[0206] Table 2 Results on the MOT20 test set

[0207]

[0208] Table 3 Comparison of tracking speed between the method of the present invention and several existing methods.

[0209]

[0210] For a fair comparison, the method in the bottom block uses detections generated by YOLOv-X, and on MOTA17, this invention has similar MOTA to advanced methods.

[0211] To verify the effectiveness of the method proposed in this invention, this invention was compared and analyzed with other state-of-the-art trackers on the MOT17 and MOT20 datasets. According to the tracking process, these trackers are all advanced post-detection tracking at each stage, and the detectors are consistent with bytetrack, using YOLOX-X.

[0212] As shown in Tables 1 and 2, the method of this invention outperforms other methods on the MOT17 and MOT20 datasets in most metrics. Specifically, it achieves the highest MOTA (76.4%) on the MOT17 dataset, while also exhibiting the lowest FP (5.9%) and IDSW (0.4%). This is because the method of this invention distinguishes between different categories of targets during the matching process, preventing different types of targets from interfering with each other, thus significantly reducing tracker false detections and ID switching. Notably, this invention was also tested on the more complex MOT20 dataset, which has a higher scene density and more severe clustering; on the MOT20 test set, the average number of pedestrians per image is 170. The method of this invention shows a more significant reduction in IDSW, and its accuracy (MOTA 78.5%, IDF 172.27%, HOTA 63.63%) is comparable to state-of-the-art trackers. Tracking speed is typically quantified in frames per second (FPS) and is a key indicator for evaluating tracker performance in practical applications. As shown in Table 3, the tracking speed of the algorithm of this invention is at a moderate level among all algorithms. This is clearly due to the use of deeper cascaded matching layers in the data association stage. However, considering the improvements brought by the algorithm and the requirements of multi-target tracking in practical applications, the algorithm of this invention is still acceptable. In conclusion, the observation of these results demonstrates that the algorithm of this invention exhibits competitive performance on multiple datasets compared to state-of-the-art methods.

[0213] The present invention presents ablation experiments on the proposed method to investigate the effects of different training data, model frameworks, hyperparameters, and different modules on the proposed method.

[0214] 1) Impact of Different Levels. To verify the effectiveness of the three-layer cascade of the overall algorithm framework of this invention, the following comparative experiments were conducted, as shown in Table 4. As a general method, the N=2 version of this invention is basically equivalent to OC-SORT and BYTE on the half-value benchmark, while the N=3 version is slightly higher than the N=2 version. Compared with several existing models, it has the highest MOTA (82.5%), the lowest false positives (FP) and false negatives (IDs), without the need to adjust the tracking parameters. This proves the overall versatility of the method of this invention. The comparison results of this invention are directly obtained from the Open MMLabmmtracking software library.

[0215] Table 4 compares the half-value mot17 of different partition intervals with state-of-the-art methods, assuming the same detector is used.

[0216]

[0217] 2) The impact of adaptive thresholding on the dataset. Taking the MOT dataset as an example, the baseline obtained the optimal threshold through repeated experiments. Although the overall data is good, the results of different subsets vary significantly due to differences in complexity. Figure 5 As shown in the figure, the lower the MOTA score of a subset, the higher its complexity. The method of the present invention shown in the figure has a better MOTA improvement effect on high-complexity subsets, and the difference is more obvious than the overall effect comparison. It is not weaker than existing methods on low-complexity subsets. This means that the adaptive threshold method of the present invention can perform better in dealing with various situations, especially complex environments.

[0218] 3) Ablation experiments of WMIOU on the MOT17 test set. Table 6 shows the impact of different IOU calculation methods on the tracker. In the table, N=1 indicates that WMIOU only participates in the association once, and it is in the high-threshold part. This is because in the low-threshold part of the match, there is a high probability and a high ratio of background and occluded target. The trajectory does not intersect with the detection box or completely contains it, resulting in WIOU greater than 1 or far from the expected value. This is also the reason why the application strategy of WMIOU is to improve the accuracy of the high-threshold association part to reduce the pressure on the low-threshold part. The detection shows the best performance when WMIOU is used and only participates in the high-threshold match once, with a HOTA gain of 0.5%, a MOTA gain of 0.2%, and an IDF1 gain of 0.8%. This verifies the earlier point that WMIOU strengthens the association of high matching values ​​and should not be applied to the low-complexity threshold range. Meanwhile, this invention also incorporates a set of two-stage matching methods (N=2) in which WMIOU is involved in both stages. The final results show that all tracking indicators are slightly lower than those of the method in N=1. This is because the method of this invention is not fully applicable to low-threshold environments. By adjusting the strategy through the overall framework, WMIOU matching is only used at high thresholds, which improves accuracy while reducing the matching difficulty at low thresholds. Overall, this provides another solution for low thresholds. Similarly, other more suitable methods can also be used to replace it.

[0219] 3) Tracking Performance. As shown in Table 5, this invention explores the performance of the proposed adaptive threshold and WMIOU matching modules compared to the baseline model (which includes a deep information fusion module). It can be seen that each module can improve the tracking performance of the baseline model to varying degrees. The module incorporating both WMIOU and the adaptive threshold partitioning module is optimal on most metrics. Specifically, MOTA increases by 1.2%, and IDs decrease by 0.2%. The proposed WMIOU module helps the network to be sensitive to objects with a high probability of being identified, thereby reducing interference from uncertain matches. The adaptive threshold can accurately distinguish targets with different levels of confidence. By combining these two modules, the network can further improve tracking performance, which verifies the effectiveness and superiority of the method proposed in this invention.

[0220] Table 5 Comparison with baseline model

[0221]

[0222] Table 6 compares WMIoU with GIOU, DIOU, etc., using bytetrack as a unified framework.

[0223]

[0224] In Table 6, all IOUs participate in one match (high threshold) of bytetrack, while a WMIOU participating in both stages is added. N represents the number of matches participated in in one tracking cycle. This invention yields the following results, where the WMIOU participating in only one high threshold stage exhibits the highest performance.

[0225] Figure 7 A schematic diagram illustrating the MOTA comparison on various subsets of the validation set of two datasets according to embodiments of this disclosure is provided. Figure 7 As shown in the figure, the MOTA comparison is shown on each subset of the validation set of the two datasets MOT17 (right) and MOT20 (left). The experiment only uses the adaptive threshold module to compare with the baseline bytetrack. It can be found that the lower the MOTA value of each subset of the baseline, the more complex the data, and the greater the improvement shown by the adaptive threshold of the present invention.

[0226] 4) Interaction between modules. Previous experience has proven that the baseline bytetrack used in this invention is a simple and effective model. The initial goal of the algorithm design in this invention was to solve the same problem—multi-target tracking in complex situations—by improving several interconnected modules in different tracking processes, based on bytetrack. In practice, this invention was designed according to this idea. The depth information fusion module implements the cross-application of confidence scores and strong / weak cues of depth information, providing clearer boundaries for the three-stage threshold set division. The adaptive threshold division upgrades the original two-stage to a three-stage adaptive division. The three stages are selected from the most complex and error-prone parts, and WMIOU is used for targeted matching. The core of WMIOU is precise matching using width information. Together with the depth information fusion module, it fully utilizes the confidence score, width, and height information output by the detector, strengthening the correlation between detection and tracking in TBD. As shown in Table 5, this ultimately achieves an effective improvement in tracking performance.

[0227] In the visualization experiments corresponding to the performance of the algorithm of this invention in complex scenarios, the method of this invention has excellent performance in ID retention among several important indicators affecting multi-target tracking technology, namely, missed detection (FN), false detection (FP), and ID switching (IDs).

[0228] In summary, the technical solution proposed in this invention is used for accurate tracking of pedestrian targets in complex environments. Experiments were conducted on the relevant MOT dataset, and the tracker of this invention significantly outperforms most existing methods of the same type. These results demonstrate that the method of this invention can accurately track pedestrian targets in dense environments and reduce the probability of erroneous tracking. The superior performance of the tracker of this invention is attributed to three improvements: first, enhanced correlation strength during trajectory matching; second, stable tracking efficiency under different environments and environmental changes through adaptive thresholding to enhance overall tracking performance; and third, reduction of estimation bias in matching through wmIoU. Furthermore, the tracker of this invention uses bytetrack as a baseline, and the post-processing of data after trajectory matching remains consistent with the baseline. This facilitates the demonstration of the method's effectiveness and allows for the concise and modular design of the improvements, enabling integration with more advanced methods. Given the rapid advancements in detector performance in recent years, more effective solutions can be explored in the future.

[0229] Figure 8 This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, or other terminal.

[0230] Reference Figure 8 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0231] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Power supply component 806 provides power to the various components of electronic device 800. Multimedia component 808 includes a screen that provides an output interface between said electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP).

[0232] Audio component 810 is configured to output and / or input audio signals. I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. Sensor component 814 includes one or more sensors for providing status assessments of various aspects of electronic device 800. Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as memory 804 including computer program instructions executable by processor 820 of electronic device 800 to perform the above-described methods.

[0233] Figure 9 This is a block diagram illustrating an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. (Refer to...) Figure 9 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0234] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0235] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0236] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0237] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0238] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0239] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0240] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0241] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0242] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0243] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0244] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An online multi-target tracking method based on feature fusion and iterative correlation, characterized in that, include: The confidence score is fused and updated using the location information and confidence score of possible targets within each frame of a given video sequence to obtain a fused updated confidence score. This includes: determining the corresponding target bounding box or target detection box based on the location information of possible targets within each frame; calculating the height of the target bounding box or target detection box corresponding to the possible target and the height of each frame, and determining the corresponding height difference; and fusion and optimization of the confidence score based on the height difference to obtain the fused updated confidence score. Multiple thresholds corresponding to the fusion update confidence are determined using iterative algorithms or adaptive thresholding methods; Based on the multiple thresholds, the fusion update confidence is divided into multiple target detection box intervals; Tracking possible targets within each frame of a given video sequence using the location information corresponding to the plurality of target detection box intervals includes: determining location information corresponding to a trajectory marker detection box interval for initializing the trajectory from the location information corresponding to the plurality of target detection box intervals; initializing a list of tracks being tracked using the location information corresponding to the trajectory marker detection box intervals; predicting the trajectories in the initialized list of tracks being tracked to obtain corresponding predicted location information; and tracking possible targets within each frame of the given video sequence based on the predicted location information and the location information corresponding to the plurality of target detection box intervals.

2. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 1, characterized in that, Also includes: The confidence level is updated using the fusion-optimized confidence level.

3. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 2, characterized in that, The step of calculating the target bounding box or target detection box corresponding to the possible target and the height corresponding to each frame of the image, and determining the corresponding height difference, includes: Determine the ordinate of the lowest or highest point of the target bounding box or target detection box corresponding to the possible target; Calculate the first difference between the height corresponding to each frame of the image and the vertical coordinate, and determine the corresponding height difference.

4. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 2 or 3, characterized in that, The step of fusing and optimizing the confidence score based on the height difference to obtain the fused updated confidence score includes: Obtain the information range level of the target bounding box or target detection box corresponding to the possible target; The information range level is multiplied by the height difference to obtain the height difference corresponding to the information range level. Calculate the second difference between the height differences corresponding to the information range levels; Based on the second difference, the confidence level is fused and optimized to obtain the fused updated confidence level.

5. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 4, characterized in that, The step of fusing and optimizing the confidence score based on the second difference to obtain the fused updated confidence score includes: Obtain adjustable weight parameters; The confidence level is adjusted using the adjustable weight parameters to obtain the adjusted confidence level; The adjusted confidence level plus the second difference is used to obtain the fused updated confidence level.

6. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 5, characterized in that, The step of adjusting the confidence level using the adjustable weight parameter to obtain the adjusted confidence level includes: multiplying the adjustable weight parameter by the confidence level to obtain the adjusted confidence level.

7. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 1-3, 5, and 6, characterized in that, Using the iterative algorithm, multiple thresholds corresponding to the fusion update confidence are determined, including: Establish an objective function that includes the inter-class variance corresponding to multiple thresholds to be determined. ; Iteratively calculate the average fusion update confidence score corresponding to each changing threshold among the multiple thresholds to be determined. or The average confidence level corresponding to the fusion update confidence level or The sum of squares corresponding to multiple third differences between them; During the iterative calculation, the inter-class variance objective function is... The multiple thresholds corresponding to the maximum value are determined as multiple thresholds; wherein, the sum of the multiple thresholds is configured to be 1.

8. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 4, characterized in that, Using the iterative algorithm, multiple thresholds corresponding to the fusion update confidence are determined, including: Establish an objective function that includes the inter-class variance corresponding to multiple thresholds to be determined. ; Iteratively calculate the average fusion update confidence score corresponding to each changing threshold among the multiple thresholds to be determined. or The average confidence level corresponding to the fusion update confidence level or The sum of squares corresponding to multiple third differences between them; During the iterative calculation, the inter-class variance objective function is... The multiple thresholds corresponding to the maximum value are determined as multiple thresholds; wherein, the sum of the multiple thresholds is configured to be 1.

9. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 7, characterized in that, The establishment of the objective function for inter-class variance includes multiple thresholds to be determined. ,include: Obtain the preset threshold; Establish an objective function for the inter-class variance corresponding to the preset threshold and the multiple thresholds to be determined; wherein, the preset threshold is configured as the threshold corresponding to the closest non-possible target to the background in each frame of the given video sequence; the sum of the preset threshold and the multiple thresholds is configured to be 1.

10. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 8, characterized in that, The establishment of the objective function for inter-class variance includes multiple thresholds to be determined. ,include: Obtain the preset threshold; Establish an objective function for the inter-class variance corresponding to the preset threshold and the multiple thresholds to be determined; wherein, the preset threshold is configured as the threshold corresponding to the closest non-possible target to the background in each frame of the given video sequence; the sum of the preset threshold and the multiple thresholds is configured to be 1.

11. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 7, characterized in that, The iterative calculation of the average fusion update confidence level corresponding to each changing threshold among the multiple thresholds to be determined or The average confidence level corresponding to the fusion update confidence level or The sum of squares corresponding to multiple third differences between them includes: Obtain the set number of iterations, the multiple initial change thresholds corresponding to the multiple thresholds to be determined, and the initial change average fusion update confidence. or And its corresponding first set rate of change, and the second set rate of change corresponding to the plurality of initial thresholds; Calculate the average confidence level corresponding to the fusion update confidence level. or ; Based on the multiple initial change thresholds corresponding to the multiple thresholds to be determined, and the average initial change confidence level, the confidence level is fused and updated. or The first set rate of change and the second set rate of change corresponding to the plurality of initial thresholds, and the average fusion update confidence of the change corresponding to each of the plurality of thresholds to be determined within the set number of iterations. or The average confidence level corresponding to the fusion update confidence level or The sum of squares corresponding to multiple third differences between them.

12. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 8-10, characterized in that, The iterative calculation of the average fusion update confidence level corresponding to each changing threshold among the multiple thresholds to be determined or The average confidence level corresponding to the fusion update confidence level or The sum of squares corresponding to multiple third differences between them includes: Obtain the set number of iterations, the multiple initial change thresholds corresponding to the multiple thresholds to be determined, and the initial change average fusion update confidence. or And its corresponding first set rate of change, and the second set rate of change corresponding to the plurality of initial thresholds; Calculate the average confidence level corresponding to the fusion update confidence level. or ; Based on the multiple initial change thresholds corresponding to the multiple thresholds to be determined, and the average initial change confidence level, the confidence level is fused and updated. or The first set rate of change and the second set rate of change corresponding to the plurality of initial thresholds, and the average fusion update confidence of the change corresponding to each of the plurality of thresholds to be determined within the set number of iterations. or The average confidence level corresponding to the fusion update confidence level or The sum of squares corresponding to multiple third differences between them.

13. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 1-3, 5, and 6, characterized in that, Using the adaptive thresholding method, multiple thresholds corresponding to the fusion update confidence are determined, including: The fusion update confidence scores are sorted by size to obtain the sorted fusion update confidence scores; Based on the sorted fusion update confidence and the set clustering degree, the clustering interval corresponding to the set clustering degree is determined; Using the clustering interval and the number corresponding to the multiple thresholds, the multiple thresholds corresponding to the fusion update confidence are determined.

14. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 13, characterized in that, The step of determining the multiple thresholds corresponding to the fusion update confidence by utilizing the clustering interval and the number corresponding to the multiple thresholds includes: Based on the minimum and / or maximum fusion update confidence corresponding to the sorted fusion update confidence, the number of clustering intervals and the number of the multiple thresholds, determine the multiple clustering interval coefficients corresponding to the multiple thresholds; The multiple thresholds are determined on the curve corresponding to the sorted fusion update confidence based on the multiple clustering interval coefficients; wherein, the interval range corresponding to each of the multiple clustering interval coefficients includes the clustering interval.

15. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 14, characterized in that, The step of determining multiple clustering interval coefficients corresponding to the multiple thresholds based on the minimum and / or maximum fusion update confidence corresponding to the sorted fusion update confidence, the number of clustering intervals, and the number of the multiple thresholds includes: taking the minimum and / or maximum fusion update confidence as the starting point, determining multiple clustering interval coefficients corresponding to the multiple thresholds, including the clustering intervals.

16. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 1-3, 5, 6, 8-11, 14, and 15, characterized in that, Also includes: Based on the multiple thresholds corresponding to the fusion update confidence level determined by the iterative algorithm, the multiple thresholds corresponding to the fusion update confidence level determined by the adaptive thresholding method are verified.

17. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 4, characterized in that, Also includes: Based on the multiple thresholds corresponding to the fusion update confidence level determined by the iterative algorithm, the multiple thresholds corresponding to the fusion update confidence level determined by the adaptive thresholding method are verified.

18. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 7, characterized in that, Also includes: Based on the multiple thresholds corresponding to the fusion update confidence level determined by the iterative algorithm, the multiple thresholds corresponding to the fusion update confidence level determined by the adaptive thresholding method are verified.

19. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 12, characterized in that, Also includes: Based on the multiple thresholds corresponding to the fusion update confidence level determined by the iterative algorithm, the multiple thresholds corresponding to the fusion update confidence level determined by the adaptive thresholding method are verified.

20. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 13, characterized in that, Also includes: Based on the multiple thresholds corresponding to the fusion update confidence level determined by the iterative algorithm, the multiple thresholds corresponding to the fusion update confidence level determined by the adaptive thresholding method are verified.

21. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 1-3, 5, 6, 8-11, 14, 15, 17-20, is characterized in that, Based on the multiple thresholds, the fused update confidence is divided into multiple target detection box intervals, including: The interval corresponding to the minimum fusion update confidence among the fusion update confidence and the minimum threshold among the multiple thresholds is configured as the first detection box interval; Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval.

22. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 4, characterized in that, Based on the multiple thresholds, the fused update confidence is divided into multiple target detection box intervals, including: The interval corresponding to the minimum fusion update confidence among the fusion update confidence and the minimum threshold among the multiple thresholds is configured as the first detection box interval; Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval.

23. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 7, characterized in that, Based on the multiple thresholds, the fused update confidence is divided into multiple target detection box intervals, including: The interval corresponding to the minimum fusion update confidence among the fusion update confidence and the minimum threshold among the multiple thresholds is configured as the first detection box interval; Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval.

24. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 12, characterized in that, Based on the multiple thresholds, the fused update confidence is divided into multiple target detection box intervals, including: The interval corresponding to the minimum fusion update confidence among the fusion update confidence and the minimum threshold among the multiple thresholds is configured as the first detection box interval; Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval.

25. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 13, characterized in that, Based on the multiple thresholds, the fused update confidence is divided into multiple target detection box intervals, including: The interval corresponding to the minimum fusion update confidence among the fusion update confidence and the minimum threshold among the multiple thresholds is configured as the first detection box interval; Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval.

26. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 16, characterized in that, Based on the multiple thresholds, the fused update confidence is divided into multiple target detection box intervals, including: The interval corresponding to the minimum fusion update confidence among the fusion update confidence and the minimum threshold among the multiple thresholds is configured as the first detection box interval; Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval.

27. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 1-3, 5, 6, 8-11, 14, 15, 17-20, 22-26, is characterized in that, The step of determining the position information corresponding to the trajectory marker detection box interval used for trajectory initialization from the position information corresponding to the multiple target detection box intervals includes: Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval; The location information corresponding to the trajectory marker detection frame interval used for trajectory initialization is determined from the location information corresponding to the third detection frame interval and / or any one or more detection frame intervals among the plurality of second detection frame intervals.

28. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 4, characterized in that, The step of determining the position information corresponding to the trajectory marker detection box interval used for trajectory initialization from the position information corresponding to the multiple target detection box intervals includes: Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval; The location information corresponding to the trajectory marker detection frame interval used for trajectory initialization is determined from the location information corresponding to the third detection frame interval and / or any one or more detection frame intervals among the plurality of second detection frame intervals.

29. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 7, characterized in that, The step of determining the position information corresponding to the trajectory marker detection box interval used for trajectory initialization from the position information corresponding to the multiple target detection box intervals includes: Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval; The location information corresponding to the trajectory marker detection frame interval used for trajectory initialization is determined from the location information corresponding to the third detection frame interval and / or any one or more detection frame intervals among the plurality of second detection frame intervals.

30. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 12, characterized in that, The step of determining the position information corresponding to the trajectory marker detection box interval used for trajectory initialization from the position information corresponding to the multiple target detection box intervals includes: Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval; The location information corresponding to the trajectory marker detection frame interval used for trajectory initialization is determined from the location information corresponding to the third detection frame interval and / or any one or more detection frame intervals among the plurality of second detection frame intervals.

31. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 13, characterized in that, The step of determining the position information corresponding to the trajectory marker detection box interval used for trajectory initialization from the position information corresponding to the multiple target detection box intervals includes: Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval; The location information corresponding to the trajectory marker detection frame interval used for trajectory initialization is determined from the location information corresponding to the third detection frame interval and / or any one or more detection frame intervals among the plurality of second detection frame intervals.

32. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 16, characterized in that, The step of determining the position information corresponding to the trajectory marker detection box interval used for trajectory initialization from the position information corresponding to the multiple target detection box intervals includes: Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval; The location information corresponding to the trajectory marker detection frame interval used for trajectory initialization is determined from the location information corresponding to the third detection frame interval and / or any one or more detection frame intervals among the plurality of second detection frame intervals.

33. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 21, characterized in that, The step of determining the position information corresponding to the trajectory marker detection box interval used for trajectory initialization from the position information corresponding to the multiple target detection box intervals includes: Each of the multiple thresholds is configured as a second detection box interval, corresponding to the intervals of adjacent thresholds. The interval corresponding to the maximum threshold among the multiple thresholds and the maximum fusion update confidence among the fusion update confidence is configured as the third detection box interval; The location information corresponding to the trajectory marker detection frame interval used for trajectory initialization is determined from the location information corresponding to the third detection frame interval and / or any one or more detection frame intervals among the plurality of second detection frame intervals.

34. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 1-3, 5, 6, 8-11, 14, 15, 17-20, 22-26, 28-33, is characterized in that, The tracking of possible targets within each frame of the given video sequence based on predicted location information and the location information corresponding to the multiple target detection box intervals includes: Using the predicted location information and the location information corresponding to the third detection box interval with the maximum threshold among multiple thresholds in the multiple target detection box intervals and the maximum fusion update confidence among the fusion update confidence, a corresponding first cost matrix is ​​constructed; The predicted location information in the first cost matrix is ​​matched with the location information to obtain the first matching trajectory corresponding to the first optimal matching combination, the first unmatched detection value, the predicted location information or the first unmatched trajectory corresponding to the tracked trajectory list; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Multiple second cost matrices are constructed by using the location information of the intervals corresponding to adjacent thresholds among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory. The first unmatched trajectory in the plurality of second cost matrices is matched with the location information to obtain the second matched trajectory corresponding to the second optimal matching combination, the second unmatched detection value, and the second unmatched trajectory corresponding to the first unmatched trajectory; wherein, the second unmatched detection value is configured as the location information corresponding to the second detection box interval; A third cost matrix is ​​constructed using the location information corresponding to the first detection box interval from the minimum fusion update confidence to the minimum threshold among the multiple thresholds, and the second unmatched trajectory; The second unmatched trajectory in the third cost matrix is ​​matched with the location information to obtain the third matched trajectory corresponding to the third optimal matching combination, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Based on the first matching trajectory, the second matching trajectory, and the third matching trajectory, track possible targets within each frame of the given video sequence; and / or update the trajectories in the tracked trajectory list using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

35. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 4, characterized in that, The tracking of possible targets within each frame of the given video sequence based on predicted location information and the location information corresponding to the multiple target detection box intervals includes: Using the predicted location information and the location information corresponding to the third detection box interval with the maximum threshold among multiple thresholds in the multiple target detection box intervals and the maximum fusion update confidence among the fusion update confidence, a corresponding first cost matrix is ​​constructed; The predicted location information in the first cost matrix is ​​matched with the location information to obtain the first matching trajectory corresponding to the first optimal matching combination, the first unmatched detection value, the predicted location information or the first unmatched trajectory corresponding to the tracked trajectory list; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Multiple second cost matrices are constructed by using the location information of the intervals corresponding to adjacent thresholds among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory. The first unmatched trajectory in the plurality of second cost matrices is matched with the location information to obtain the second matched trajectory corresponding to the second optimal matching combination, the second unmatched detection value, and the second unmatched trajectory corresponding to the first unmatched trajectory; wherein, the second unmatched detection value is configured as the location information corresponding to the second detection box interval; A third cost matrix is ​​constructed using the location information corresponding to the first detection box interval from the minimum fusion update confidence to the minimum threshold among the multiple thresholds, and the second unmatched trajectory; The second unmatched trajectory in the third cost matrix is ​​matched with the location information to obtain the third matched trajectory corresponding to the third optimal matching combination, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Based on the first matching trajectory, the second matching trajectory, and the third matching trajectory, track possible targets within each frame of the given video sequence; and / or update the trajectories in the tracked trajectory list using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

36. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 7, characterized in that, The tracking of possible targets within each frame of the given video sequence based on predicted location information and the location information corresponding to the multiple target detection box intervals includes: Using the predicted location information and the location information corresponding to the third detection box interval with the maximum threshold among multiple thresholds in the multiple target detection box intervals and the maximum fusion update confidence among the fusion update confidence, a corresponding first cost matrix is ​​constructed; The predicted location information in the first cost matrix is ​​matched with the location information to obtain the first matching trajectory corresponding to the first optimal matching combination, the first unmatched detection value, the predicted location information or the first unmatched trajectory corresponding to the tracked trajectory list; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Multiple second cost matrices are constructed by using the location information of the intervals corresponding to adjacent thresholds among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory. The first unmatched trajectory in the plurality of second cost matrices is matched with the location information to obtain the second matched trajectory corresponding to the second optimal matching combination, the second unmatched detection value, and the second unmatched trajectory corresponding to the first unmatched trajectory; wherein, the second unmatched detection value is configured as the location information corresponding to the second detection box interval; A third cost matrix is ​​constructed using the location information corresponding to the first detection box interval from the minimum fusion update confidence to the minimum threshold among the multiple thresholds, and the second unmatched trajectory; The second unmatched trajectory in the third cost matrix is ​​matched with the location information to obtain the third matched trajectory corresponding to the third optimal matching combination, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Based on the first matching trajectory, the second matching trajectory, and the third matching trajectory, track possible targets within each frame of the given video sequence; and / or update the trajectories in the tracked trajectory list using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

37. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 12, characterized in that, The tracking of possible targets within each frame of the given video sequence based on predicted location information and the location information corresponding to the multiple target detection box intervals includes: Using the predicted location information and the location information corresponding to the third detection box interval with the maximum threshold among multiple thresholds in the multiple target detection box intervals and the maximum fusion update confidence among the fusion update confidence, a corresponding first cost matrix is ​​constructed; The predicted location information in the first cost matrix is ​​matched with the location information to obtain the first matching trajectory corresponding to the first optimal matching combination, the first unmatched detection value, the predicted location information or the first unmatched trajectory corresponding to the tracked trajectory list; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Multiple second cost matrices are constructed by using the location information of the intervals corresponding to adjacent thresholds among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory. The first unmatched trajectory in the plurality of second cost matrices is matched with the location information to obtain the second matched trajectory corresponding to the second optimal matching combination, the second unmatched detection value, and the second unmatched trajectory corresponding to the first unmatched trajectory; wherein, the second unmatched detection value is configured as the location information corresponding to the second detection box interval; A third cost matrix is ​​constructed using the location information corresponding to the first detection box interval from the minimum fusion update confidence to the minimum threshold among the multiple thresholds, and the second unmatched trajectory; The second unmatched trajectory in the third cost matrix is ​​matched with the location information to obtain the third matched trajectory corresponding to the third optimal matching combination, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Based on the first matching trajectory, the second matching trajectory, and the third matching trajectory, track possible targets within each frame of the given video sequence; and / or update the trajectories in the tracked trajectory list using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

38. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 13, characterized in that, The tracking of possible targets within each frame of the given video sequence based on predicted location information and the location information corresponding to the multiple target detection box intervals includes: Using the predicted location information and the location information corresponding to the third detection box interval with the maximum threshold among multiple thresholds in the multiple target detection box intervals and the maximum fusion update confidence among the fusion update confidence, a corresponding first cost matrix is ​​constructed; The predicted location information in the first cost matrix is ​​matched with the location information to obtain the first matching trajectory corresponding to the first optimal matching combination, the first unmatched detection value, the predicted location information or the first unmatched trajectory corresponding to the tracked trajectory list; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Multiple second cost matrices are constructed by using the location information of the intervals corresponding to adjacent thresholds among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory. The first unmatched trajectory in the plurality of second cost matrices is matched with the location information to obtain the second matched trajectory corresponding to the second optimal matching combination, the second unmatched detection value, and the second unmatched trajectory corresponding to the first unmatched trajectory; wherein, the second unmatched detection value is configured as the location information corresponding to the second detection box interval; A third cost matrix is ​​constructed using the location information corresponding to the first detection box interval from the minimum fusion update confidence to the minimum threshold among the multiple thresholds, and the second unmatched trajectory; The second unmatched trajectory in the third cost matrix is ​​matched with the location information to obtain the third matched trajectory corresponding to the third optimal matching combination, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Based on the first matching trajectory, the second matching trajectory, and the third matching trajectory, track possible targets within each frame of the given video sequence; and / or update the trajectories in the tracked trajectory list using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

39. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 21, characterized in that, The tracking of possible targets within each frame of the given video sequence based on predicted location information and the location information corresponding to the multiple target detection box intervals includes: Using the predicted location information and the location information corresponding to the third detection box interval with the maximum threshold among multiple thresholds in the multiple target detection box intervals and the maximum fusion update confidence among the fusion update confidence, a corresponding first cost matrix is ​​constructed; The predicted location information in the first cost matrix is ​​matched with the location information to obtain the first matching trajectory corresponding to the first optimal matching combination, the first unmatched detection value, the predicted location information or the first unmatched trajectory corresponding to the tracked trajectory list; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Multiple second cost matrices are constructed by using the location information of the intervals corresponding to adjacent thresholds among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory. The first unmatched trajectory in the plurality of second cost matrices is matched with the location information to obtain the second matched trajectory corresponding to the second optimal matching combination, the second unmatched detection value, and the second unmatched trajectory corresponding to the first unmatched trajectory; wherein, the second unmatched detection value is configured as the location information corresponding to the second detection box interval; A third cost matrix is ​​constructed using the location information corresponding to the first detection box interval from the minimum fusion update confidence to the minimum threshold among the multiple thresholds, and the second unmatched trajectory; The second unmatched trajectory in the third cost matrix is ​​matched with the location information to obtain the third matched trajectory corresponding to the third optimal matching combination, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Based on the first matching trajectory, the second matching trajectory, and the third matching trajectory, track possible targets within each frame of the given video sequence; and / or update the trajectories in the tracked trajectory list using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

40. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 27, characterized in that, The tracking of possible targets within each frame of the given video sequence based on predicted location information and the location information corresponding to the multiple target detection box intervals includes: Using the predicted location information and the location information corresponding to the third detection box interval with the maximum threshold among multiple thresholds in the multiple target detection box intervals and the maximum fusion update confidence among the fusion update confidence, a corresponding first cost matrix is ​​constructed; The predicted location information in the first cost matrix is ​​matched with the location information to obtain the first matching trajectory corresponding to the first optimal matching combination, the first unmatched detection value, the predicted location information or the first unmatched trajectory corresponding to the tracked trajectory list; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Multiple second cost matrices are constructed by using the location information of the intervals corresponding to adjacent thresholds among the multiple thresholds, the first unmatched detection value, and the first unmatched trajectory. The first unmatched trajectory in the plurality of second cost matrices is matched with the location information to obtain the second matched trajectory corresponding to the second optimal matching combination, the second unmatched detection value, and the second unmatched trajectory corresponding to the first unmatched trajectory; wherein, the second unmatched detection value is configured as the location information corresponding to the second detection box interval; A third cost matrix is ​​constructed using the location information corresponding to the first detection box interval from the minimum fusion update confidence to the minimum threshold among the multiple thresholds, and the second unmatched trajectory; The second unmatched trajectory in the third cost matrix is ​​matched with the location information to obtain the third matched trajectory corresponding to the third optimal matching combination, the third unmatched detection value, and the third unmatched trajectory corresponding to the second unmatched trajectory; wherein, the first unmatched detection value is configured as the location information corresponding to the third detection box interval; Based on the first matching trajectory, the second matching trajectory, and the third matching trajectory, track possible targets within each frame of the given video sequence; and / or update the trajectories in the tracked trajectory list using the first matching trajectory, the second matching trajectory, and the third matching trajectory.

41. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 34, characterized in that, The step of matching the first unmatched trajectory in the plurality of second cost matrices with the location information includes: The coordinates of the first bounding box describing the first unmatched trajectory and the coordinates of the second bounding box describing the position information are determined in multiple second cost matrices respectively; Based on the coordinates of the first rectangle and the second rectangle, determine the corresponding intersection-union ratio and width intersection-union ratio respectively; Based on the cross-connect ratio and the width cross-connect ratio, the corresponding width modulation cross-connect ratio is determined respectively; Width-modulated cross-interchange ratio (CUI) matching is performed between the predicted location information and the location information in the second cost matrix.

42. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 35-40, characterized in that, The step of matching the first unmatched trajectory in the plurality of second cost matrices with the location information includes: The coordinates of the first bounding box describing the first unmatched trajectory and the coordinates of the second bounding box describing the position information are determined in multiple second cost matrices respectively; Based on the coordinates of the first rectangle and the second rectangle, determine the corresponding intersection-union ratio and width intersection-union ratio respectively; Based on the cross-connect ratio and the width cross-connect ratio, the corresponding width modulation cross-connect ratio is determined respectively; Width-modulated cross-interchange ratio (CUI) matching is performed between the predicted location information and the location information in the second cost matrix.

43. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 41, characterized in that, The step of determining the corresponding intersection-union ratio and width intersection-union ratio based on the coordinates of the first and second rectangular frames includes: Determine the coordinates of the first point corresponding to the coordinates of the first rectangle and the coordinates of the second point corresponding to the coordinates of the second rectangle; wherein the coordinates of the first point and the coordinates of the second point are diagonally related. Determine the maximum and minimum x-coordinates of the first x-coordinate in the coordinates of the first point and the second x-coordinate in the coordinates of the second point, respectively; The corresponding width intersection-over-union ratio is determined based on the maximum and minimum horizontal coordinates, respectively.

44. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 42, characterized in that, The step of determining the corresponding intersection-union ratio and width intersection-union ratio based on the coordinates of the first and second rectangular frames includes: Determine the coordinates of the first point corresponding to the coordinates of the first rectangle and the coordinates of the second point corresponding to the coordinates of the second rectangle; wherein the coordinates of the first point and the coordinates of the second point are diagonally related. Determine the maximum and minimum x-coordinates of the first x-coordinate in the coordinates of the first point and the second x-coordinate in the coordinates of the second point, respectively; The corresponding width intersection-over-union ratio is determined based on the maximum and minimum horizontal coordinates, respectively.

45. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 43 or 44, characterized in that, The step of determining the corresponding width intersection-over-union ratio based on the maximum and minimum abscissas respectively includes: Calculate the difference between the first abscissa corresponding to the minimum abscissa and the maximum abscissa, respectively; Calculate the difference between the second abscissa corresponding to the maximum abscissa and the minimum abscissa, respectively; Calculate the ratio of the first horizontal coordinate difference to the second horizontal coordinate difference to determine the corresponding width intersection-union ratio.

46. ​​The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 41, 43, and 44, characterized in that, Before performing width-modulated intersection-over-union (CUI) matching on the predicted location information and the location information in the second cost matrix, the following steps are included: The width modulation crossover ratio is adjusted using preset adjustment parameters to obtain the adjusted width modulation crossover ratio; The predicted location information in the second cost matrix is ​​matched with the adjusted width modulation cross-union ratio (VIN) of the location information.

47. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 42, characterized in that, Before performing width-modulated intersection-over-union (CUI) matching on the predicted location information and the location information in the second cost matrix, the following steps are included: The width modulation crossover ratio is adjusted using preset adjustment parameters to obtain the adjusted width modulation crossover ratio; The predicted location information in the second cost matrix is ​​matched with the adjusted width modulation cross-union ratio (VIN) of the location information.

48. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 45, characterized in that, Before performing width-modulated intersection-over-union (CUI) matching on the predicted location information and the location information in the second cost matrix, the following steps are included: The width modulation crossover ratio is adjusted using preset adjustment parameters to obtain the adjusted width modulation crossover ratio; The predicted location information in the second cost matrix is ​​matched with the adjusted width modulation cross-union ratio (VIN) of the location information.

49. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 46, characterized in that, Determining the preset adjustment parameters includes: Obtain the threshold values ​​of adjacent threshold values ​​among the multiple threshold values ​​corresponding to each of the multiple second detection box intervals; Calculate the difference between adjacent thresholds; determine the preset adjustment parameter.

50. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 47 or 48, characterized in that, Determining the preset adjustment parameters includes: Obtain the threshold values ​​of adjacent threshold values ​​among the multiple threshold values ​​corresponding to each of the multiple second detection box intervals; Calculate the difference between adjacent thresholds; determine the preset adjustment parameter.

51. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 46, characterized in that, Determining the preset adjustment parameters includes: Determine the order of magnitude parameters for the intersection-union ratio or width intersection-union ratio corresponding to multiple second cost matrices; The preset adjustment parameters are determined based on the difference between adjacent thresholds and the set order of magnitude parameter, respectively.

52. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 47 or 48, characterized in that, Determining the preset adjustment parameters includes: Determine the order of magnitude parameters for the intersection-union ratio or width intersection-union ratio corresponding to multiple second cost matrices; The preset adjustment parameters are determined based on the difference between adjacent thresholds and the set order of magnitude parameter, respectively.

53. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 51, characterized in that, The step of determining the preset adjustment parameter based on the difference between adjacent thresholds and the set order-of-magnitude parameter includes: Calculate the ratio of the difference between adjacent thresholds to the set order of magnitude parameter; The ratios are configured or determined as the corresponding preset adjustment parameters.

54. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 52, characterized in that, The step of determining the preset adjustment parameter based on the difference between adjacent thresholds and the set order-of-magnitude parameter includes: Calculate the ratio of the difference between adjacent thresholds to the set order of magnitude parameter; The ratios are configured or determined as the corresponding preset adjustment parameters.

55. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 1-3, 5, 6, 8-11, 14, 15, 17-20, 22-26, 28-33, 35-41, 43, 44, 47-49, 51, 53, and 54, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

56. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 4, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

57. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 7, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

58. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 12, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

59. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 13, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

60. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 16, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

61. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 21, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

62. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 27, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

63. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 34, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

64. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 42, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

65. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 45, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

66. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 46, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

67. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 50, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

68. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 52, characterized in that, Before fusing and updating the confidence score using the location information and confidence score of possible targets within each frame of a given video sequence, the method further includes: acquiring or detecting the location information and confidence score of possible targets within each frame of a given video sequence; The step of detecting the location information and confidence level of possible targets within each frame of a given video sequence includes: acquiring a target detector; and using the target detector to detect the location information and confidence level of possible targets within each frame of a given video sequence.

69. The online multi-target tracking method based on feature fusion and iterative correlation according to any one of claims 1-3, 5, 6, 8-11, 14, 15, 17-20, 22-26, 28-33, 35-41, 43, 44, 47-49, 51, 53, 54, 56-68, is characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

70. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 4, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

71. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 7, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

72. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 12, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

73. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 13, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

74. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 16, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

75. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 21, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

76. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 27, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

77. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 34, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

78. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 42, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

79. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 45, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

80. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 46, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

81. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 50, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

82. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 52, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

83. The online multi-target tracking method based on feature fusion and iterative correlation according to claim 55, characterized in that, The given video sequence is configured as a video sequence corresponding to vehicles and / or pedestrians.

84. An online multi-target tracking system based on feature fusion and iterative correlation, characterized in that, include: The fusion update unit is used to fuse and update the confidence score by utilizing the location information and confidence score of possible targets within each frame of a given video sequence to obtain a fused updated confidence score. This includes: determining the corresponding target bounding box or target detection box based on the location information of possible targets within each frame; calculating the height of the target bounding box or target detection box corresponding to the possible target and the height of each frame, and determining the corresponding height difference; and performing fusion optimization on the confidence score based on the height difference to obtain the fused updated confidence score. The threshold determination unit is used to determine multiple thresholds corresponding to the fusion update confidence using an iterative algorithm; The target detection box interval division unit is used to divide the fused update confidence into multiple target detection box intervals based on the multiple thresholds. A target tracking unit is configured to track possible targets within each frame of a given video sequence using the position information corresponding to the plurality of target detection box intervals, including: determining position information corresponding to a trajectory marker detection box interval for initializing the trajectory from the position information corresponding to the plurality of target detection box intervals; initializing a list of tracks being tracked using the position information corresponding to the trajectory marker detection box intervals; predicting the trajectories in the initialized list of tracks being tracked to obtain corresponding predicted position information; and tracking possible targets within each frame of the given video sequence based on the predicted position information and the position information corresponding to the plurality of target detection box intervals.

85. An online multi-target tracking system based on feature fusion and iterative correlation, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the online multi-target tracking method based on feature fusion and iterative correlation as described in any one of claims 1 to 83.

86. An online multi-target tracking system based on feature fusion and iterative correlation, characterized in that, include: A computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the online multi-target tracking method based on feature fusion and iterative correlation as described in any one of claims 1 to 83.

87. An online multi-target tracking system based on feature fusion and iterative correlation, characterized in that, include: A computer program product, wherein the computer program product is configured with computer programs / instructions that, when executed by a processor, implement the online multi-target tracking method based on feature fusion and iterative correlation as described in any one of claims 1 to 83.

Images