Multi-target tracking method based on adaptive iterative expansion intersection-to-union ratio and related device

Abstract

The invention discloses a multi-target tracking method based on a self-adaptive iterative expansion intersection-to-union ratio and a related device, and belongs to the technical field of computer vision and graphics. The method follows the normal form of tracking after detection, firstly, independent target detection is carried out on each frame of image, and then a motion prediction module based on training during testing is used for predicting the position coordinates of a target in the next frame. And meanwhile, calculating a predicted position offset mean value, if the predicted position offset mean value is greater than a threshold value, constructing a cost matrix by adopting a self-adaptive iteration expansion intersection-union ratio method and utilizing an expansion intersection-union ratio between the predicted position and the detection frame and appearance characteristics, and solving the cost matrix by utilizing a Hungary algorithm so as to realize association of nonlinear motion targets between two adjacent frames. Experiments prove that the method has the best performance on two large-scale common nonlinear data sets, and irregular nonlinear motion of athletes in the tracking process can be effectively processed.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and graphics technology, and relates to a multi-target tracking method and related apparatus based on adaptive iterative expansion of intersection-union ratio. Background Technology

[0002] Multi-object tracking (MOT) is one of the fundamental tasks in computer vision, aiming to track multiple objects in a video and locate them in each frame. Successful MOT can support downstream research such as action detection and recognition, pose tracking, and video understanding.

[0003] In the field of multi-object tracking, the "detect-after-tracking" (TBD) paradigm has long been popular. The implementation of "detect-after-tracking" relies on powerful detectors and accurate motion predictors to predict the trajectory of objects. Currently, most methods mainly target pedestrians on roads, whose motion exhibits a linear pattern characterized by constant speed and a single direction. Based on this, Kalman filters with linear prediction characteristics have become an ideal choice for this task, and can achieve object tracking relatively effectively. However, the motion of targets in sports scenes is nonlinear, irregular, and highly variable, which makes multi-object tracking in sports scenes more challenging. Existing methods still face the following problems when tracking athletes: (1) intense competition in sports scenes can lead to severe occlusion; (2) players on the same team wear the same colored jerseys, making them look very similar; (3) athletes in sports scenes exhibit nonlinear motion patterns, which are difficult to model.

[0004] While existing "detection-after-tracking" methods offer high speed, the Kalman filter, based on linear assumptions, limits the accuracy of motion prediction. Therefore, a more efficient and accurate multi-target tracking method is urgently needed to address these issues. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-target tracking method and related device based on adaptive iterative extended intersection-union ratio, so as to solve the technical problems of poor accuracy of existing multi-target tracking methods and difficulty in meeting the application requirements of sports scenarios with poor target movement regularity.

[0006] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention provides a multi-target tracking method based on adaptive iterative expanded intersection-union ratio, comprising the following steps: The video sequence is acquired, and the detector is used to perform target detection on each frame of the video sequence to obtain the historical motion information of the target trajectory. Based on the motion predictor trained during testing, the position coordinates of the target in the next frame are predicted using the historical motion information of the target trajectory, and the trajectory prediction result is obtained. Based on the trajectory prediction results, an adaptive iterative extended intersection-union ratio (IUU) method is used to construct a cost matrix and solve the cost matrix to realize the association of moving targets between adjacent frames and complete multi-target tracking.

[0007] Furthermore, the step of acquiring the video sequence and using a detector to perform target detection on each frame of the video sequence to obtain historical motion information of the target trajectory specifically includes: Get video sequence Using a detector on video sequences Conduct testing to obtain In the frame Detection box Each detection box , representing the coordinates of the center point of the detection box. ,Width and high ;definition The set of trajectories is ,in Indicate target The trajectory Indicates that it is in the first Bounding box in the frame, This indicates the frame number of its first appearance.

[0008] Furthermore, the test-time trained motion predictor includes a linear mapping layer, a motion encoding layer, and a trajectory prediction head connected in sequence; the linear mapping layer is used to map the historical motion information of the target trajectory into temporal tokens; the motion encoding layer is used to learn the motion patterns in the temporal tokens; the trajectory prediction head includes a prediction head of a two-layer fully connected network, capable of predicting the trajectory in... Position offset in frame .

[0009] Furthermore, the step of constructing a cost matrix based on the trajectory prediction results using an adaptive iterative extended intersection-union ratio (IUCN) method, and solving the cost matrix to achieve motion target association between adjacent frames and complete multi-target tracking specifically includes: Based on the trajectory prediction results, the predicted position offset of the trajectory is obtained, and the average position offset of all trajectories is calculated as follows: ; When the average position offset When the number of pixels in the horizontal and vertical directions is greater than K, the bounding boxes of the target detection results and trajectory prediction results are iteratively enlarged to construct a cost matrix; Solve the cost matrix to establish the correlation between the target detection results and the trajectory prediction results.

[0010] Furthermore, the process of iteratively amplifying the bounding boxes of the target detection results and trajectory prediction results specifically includes: Assuming the original expansion scale is The width of the bounding box is Gao Wei Then the expanded width and height are:

[0011] In the formula, The width after expansion; For the expanded height; Let the coordinates of the top left corner of the original bounding box be... The coordinates of the lower right corner are The expanded bounding box is then:

[0012] During the tracking process, the size is gradually increased iteratively. , ;in, This is the initial expansion scale; To increase step size; This represents the number of iterations.

[0013] Furthermore, the process of constructing the cost matrix specifically includes: In the first stage, association is performed using appearance features and expanded intersection-union ratio (IUCN). The cost matrix is ​​calculated as follows:

[0014] in, It is the appearance feature cost matrix, defined as:

[0015] in, The appearance features of the trajectory; The detected appearance features; In the second stage, only the intersection-union ratio (CIU) is used for association, and the cost matrix for association is defined as:

[0016] in, It is the cost matrix.

[0017] Furthermore, the process of solving the cost matrix is ​​performed using the Hungarian algorithm.

[0018] Secondly, the present invention provides a multi-target tracking system based on adaptive iterative extended intersection-union ratio, comprising: The multi-target detection module is used to acquire video sequences, use detectors to detect targets in each frame of the video sequence, and obtain historical motion information of the target trajectory; The motion prediction module is used to predict the position coordinates of the target in the next frame based on the motion predictor trained during test time, using the historical motion information of the target trajectory, and obtain the trajectory prediction result. The trajectory association module is used to construct a cost matrix based on the trajectory prediction results using an adaptive iterative extended intersection-union ratio method, and solve the cost matrix to achieve motion target association between adjacent frames and complete multi-target tracking.

[0019] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multi-target tracking method based on the adaptive iterative extended cross-union ratio described above.

[0020] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the multi-target tracking method based on adaptive iterative extended intersection-union ratio described above.

[0021] Compared with the prior art, the present invention has the following beneficial effects: This invention discloses a multi-target tracking method and related apparatus based on adaptive iterative expanded intersection-union ratio (ECR), specifically designed for targets with nonlinear motion. First, independent target detection is performed on each frame of the image. Then, a motion predictor trained during test time is used to predict the target's position coordinates in the next frame. Simultaneously, the mean of the predicted position offset is calculated. If it exceeds a threshold, an adaptive iterative expanded ECR method is employed. A cost matrix is ​​constructed using the expanded ECR between the predicted and detected bounding boxes and appearance features, and then solved. This achieves the association between nonlinearly moving targets in adjacent frames, enabling stable and accurate target tracking.

[0022] Furthermore, by calculating the mean of the predicted position offset, the present invention estimates whether the target in the current frame is moving at high speed, thereby automatically determining whether to use the adaptive iterative extended intersection-over-union module to improve the success rate of target tracking.

[0023] Furthermore, this invention effectively addresses the issues of appearance similarity and occlusion by designing a more reasonable motion predictor and an adaptive iterative extended cross-union ratio method for high-speed moving targets, and improves the tracking accuracy for nonlinear moving targets through a two-stage association mechanism of multi-feature fusion. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the system of the present invention; Figure 3 This is a schematic diagram of a multi-target tracking method according to an embodiment of the present invention; Figure 4 This is a network architecture diagram of a motion predictor trained during testing, as described in an embodiment of the present invention. Figure 5 This paper compares the multi-target tracking performance of the embodiments of the present invention with that of existing methods in three scenarios: basketball, volleyball, and football. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0027] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0028] See Figure 1 and Figure 3 This invention proposes a multi-target tracking method based on test-time training and adaptive iterative expansion of intersection-union ratio. First, a symbol definition is given: given a video sequence... This invention uses a detector to obtain In the frame Detection box Each detection box , representing the coordinates of the center point of the detection box. ,Width and high .definition The set of trajectories is ,in Indicate target The trajectory It indicates that it is in the first Bounding box in the frame, This indicates the frame number where it first appears. In the first frame of the video, this invention directly uses... Initialize trajectory .

[0029] Specifically, the following steps are included: Step 1: Multi-object detection. First, acquire the video sequence, and use a detector to detect objects in each frame of the video sequence to obtain the historical motion information of the object trajectory; In this step, a detector is used in Detection is performed on the frame to obtain the detection result. The detection boxes are then categorized into high-scoring and low-scoring boxes based on their confidence thresholds. High-scoring boxes are prioritized for matching with trajectories. Unmatched trajectories are then associated with low-scoring boxes.

[0030] Step 2: Motion prediction based on TTT. Based on the motion predictor trained during testing, the target's position coordinates in the next frame are predicted using the historical motion information of the target trajectory, thus obtaining the trajectory prediction result; In this step, the network architecture of the motion predictor trained during test time mainly consists of three parts. First is the linear mapping layer, which takes the historical motion information of the target trajectory as input and obtains temporal tokens through linear mapping. Next is the motion coding layer, composed of TTT blocks, with its core being the TTT MLP layer, which processes the temporal tokens and learns the motion patterns within them. Finally, there is the trajectory prediction head, whose output is the predicted trajectory's T-frame inter-position offset. .

[0031] First, there's the linear mapping layer, considering a trajectory. It is The position coordinates in the frame are Then the displacement between two adjacent frames is:

[0032] make: As Location information at any given time. Let:

[0033] The sequence motion information of the target is used to predict in Frame position offset Where q is the size of the historical motion information window. This invention uses a linear mapping layer to obtain the time-series token:

[0034] in It is a weight matrix. d represents the dimension of the time-series token.

[0035] Then comes the motion coding layer, which obtains the trajectory. After obtaining the timing information, it is fed into the motion coding module, and the motion patterns of the target are explored from the dynamic historical information. For example... Figure 4 As shown, the TTT Block is the core of the motion coding layer. It is based on the Mamba backbone network, but replaces the self-attention layer with a TTT MLP layer. Since the ability of recurrent neural networks to store long text information is limited by the hidden state, the goal of TTT MLP is to design a network layer that can compress a large amount of contextual information into a single hidden state. Its core is to use self-supervised learning to compress a large amount of historical information... Compress to hidden state In the middle, the hidden state is the weight of the self-supervised learning. The update rule of the hidden state is a step of some self-supervised learning gradient descent:

[0036] in, It's the learning rate. The output token is generated by the model. Use the updated weights right Prediction:

[0037] The goal of self-supervised learning is to rebuild... The invention itself. The invention firstly... It became a loss of some information Then optimize:

[0038] Similar to a denoising autoencoder, in order to... China Reconstruction , Need to discover The relationship between dimensions. The algorithm takes the input sequence... Mapped to output sequence This can be encoded as the forward propagation process of the sequence modeling layer. Even during testing, for any input sequence, the TTT MLP layer will still train a different set of sequence weights. Therefore, it is called Test-Time Training (TTT). The training of the end-to-end network is called the outer loop, and the weights are trained in the TTT layers. This is called the inner loop. It can be obtained through low-rank projection:

[0039] in, It is a learnable matrix in the outer loop. Furthermore, it is not... All the information in the data is worth remembering, so the reconstructed labels are still obtained through low-rank projection. Instead Therefore, the loss of self-supervised learning is:

[0040] because The dimension is lower than Therefore, construct another mapping. The output rule becomes:

[0041] During the inner loop process, only Optimized. It is a hyperparameter in the inner loop loss and is optimized in the outer loop.

[0042] Finally, there's the trajectory prediction head. After the motion coding layer, this invention uses a prediction head containing two fully connected network layers to predict the trajectory. Position offset in frame Finally, smooth L1 loss is used to supervise network training:

[0043] in This represents the actual position offset.

[0044] Step 3: Trajectory Association. Based on the trajectory prediction results, an adaptive iterative extended intersection-union ratio (IUU) method is used to construct a cost matrix and solve for the cost matrix to achieve motion target association between adjacent frames, thus completing multi-target tracking.

[0045] In this step, the present invention correlates the detection results and trajectory prediction results based on an adaptive iterative extended intersection-over-union (IoU) method. By iteratively amplifying the bounding boxes of the detection and prediction results, the correlation problem of high-speed moving targets can be effectively solved. The iterative extension module is only used in the high-resolution correlation stage.

[0046] After TTT-based motion prediction, let the trajectory The predicted position offset is The average position offset of all trajectories is When the average positional offset in both the horizontal and vertical directions exceeds K pixels, the inter-frame positional offset is considered relatively large, and an adaptive iterative expansion strategy is employed. If the offset is less than a threshold, the adaptive iterative expansion strategy is not used, and the intersection-union ratio (IUU) between the detection box and the trajectory is directly calculated. Assume the original expansion scale is... The width and height of the bounding box are respectively Then the expanded width and height are:

[0047] Let the coordinates of the top left corner and the bottom right corner of the original bounding box be respectively and Then the enlarged bounding box will become: .

[0048] By gradually increasing during the tracking process Then, extended intersection-union-comparison correlation is performed. During the iterative amplification process, ,in This is the initial expansion scale; To increase step size; This represents the number of iterations.

[0049] The association process can be divided into two stages. In the first stage, this invention uses appearance features and the extended intersection-union ratio (EIoU) for association, and the cost matrix can be calculated as follows: , in It is the appearance feature cost matrix, defined as: , in These are the appearance features of the trajectory and the appearance features of the detection, respectively. In the second stage, only the intersection-union ratio (IUU) is used for association, and the association cost matrix is ​​defined as:

[0050] in This is the cost matrix, obtained by calculating the intersection-union ratio (IU) of the bounding boxes. After solving the cost matrix using the Hungarian algorithm, this invention can obtain the matching results and update the trajectory information simultaneously. Once the tracking of a frame is complete, the above process will be repeated to track the next frame.

[0051] Step 4: Testing and Evaluation. The nonlinear moving target tracking was tested on the test set, and qualitative and quantitative evaluations were performed to verify the effectiveness and robustness of the invention.

[0052] This invention conducted its main experiments on two large-scale multi-sports player tracking datasets (SportsMOT and SoccerNet-Tracking), where the target objects exhibit non-linear motion patterns. The SportsMOT dataset is a large-scale, high-quality dataset containing 240 video sequences (45 for training, 45 for validation, and 150 for testing), covering basketball, volleyball, and soccer. Athletes exhibit significant acceleration and deceleration, thus requiring the tracker to be robust to non-linear motion. The SoccerNet-Tracking dataset consists of 201 soccer match sequences, each 30 seconds long. Unlike SportsMOT, which only tracks athletes, SoccerNet-Tracking has a broader tracking scope, including various objects such as regular players, goalkeepers, referees, and the soccer ball, creating a more complex and comprehensive tracking scenario for multi-object tracking research.

[0053] This invention employs high-order metrics (HOTA, AssA, and DetA), IDF1, and the CLEAR metric (MOTA) for evaluation. Among these diverse metrics, HOTA serves as a key indicator, balancing the impact of achieving accurate detection with precise correlation. IDF1 and AssA focus on measuring the performance of the correlation process, while DetA and MOTA are primarily used to evaluate the effectiveness of the detection process.

[0054] Regarding parameter settings: In the TTT-based motion predictor, the historical motion information q is set to 5 frames, and the input token dimension d is 256. For optimization, this method uses the Adam optimizer trained for 1000 epochs with a learning rate of 10. -4 The batch size was 4096. All experiments were trained on four GeForce RTX 3090 GPUs. During the association phase, this method performs separate matching for high-scoring and low-scoring bounding boxes, with thresholds of [insert thresholds here]. and For high-resolution detection association, the predicted average position offset K is set to 2 pixels. The initial value for the expansion ratio... Set the step size to 0.4, the step size λ to 0.1, and the total number of iterations to 2. The maximum number of frames to retain the lost trajectory. The value is 90. Furthermore, this method employs the same dynamic appearance and adaptive weighting techniques as BOT-SORT. After tracking is complete, linear interpolation is applied following the online tracking algorithm to improve the final tracking performance. All tracking results are evaluated on the official evaluation server of the dataset.

[0055] Tables 1 and 2 present the test results of the proposed method and the comparison method on the SportsMOT dataset and the SoccerNet-Tracking dataset, respectively. The proposed method achieves leading performance in most scenarios. When trained using both training and validation sets, TTT-MOT also achieves the best performance across all metrics, with HOTA of 78.8, IDF1 of 80.7, AssA of 69.7, MOTA of 97.1, DetA of 89.3, IDs of 2104, and Frag of 2782. Compared to the previous state-of-the-art tracker Deep-EIoU, HOTA is improved by 1.6, IDF1 by 0.9, AssA by 2.0, MOTA by 0.8, and DetA by 1.1. These results further demonstrate that TTT-MOT exhibits strong robustness in scenarios with diverse target motion.

[0056] Table 1. Experimental results of this method on the SportsMOT dataset.

[0057] Table 2. Experimental results of this method on the SoccerNet-Tracking dataset.

[0058] Figure 5 The paper further presents a comparison of the tracking results between a multi-target tracking algorithm based on Kalman filtering and the method proposed in this invention. From... Figure 5 As can be seen, Kalman filter-based multi-target tracking algorithms face significant challenges when tracking targets with nonlinear motion. In multiple scenarios, Kalman filter-based multi-target tracking algorithms encounter ID transformation problems when tracking targets. In stark contrast, the method proposed in this invention successfully overcomes these difficulties. Whether the target is occluded or re-enters the field of view, the proposed method exhibits excellent tracking performance, maintaining relatively stable tracking results even when targets have similar appearances or are occluded.

[0059] See Figure 2This invention discloses a multi-target tracking system based on adaptive iterative extended intersection-union ratio (IUOR), comprising a multi-target detection module, a motion prediction module, and a trajectory association module. The multi-target detection module acquires a video sequence and uses a detector to detect targets in each frame of the video sequence, obtaining historical motion information of the target trajectory. The motion prediction module, based on a motion predictor trained during testing, uses the historical motion information of the target trajectory to predict the target's position coordinates in the next frame, obtaining a trajectory prediction result. The trajectory association module, based on the trajectory prediction result, uses an adaptive iterative extended IUOR method to construct a cost matrix and solve for the cost matrix, achieving motion target association between adjacent frames and completing multi-target tracking.

[0060] In one embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a multi-target tracking method based on adaptive iterative extended intersection-union ratio.

[0061] This invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the multi-target tracking method based on adaptive iterative extended intersection-union ratio in the above embodiments.

[0062] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0064] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0065] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A multi-target tracking method based on adaptive iterative extended intersection-union ratio, characterized in that, Includes the following steps: The video sequence is acquired, and the detector is used to perform target detection on each frame of the video sequence to obtain the historical motion information of the target trajectory. Based on the motion predictor trained during testing, the position coordinates of the target in the next frame are predicted using the historical motion information of the target trajectory, and the trajectory prediction result is obtained. Based on the trajectory prediction results, an adaptive iterative extended intersection-union ratio (IUU) method is used to construct a cost matrix and solve the cost matrix to realize the association of moving targets between adjacent frames and complete multi-target tracking.

2. The multi-target tracking method based on adaptive iterative extended intersection-union ratio according to claim 1, characterized in that, The step of acquiring a video sequence and using a detector to perform target detection on each frame of the video sequence to obtain historical motion information of the target trajectory specifically includes: Get video sequence Using a detector on video sequences Conduct testing to obtain In the frame Detection box Each detection box , representing the coordinates of the center point of the detection box. ,Width and high ;definition The set of trajectories is ,in Indicate target The trajectory Indicates that it is in the first Bounding box in the frame, This indicates the frame number of its first appearance.

3. The multi-target tracking method based on adaptive iterative extended intersection-union ratio according to claim 1, characterized in that, The test-time training-based motion predictor comprises a linear mapping layer, a motion encoding layer, and a trajectory prediction head connected in sequence. The linear mapping layer maps historical motion information of the target trajectory to temporal tokens. The motion encoding layer learns motion patterns from the temporal tokens. The trajectory prediction head comprises a two-layer fully connected network capable of predicting the trajectory in... Position offset in frame .

4. The multi-target tracking method based on adaptive iterative extended intersection-union ratio according to claim 1, characterized in that, The steps of constructing a cost matrix based on trajectory prediction results using an adaptive iterative extended intersection-union ratio (IUCN) method, solving the cost matrix, and achieving motion target association between adjacent frames to complete multi-target tracking specifically include: Based on the trajectory prediction results, the predicted position offset of the trajectory is obtained, and the average position offset of all trajectories is calculated as follows: ; When the average position offset When the number of pixels in the horizontal and vertical directions is greater than K, the bounding boxes of the target detection results and trajectory prediction results are iteratively enlarged to construct a cost matrix; Solve the cost matrix to establish the correlation between the target detection results and the trajectory prediction results.

5. A multi-target tracking method based on adaptive iterative extended intersection-union ratio according to claim 4, characterized in that, The process of iteratively amplifying the bounding boxes of the target detection results and trajectory prediction results specifically includes: Assuming the original expansion scale is The width of the bounding box is Gao Wei Then the expanded width and height are: In the formula, The width after expansion; For the expanded height; Let the coordinates of the top left corner of the original bounding box be... The coordinates of the lower right corner are The expanded bounding box is then: During the tracking process, the size is gradually increased iteratively. , ;in, This is the initial expansion scale; To increase step size; This represents the number of iterations.

6. A multi-target tracking method based on adaptive iterative extended intersection-union ratio according to claim 4, characterized in that, The process of constructing the cost matrix specifically includes: In the first stage, association is performed using appearance features and expanded intersection-union ratio (IUCN). The cost matrix is ​​calculated as follows: in, It is the appearance feature cost matrix, defined as: in, The appearance features of the trajectory; The detected appearance features; In the second stage, only the intersection-union ratio (CIU) is used for association, and the cost matrix for association is defined as: in, It is the cost matrix.

7. A multi-target tracking method based on adaptive iterative extended intersection-union ratio according to claim 4, characterized in that, The process of solving the cost matrix is ​​performed using the Hungarian algorithm.

8. A multi-target tracking system based on adaptive iterative extended intersection-union ratio, characterized in that, include: The multi-target detection module is used to acquire video sequences, use detectors to detect targets in each frame of the video sequence, and obtain historical motion information of the target trajectory; The motion prediction module is used to predict the position coordinates of the target in the next frame based on the motion predictor trained during test time, using the historical motion information of the target trajectory, and obtain the trajectory prediction result. The trajectory association module is used to construct a cost matrix based on the trajectory prediction results using an adaptive iterative extended intersection-union ratio method, and solve the cost matrix to achieve motion target association between adjacent frames and complete multi-target tracking.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the multi-target tracking method based on adaptive iterative extended cross-union ratio as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-target tracking method based on adaptive iterative extended cross-union ratio as described in any one of claims 1-7.

Images