A multi-target tracking method and device based on a temporary storage matching mechanism
By employing a temporary matching mechanism and a hierarchical strategy involving multi-frame verification, the problem of directly initializing unmatched detection boxes in the TransTrack algorithm is solved, improving the accuracy and robustness of multi-target tracking and ensuring trajectory continuity and tracking accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU DIANZI UNIV
- Filing Date
- 2026-05-06
- Publication Date
- 2026-06-02
Smart Images
Figure CN122134759A_ABST
Abstract
Description
Technical Field
[0001] This invention application relates to the field of computer vision, and more particularly to a multi-target tracking method and apparatus based on a temporary matching mechanism. Background Technology
[0002] Multi-target tracking is one of the core tasks of computer vision. Its core objective is to locate multiple targets in consecutive video frames and assign a unique and persistent tracking (identity) ID to each target, thereby achieving continuous tracking of the target's trajectory. The TransTrack algorithm, based on the Transformer architecture, is one of the mainstream multi-target tracking solutions. This algorithm combines the feature extraction capabilities of Transformer with target detection technology, simultaneously outputting detection boxes and tracking boxes. It uses box matching to inherit the target ID and initialize a new trajectory, making it widely used in intelligent systems requiring real-time, high-precision target tracking.
[0003] The core of the existing TransTrack algorithm is as follows: 1. Input the current frame image, and output the detection boxes (including target position, confidence, etc.) of all targets in the current frame through the detector based on DETR (DEtectionTransformer, end-to-end target detection model); 2. The tracker combines historical tracking trajectories to predict the tracking boxes of targets in the current frame based on historical trajectories; 3. Calculate the intersection over union (IoU) ratio of the detection boxes and the tracking boxes to complete the box matching. The successfully matched detection boxes inherit the tracking ID of the corresponding tracking box, and the unmatched detection boxes are directly initialized as new tracking trajectories and assigned new IDs.
[0004] However, the TransTrack algorithm has significant drawbacks, such as:
[0005] First, the assumption is unreasonable: it is assumed that when a historical target appears in the current frame, the tracker will definitely be able to output the corresponding tracking box, but in actual scenarios, the tracker is prone to missing detection due to factors such as target occlusion, motion blur, and changes in lighting.
[0006] Second, the logic for handling unmatched detection boxes is crude: the existing solution directly initializes all unmatched detection boxes as new trajectories, which leads to the incorrect initialization of falsely detected targets, introduces invalid trajectories, and reduces tracking accuracy; the missed historical targets are repeatedly initialized, which leads to the same target being assigned multiple IDs, destroys the continuity of the trajectory, and significantly reduces core tracking metrics such as IDF1 (Identity F1Score, a key indicator used to evaluate the performance of the algorithm in maintaining the consistency of target identity) and MOTA (Multiple Object Tracking Accuracy).
[0007] Third, there is no verification mechanism for trajectory initialization: the new trajectory lacks validity verification and is initialized without a single match, which amplifies the impact of detection errors on the tracking results. Summary of the Invention
[0008] In view of this, this application provides a multi-target tracking method and apparatus based on a temporary matching mechanism to improve the accuracy and robustness of multi-target tracking under the TransTrack framework.
[0009] Specifically, this application is implemented through the following technical solution:
[0010] According to the embodiments of this specification, a multi-target tracking method based on a temporary matching mechanism is provided, including the following steps:
[0011] Step S1: Obtain detection information of all targets in the current frame and tracking information of all historical targets; the detection information includes the detection bounding boxes of the targets in the current frame, and the tracking information includes the tracking bounding boxes of the historical targets.
[0012] Step S2: Match the detection box with the tracking box to determine candidate detection boxes that do not match the tracking box.
[0013] Step S3: Match each candidate detection box with the temporary boxes in the temporary box list one by one, and update the temporary box list based on the matching results.
[0014] Step S4: Initialize the temporary frames in the updated temporary frame list that meet the new target conditions as the new target, and delete the temporary frame from the temporary frame list of the previous frame to obtain the temporary frame list of the current frame.
[0015] According to embodiments of this specification, a multi-target tracking device based on a temporary matching mechanism is also provided, comprising:
[0016] An information acquisition unit is used to acquire detection information of all targets in the current frame and tracking information of all historical targets; the detection information includes the detection bounding boxes of the targets in the current frame, and the tracking information includes the tracking bounding boxes of the historical targets.
[0017] The first matching unit is used to match the detection box with the tracking box and determine the candidate detection box that did not match the tracking box.
[0018] The second matching unit is used to match each candidate detection box with the temporary boxes in the temporary box list one by one, and update the temporary box list based on the matching results.
[0019] The target processing unit is used to initialize the temporary frames that meet the new target conditions in the updated temporary frame list as new targets, and delete the temporary frames from the temporary frame list of the previous frame to obtain the temporary frame list of the current frame.
[0020] According to embodiments of this specification, an electronic device is also provided, including: a processor; a computer-readable storage medium storing computer program instructions, which, when executed by the processor, cause the processor to perform a multi-target tracking method based on a temporary matching mechanism as described above.
[0021] According to embodiments of this specification, a computer-readable storage medium is also provided, on which a computer program is stored, the computer program being executed by a processor as described above for a multi-target tracking method based on a temporary matching mechanism.
[0022] The embodiments of this application have at least the following beneficial effects:
[0023] This application's embodiments analyze the spatiotemporal correlation features of unmatched detection boxes in continuous video frames, introduce temporary storage management and multi-frame verification mechanisms to enhance the effectiveness of unmatched detection boxes in screening, thereby more accurately distinguishing new targets, falsely detected targets, and missed historical targets, improving the accuracy and robustness of TransTrack-based multi-target tracking, bringing significant technological progress to the field of multi-target tracking, and improving the practical application value of intelligent systems. Attached Figure Description
[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:
[0025] Figure 1 This is a flowchart illustrating an exemplary embodiment of a multi-target tracking method based on a temporary matching mechanism.
[0026] Figure 2 This is an overall flowchart illustrating a multi-target tracking method in an exemplary embodiment of this application;
[0027] Figure 3 This is a schematic diagram illustrating the multi-target tracking effect based on the traditional TransTrack algorithm in a scenario according to an exemplary embodiment of this application;
[0028] Figure 4This is a schematic diagram illustrating the multi-target tracking effect based on the optimized TransTrack algorithm in a scenario according to an exemplary embodiment of this application;
[0029] Figure 5 This is a schematic diagram illustrating the multi-target tracking effect based on the traditional TransTrack algorithm in another scenario, as shown in an exemplary embodiment of this application;
[0030] Figure 6 This is a schematic diagram illustrating the multi-target tracking effect based on the optimized TransTrack algorithm in another scenario, as shown in an exemplary embodiment of this application;
[0031] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment of this application;
[0032] Figure 8 This is a block diagram of a multi-target tracking device based on a temporary matching mechanism, as illustrated in an exemplary embodiment of this application. Detailed Implementation
[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this application as detailed in the appended claims.
[0034] This invention relates specifically to deep learning-based multi-object detection and tracking technology, which is an improvement and optimization of the Multi-Object Tracking (MOT) algorithm based on the Transformer architecture. This application is mainly applied to scenarios such as video surveillance, autonomous driving, intelligent transportation, and security monitoring. It is suitable for systems that continuously, stably, and with high precision track multiple types of targets, including pedestrians and vehicles, and can significantly improve the trajectory continuity, ID switch suppression capability, and overall tracking accuracy of multi-object tracking systems.
[0035] To achieve the above objectives, this application proposes a multi-target tracking scheme based on a temporary matching mechanism. The technical solution of this application avoids the inherent logic of TransTrack's single-match initialization of a new trajectory by using a temporary box plus multi-frame verification mechanism for handling unmatched detection boxes. It filters valid new targets through multi-frame verification. Furthermore, it designs a hierarchical matching strategy: the first match between the detection box and the tracking box ensures the inheritance of historical target IDs; the second match between the unmatched detection box and the temporary box verifies the validity of new targets, distinguishing between new targets, falsely detected targets, and missed historical targets. Additionally, it introduces lifecycle management and verification thresholds for temporary boxes to adapt to different scenarios through parameter adjustment, improving the algorithm's versatility. The technical solution of this application only adds a temporary box management module to the original TransTrack architecture, without modifying the core network structure of the detector and tracker, exhibiting strong compatibility and ease of engineering implementation.
[0036] The newly added temporary storage management module is used to maintain the following data:
[0037] The temporary box list is configured to store information about unmatched detection boxes for each frame, including box coordinates, confidence score, number of temporary frames, and number of matches.
[0038] Temporary frame matching rules are used to configure the preset matching conditions between temporary frames and unmatched detection frames in subsequent frames (hereinafter referred to as candidate detection frames);
[0039] Verification thresholds are used to configure various thresholds in the preset matching conditions, such as a preset first threshold, a preset second threshold, etc.
[0040] The optimized TransTrack tracking process in this embodiment is as follows:
[0041] First, the current frame image is input, the detector of the TransTrack network outputs the detection box, and the tracker outputs the tracking box of the historical target;
[0042] Next, the IoU between the detection box and the tracking box is calculated to complete the initial matching. Successfully matched detection boxes inherit the corresponding tracking ID, and the historical trajectory of the target is updated. Unmatched detection boxes are entered into the temporary box management module, which processes the unmatched detection boxes, hereinafter referred to as candidate detection boxes.
[0043] 1) Calculate the IoU between the candidate detection boxes in the current frame and the temporary boxes in the temporary box list, and perform a second matching;
[0044] 2) The cumulative match count of a successfully matched temporary frame is incremented by 1, and the number of temporary frames is set to zero; the cumulative number of temporary frames of a failed match is incremented by 1; if a candidate detection frame does not match any of the temporary frames in the temporary frame list, then the candidate detection frame is added to the temporary frame list and the match count and number of temporary frames are initialized to zero.
[0045] 3) After all candidate detection boxes in the current frame have completed secondary matching, update the temporary box list, initialize the temporary boxes with more than 3 matching times as new trajectories, assign new IDs, and remove them from the temporary box list; delete the temporary boxes with more than 1 temporary frames directly from the temporary box list.
[0046] 4) Output the final tracking result, including the historical target with inherited ID and the newly initialized target, and proceed to the next frame for processing.
[0047] It is evident that the technical solution of this application overcomes the logical defects of the traditional TransTrack algorithm in handling unmatched detection boxes, while providing highly reliable tracking data for subsequent upper-level vision tasks such as target behavior analysis and abnormal event detection.
[0048] The embodiments described in this specification will now be described in detail.
[0049] This application provides a multi-target tracking method based on a temporary matching mechanism. Figure 1 This is a flowchart illustrating an exemplary embodiment of a multi-target tracking method based on a temporary matching mechanism, as shown in this application. Figure 1 As shown, the multi-target tracking method of this application embodiment includes at least the following steps:
[0050] Step S1: Obtain detection information of all targets in the current frame and tracking information of all historical targets; the detection information includes the detection bounding boxes of the targets in the current frame, and the tracking information includes the tracking bounding boxes of the historical targets.
[0051] This application uses the TransTrack algorithm as the target detection network. The TransTrack algorithm includes a detector and a tracker. The detector outputs detection information for all targets in the current frame, including bounding boxes, box coordinates, confidence scores, and target categories. The tracker combines historical tracking trajectories to output the tracking boxes of historical targets. The TransTrack algorithm is based on the Transformer deep learning mechanism, performing detection and tracking in parallel, and finally merging the results to achieve accurate target localization in complex scenes. Compared to traditional convolutional neural network-based trackers, the TransTrack algorithm effectively solves challenges such as target deformation, occlusion, and motion blur by capturing the spatial and temporal dependencies between the target and the background.
[0052] Step S2: Match the detection box with the tracking box to determine candidate detection boxes that do not match the tracking box.
[0053] This step calculates the IoU between the detection box and the tracking box, comparing the target categories of the two boxes to achieve inter-box matching. Successfully matched detection boxes inherit the tracking ID of the corresponding tracking box. For detection boxes that fail to match, the traditional TransTrack algorithm directly initializes them as new targets and assigns them new IDs. This approach leads to falsely detected targets being incorrectly initialized, introducing invalid trajectories and reducing tracking accuracy. Furthermore, missed historical targets are repeatedly initialized, resulting in the same target being assigned multiple IDs, disrupting trajectory continuity and significantly reducing core tracking metrics such as IDF1 and MOTA. In other words, initializing a new trajectory immediately after a single mismatch amplifies the impact of detection errors on the tracking results.
[0054] Step S3: Match each candidate detection box with the temporary boxes in the temporary box list one by one, and update the temporary box list based on the matching results.
[0055] In this embodiment, the candidate detection boxes of the current frame are cached in the candidate box set, and each candidate detection box in the candidate box set is matched again with the temporary boxes in the temporary box list. After multi-layer matching, the new target is determined. This breaks the inherent logic of the traditional TransTrack algorithm that initializes a new trajectory in a single match. By verifying and filtering through multiple frames, it is decided whether to initialize a new trajectory to ensure that a valid new target is selected.
[0056] The temporary bounding box list can be set during model initialization, such as when initializing the TransTrack tracker and loading pre-trained DETR detector weights; an empty temporary bounding box list is then created. When processing the second frame of a video sequence, all candidate detection boxes from the second frame are added to the temporary bounding box list.
[0057] Step S4: Initialize the temporary frames in the updated temporary frame list that meet the new target conditions as the new target, and delete the temporary frame from the temporary frame list of the previous frame to obtain the temporary frame list of the current frame.
[0058] In some embodiments, the temporary frame includes frame coordinate information, the number of temporary frames, and the number of matches. Accordingly, the temporary frames that meet the new target conditions in step S4 include those with a number of matches greater than a first threshold. Here, the first threshold is generally a value greater than 2, for example, the first threshold is 3.
[0059] In some embodiments, step S3 includes updating the temporary frame list through the following steps: if any temporary frame in the temporary frame list matches a candidate detection frame, the matching count of the temporary frame is incremented by 1, and its temporary frame count is set to zero; if the temporary frame in the temporary frame list does not match any of the candidate detection frames in the current frame, the temporary frame count of the temporary frame is incremented by 1; if the candidate detection frame does not match any of the temporary frames in the temporary frame list, the candidate detection frame is added to the temporary frame list, and its temporary frame count and matching count are initialized to zero.
[0060] In some embodiments, step S3 further includes: when the cumulative matching count of the temporary boxes in the temporary box list is incremented by 1, updating the box coordinate information of the temporary box to the box coordinate information of the current candidate detection box that matches it. That is, for any temporary box, when it matches the current candidate detection box, the box coordinate information of the current candidate detection box is updated to the box coordinate information of the temporary box.
[0061] In some embodiments, step S4 includes: when all candidate detection boxes in the current frame have been matched with the temporary box list, initializing the temporary boxes in the updated temporary box list that meet the new target conditions as the new target.
[0062] In some embodiments, step S4 further includes: deleting temporary frames that meet the false detection condition from the updated temporary frame list, wherein the temporary frames that meet the false detection condition include temporary frames with a temporary frame count greater than a second threshold.
[0063] In some embodiments, the temporary frame list of the current frame in step S4 is used to match with the candidate detection frames of the next frame.
[0064] Next, combined Figures 2 to 4 This application describes in detail the multi-target tracking method according to its embodiments.
[0065] To facilitate understanding of this embodiment, it is assumed that the first threshold is 3, the second threshold is 1, and the IoU matching threshold is 0.5. It is understood that in other embodiments, the above parameter values can be flexibly set.
[0066] like Figure 2 As shown, this embodiment first initializes the tracker and temporary storage management module.
[0067] Specifically, the tracker is initialized, the pre-trained detector weights are loaded, the temporary box management module is initialized, an empty temporary box list is created, and the IoU threshold is set to 0.5, the first threshold to 3, and the second threshold to 1.
[0068] Then input the image of frame t and output the detection box and tracking box.
[0069] Input the image of frame t into the detector and output a set of detection boxes Dt={d1,d2,...,dn}. Each detection box contains coordinates (x1,y1,x2,y2), confidence s, target category c, etc.; obtain the historical trajectory of the tracker combined with frame t-1 and output a set of tracking boxes Tt={t1,t2,...,tm}. Each tracking box contains trajectory information such as box coordinates and tracking ID.
[0070] Next, the initial matching of the detection box and the tracking box is performed.
[0071] Calculate the intersection-union ratio (IoU) between each detection box in the detection box set Dt and each tracking box in the tracking box set Tt. If the IoU is ≥ 0.5, the match is considered successful. The successfully matched detection box inherits the ID of the corresponding tracking box, updates the trajectory information of the ID, and removes the detection box from the detection box set Dt. Unmatched detection boxes are retained as the candidate detection box set Ut = {u1, u2, ..., uk}.
[0072] IoU is a core metric for calculating the degree of overlap between the detection bounding box and the tracking / temporary bounding box. The formula is as follows:
[0073] (1)
[0074] Where A is the rectangular region of the detection box, B is the rectangular region of the tracking box / temporary box; Area(A∩B) is the intersection area of A and B, and Area(A∪B) is the union area of A and B.
[0075] Then, a second matching process is performed between the candidate detection boxes and the temporary boxes.
[0076] Read the list of temporary boxes S{t-1}={s1,s2,...,sp}, where each temporary box contains box coordinates, matching count cnt, and number of temporary frames frame_cnt; for each candidate detection box ui in Ut, iterate through each temporary box sj in the list of temporary boxes S{t-1} and calculate the IoU:
[0077] 1) If IoU≥0.5, the two are considered to be matched successfully. The number of matches cnt in the temporary frame sj is incremented by 1 to obtain the cumulative number of matches, and the number of temporary frames frame_cnt in the temporary frame is reset to 0.
[0078] 2) If IoU < 0.5, it is determined that the two failed to match. The temporary frame sj failed to match each candidate detection frame in the candidate detection frame set Ut. Then the number of temporary frames frame_cnt of the temporary frame sj is incremented by 1.
[0079] 3) If the candidate detection box ui fails to match any of the temporary boxes in the temporary box list S{t-1}, then add the candidate detection box ui as a new temporary box to the list, initialize the number of matching times cnt=0, and the number of temporary frames frame_cnt=0.
[0080] Then update the temporary storage list.
[0081] After each candidate detection box in the candidate detection box set Ut has completed the above two-stage matching, the temporary boxes in the temporary box list S{t-1} with a matching count cnt≥3 are determined as valid targets, removed from S{t-1}, initialized as new trajectories, assigned new IDs, and added to the tracking results; the temporary boxes in the temporary box list S{t-1} with a matching count cnt<3 are retained in the temporary box list; and the temporary boxes in the temporary box list S{t-1} with a temporary frame count frame_cnt t>1 are determined as false detection targets and deleted from the temporary box list.
[0082] The cumulative matching count of the temporary storage box in this embodiment follows the following rules:
[0083] (2)
[0084] in, Let be the number of times the temporary frame in frame t is matched. This represents the number of times the temporary frame is matched in frame t-1.
[0085] The formula for accumulating temporary frames is as follows:
[0086] (3)
[0087] in, Let t be the number of frames that are temporarily stored in this temporary storage frame. This is the number of temporary frames for this temporary frame in frame t-1.
[0088] Next, proceed to the processing of frame t+1, and repeat the above steps until the video sequence processing is complete.
[0089] This application uses the training set of the QuadTrack dataset for validation. The training set contains 17 video sequences (numbered 0015 to 0031), covering multi-object tracking tasks in complex scenes, and presenting challenges such as target occlusion, motion blur, and lighting changes. These 17 video sequences are divided into training and validation sets for testing, as follows:
[0090] Training set: Sequences 0017, 0018, 0019, 0022, 0023, 0024, 0025, 0027, 0028, 0029, 0030, 0031, a total of 12 sequences;
[0091] Validation set: sequences 0015, 0016, 0020, 0021, and 0026, a total of 5 sequences.
[0092] Both the traditional TransTrack algorithm and the optimized algorithm of this application embodiment were used, keeping the core parameters of the detector and tracker of the two algorithms consistent. Tracking tests were conducted on this dataset, and the experimental results are shown in Table 1.
[0093]
[0094] Table 1
[0095] This application tested the performance of the traditional TransTrack algorithm after integrating a temporary matching mechanism on the QuadTrack dataset. The results are shown in Table 1. Table 1 shows that after integrating the temporary matching mechanism to temporarily store and verify unmatched detection boxes across multiple frames, the tracking performance of all video sequences was improved. Core metrics, including IDF1 and MOTA, demonstrate the improvement in tracking accuracy and trajectory continuity brought about by the temporary matching mechanism. After integrating the temporary matching mechanism, the IDF1 and MOTA metrics improved by 1.7%–11.6% and 2.1%–15.2%, respectively, with overall improvements of 2.8% and 4.9%, respectively. This demonstrates that the temporary matching mechanism enhances the model's ability to analyze the spatiotemporal correlation features of targets through multi-frame verification of unmatched detection boxes, enabling the selection of truly effective new targets from complex detection results and improving the overall tracking accuracy of the algorithm.
[0096] The optimized temporary matching mechanism significantly improves the continuity and robustness of the trajectory in the traditional algorithm. Specifically, the IDF1 index of sequence 0015 increased from 30.3% to 40.5%, and the MOTA index increased from 0.5% to 15.7%. This demonstrates that the temporary matching mechanism learns and understands the spatiotemporal correlation features of the target while correcting trajectory initialization errors caused by single matching, effectively suppressing ID switching and the generation of invalid trajectories.
[0097] also, Figure 3 and Figure 4 This is a schematic diagram illustrating the comparison of multi-target tracking performance in a scenario according to an exemplary embodiment of this application. Figure 5 and Figure 6 This is a schematic diagram illustrating a comparison of multi-target tracking performance in another scenario, as shown in an exemplary embodiment of this application; Reference Figure 3 and Figure 4 , Figure 3The traditional TransTrack algorithm, due to detector false detection, mistakenly identifies target 10 as a newly appeared target and assigns a new ID to target 10. However, the optimized algorithm in this application successfully maintains the continuity of target IDs; while the reference... Figure 5 and Figure 6 In the traditional TransTrack algorithm, when the target position changes, the detector will misjudge the historical target as a newly appeared target because the motion blur tracker has not found the historical target (resulting in a missed detection). This will lead to the target ID being switched. However, the optimized algorithm in this application successfully maintains the continuity of the target ID.
[0098] In summary, the embodiments of this application solve the problems of trajectory repetition and excessive invalid trajectories caused by direct initialization of unmatched detection boxes by using a hierarchical strategy based on the first matching of detection boxes and tracking boxes, the second matching of candidate detection boxes and temporary boxes, and multi-frame verification.
[0099] Furthermore, the key parameters of the embodiments of this application are all set to adjustable ranges, which can be well adapted to different application scenarios, and do not require modification of the original algorithm core network, thus having strong compatibility. The logic of the embodiments of this application is clear, forming a complete closed-loop process from initialization to frame processing and temporary frame clearing. It has been experimentally verified that it can effectively improve the tracking performance indicators of the TransTrack algorithm, reduce the number of ID switching and false detection frames, and enhance the robustness of the algorithm in complex scenarios. It has important engineering application value and promotion prospects.
[0100] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment of this application. Please refer to... Figure 7 At the hardware level, the device includes a processor 702, an internal bus 704, a network interface 706, memory 708, a hardware acceleration device 710, and non-volatile memory 712, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 702 reads the corresponding computer program from the non-volatile memory 712 into the memory 708 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0101] Figure 8 This is a block diagram illustrating a multi-target tracking device according to an exemplary embodiment of this application. The multi-target tracking device can be applied to, for example... Figure 7The electronic device shown implements the technical solution of this application. The multi-target tracking device includes: an information acquisition unit 810, a first matching unit 820, a second matching unit 830, and a target processing unit 840, wherein:
[0102] The information acquisition unit 810 is used to acquire detection information of all targets in the current frame and tracking information of all historical targets; the detection information includes the detection bounding boxes of the targets in the current frame, and the tracking information includes the tracking bounding boxes of the historical targets.
[0103] The first matching unit 820 is used to match the detection box with the tracking box and determine the candidate detection box that did not match the tracking box.
[0104] The second matching unit 830 is used to match each candidate detection box with the temporary boxes in the temporary box list one by one, and update the temporary box list based on the matching results.
[0105] The target processing unit 840 is used to initialize the temporary frames that meet the new target conditions in the updated temporary frame list as new targets, and delete the temporary frames from the temporary frame list of the previous frame to obtain the temporary frame list of the current frame.
[0106] In some embodiments, the temporary frame includes frame coordinate information, a number of temporary frames, and a number of matching attempts. The second matching unit 830 is configured to: if any temporary frame in the temporary frame list matches a candidate detection frame, increment the number of matching attempts for that temporary frame by 1 and set its number of temporary frames to zero; if any temporary frame in the temporary frame list does not match any candidate detection frame in the current frame, increment the number of temporary frames for that temporary frame by 1; if a candidate detection frame does not match any of the temporary frames in the temporary frame list, add the candidate detection frame to the temporary frame list and initialize its number of temporary frames and number of matching attempts to zero.
[0107] In some embodiments, the second matching unit 830 is configured to update the frame coordinate information of the temporary frame to the frame coordinate information of the current candidate detection frame that matches it when the cumulative matching count of the temporary frames in the temporary frame list is incremented by 1.
[0108] In some embodiments, the target processing unit 840 is configured to initialize the temporary boxes in the updated temporary box list that meet the new target conditions as new targets when all candidate detection boxes in the current frame have been matched with the temporary box list.
[0109] In some embodiments, the target processing unit 840 is configured to delete temporary frames that meet the false detection condition from the updated temporary frame list, wherein the temporary frames that meet the false detection condition include temporary frames with a temporary frame count greater than a second threshold.
[0110] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0111] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.
[0112] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.
[0113] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.
[0114] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0115] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0116] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0117] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.
[0118] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
[0119] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.
[0120] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.
[0121] The above description is only a part of the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.
Claims
1. A multi-target tracking method based on a temporary matching mechanism, characterized in that, Includes the following steps: Step S1: Obtain detection information of all targets in the current frame and tracking information of all historical targets; the detection information includes the detection bounding boxes of the targets in the current frame, and the tracking information includes the tracking bounding boxes of the historical targets. Step S2: Match the detection box with the tracking box to determine candidate detection boxes that do not match the tracking box. Step S3: Match each candidate detection box with the temporary boxes in the temporary box list one by one, and update the temporary box list based on the matching results. Step S4: Initialize the temporary frames in the updated temporary frame list that meet the new target conditions as the new target, and delete the temporary frame from the temporary frame list of the previous frame to obtain the temporary frame list of the current frame.
2. The method according to claim 1, characterized in that, The temporary frame includes frame coordinate information, number of temporary frames, and number of matches. Step S3 includes updating the temporary frame list through the following steps: If any temporary frame in the temporary frame list matches a candidate detection frame, the matching count of that temporary frame is incremented by 1, and its temporary frame count is set to zero. If a temporary frame in the temporary frame list does not match any of the candidate detection frames in the current frame, then the number of temporary frames for that temporary frame is incremented by 1. If a candidate detection box does not match any of the temporary boxes in the temporary box list, then the candidate detection box is added to the temporary box list, and its temporary frame count and matching count are initialized to zero.
3. The method according to claim 2, characterized in that, In step S4, the temporary boxes that meet the new target conditions include those with a matching count greater than the first threshold.
4. The method according to claim 2, characterized in that, Step S3 further includes: When the number of matches for a temporary box in the temporary box list is incremented by 1, the box coordinates of that temporary box are updated to the box coordinates of the current candidate detection box that matches it.
5. The method according to claim 1, characterized in that, Step S4 includes: If all candidate detection boxes in the current frame match the temporary box list, the temporary boxes in the updated temporary box list that meet the new target conditions are initialized as the new target.
6. The method according to claim 1, characterized in that, Step S4 further includes: Remove the temporary frames that meet the false detection criteria from the updated temporary frame list, where the temporary frames that meet the false detection criteria include those with a temporary frame count greater than the second threshold.
7. The method according to claim 1, characterized in that, In step S4, the temporary frame list of the current frame is used to match the candidate detection frames of the next frame.
8. A multi-target tracking device based on a temporary matching mechanism, characterized in that, include: The information acquisition unit is used to acquire the detection information of all targets in the current frame and the tracking information of all historical targets. The detection information includes the detection bounding box of the target in the current frame, and the tracking information includes the tracking bounding box of the target in the past frame; The first matching unit is used to match the detection box with the tracking box and determine the candidate detection box that did not match the tracking box. The second matching unit is used to match each candidate detection box with the temporary boxes in the temporary box list one by one, and update the temporary box list based on the matching results. The target processing unit is used to initialize the temporary frames that meet the new target conditions in the updated temporary frame list as new targets, and delete the temporary frames from the temporary frame list of the previous frame to obtain the temporary frame list of the current frame.
9. An electronic device, characterized in that, include: processor; A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor according to any one of claims 1 to 7.