Target tracking method and device based on Byte Track

Abstract

The invention relates to the technical field of target tracking, and discloses a ByteTrack-based target tracking method and device, and the method comprises the steps: determining all high-score bounding boxes and all low-score bounding boxes from all bounding boxes for a to-be-tracked current video frame according to the scores of all bounding boxes in the current video frame; determining all first tracking tracks from all tracking tracks in previous video frames according to the previous video frames corresponding to the current video frame, and performing multiple target tracking operations on the current video frame based on all the first tracking tracks, all the high-resolution bounding boxes and all the low-resolution bounding boxes to obtain a basic target tracking result of the current video frame; and performing trajectory overlap analysis on the basic target tracking result to obtain a final target tracking result. Visibly, by implementing the method and the device, the situation of target loss caused by shielding or unstable target detection can be reduced, so that the target tracking accuracy of the current video frame can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and in particular to a target tracking method and device based on Byte Track. Background Art

[0002] In the field of computer vision, Multi-Object Tracking (MOT) is a crucial task and is widely used in multiple scenarios such as video surveillance, autonomous driving, and human-computer interaction.

[0003] Traditional multi-target tracking methods mainly rely on manually designed features (such as color, texture, shape, etc.) and simple motion models. These methods often have difficulty maintaining stable tracking performance in complex or dynamically changing environments. In particular, when there is occlusion, intersection, rapid movement or appearance changes between targets, traditional methods are prone to problems such as target loss and identity switch, which can easily lead to target tracking errors. It can be seen that it is particularly important to provide a method that can improve the accuracy of target tracking. Summary of the invention

[0004] The present invention provides a target tracking method and device based on Byte Track, which can reduce the occurrence of target loss due to occlusion or unstable target detection, thereby improving the target tracking accuracy of the current video frame.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a target tracking method based on Byte Track, the method comprising:

[0006] For a current video frame to be tracked, according to the scores of all bounding boxes in the current video frame, all high-score bounding boxes and all low-score bounding boxes are determined from all the bounding boxes; all the high-score bounding boxes are bounding boxes whose scores are greater than or equal to a preset score threshold, and all the low-score bounding boxes are bounding boxes whose scores are less than the preset score threshold;

[0007] According to the previous video frame corresponding to the current video frame, all first tracking trajectories are determined from all tracking trajectories in the previous video frame, and based on all the first tracking trajectories, all the high-score bounding boxes and all the low-score bounding boxes, multiple target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame; all the tracking trajectories at least include active tracking trajectories that have been successfully tracked for two frames or more in the previous video frames, and all the first tracking trajectories include trajectories that have been successfully tracked in a previous video frame corresponding to the current video frame and trajectories that have lost tracking in the previous n video frames corresponding to the current video frame, where n is a preset positive integer;

[0008] According to the basic target tracking result of the current video frame, trajectory overlap analysis is performed on the basic target tracking result to obtain a final target tracking result.

[0009] As an optional implementation, in the first aspect of the present invention, performing multiple target tracking operations on the current video frame based on all the first tracking trajectories, all the high-score bounding boxes, and all the low-score bounding boxes to obtain a basic target tracking result of the current video frame includes:

[0010] Performing a bounding box prediction operation on a next video frame corresponding to each of the first tracking trajectories to obtain bounding box prediction information corresponding to each of the first tracking trajectories; the bounding box prediction information corresponding to each of the first tracking trajectories includes bounding box position information and / or bounding box size information of a predicted bounding box in the next video frame corresponding to the first tracking trajectory;

[0011] Calculating a first intersection-and-union ratio between a predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes according to the bounding box prediction information corresponding to all of the first tracking trajectories and the bounding box information of all of the high-score bounding boxes;

[0012] Determining a first matching degree value between the predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes according to a first intersection-and-union ratio between the predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes;

[0013] According to the first matching degree value between the predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes, a first target tracking trajectory whose first matching degree value is greater than or equal to a preset first matching degree threshold and its corresponding first high-score bounding box are determined from all the first tracking trajectories and all the high-score bounding boxes; the matching relationship between the first target tracking trajectory and its corresponding first high-score bounding box is used to indicate the target tracking result corresponding to the first target tracking operation of the current video frame;

[0014] According to the first target tracking track and the first high-score bounding box corresponding to it, all second tracking tracks are determined from all the first tracking tracks, and all second high-score bounding boxes are determined from all the high-score bounding boxes, and based on all the second tracking tracks and all the second high-score bounding boxes, second, third and fourth target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame; all the second tracking tracks are tracking tracks whose first matching degree value is greater than or equal to the first matching degree threshold, and all the second high-score bounding boxes are high-score bounding boxes whose first matching degree value is greater than or equal to the first matching degree threshold.

[0015] As an optional implementation, in the first aspect of the present invention, performing a second, third, and fourth target tracking operation on the current video frame based on all the second tracking trajectories and all the second high-score bounding boxes to obtain a basic target tracking result of the current video frame includes:

[0016] Calculating a second intersection-and-union ratio between each predicted bounding box corresponding to the second tracking trajectory and each low-score bounding box according to the bounding box prediction information corresponding to all the second tracking trajectories and the bounding box information of all the low-score bounding boxes;

[0017] Determining a second matching degree value between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box according to a second intersection-and-union ratio between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box;

[0018] According to the second matching degree value between the predicted bounding box corresponding to each second tracking trajectory and each of the low-score bounding boxes, a second target tracking trajectory whose second matching degree value is greater than or equal to a preset second matching degree threshold and its corresponding target low-score bounding box are determined from all the second tracking trajectories and all the low-score bounding boxes; the matching relationship between the second target tracking trajectory and its corresponding target low-score bounding box is used to update the target tracking result corresponding to the first target tracking operation of the current video frame, as the target tracking result corresponding to the second target tracking operation of the current video frame;

[0019] According to the second target tracking trajectory, all third tracking trajectories are determined from all the second tracking trajectories, and based on all the second high-score bounding boxes and all the third tracking trajectories, third and fourth target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame; all the third tracking trajectories are tracking trajectories whose second matching degree value is greater than or equal to the second matching degree threshold.

[0020] As an optional implementation, in the first aspect of the present invention, all of the tracking trajectories further include inactive tracking trajectories that were not successfully tracked in the previous video frames;

[0021] The performing of the third and fourth target tracking operations on the current video frame based on all the second high-resolution bounding boxes and all the third tracking trajectories to obtain a basic target tracking result of the current video frame includes:

[0022] Performing a bounding box prediction operation on a next video frame corresponding to the inactive tracking track to obtain bounding box prediction information corresponding to the inactive tracking track; the bounding box prediction information corresponding to the inactive tracking track includes bounding box position information and / or bounding box size information of a predicted bounding box in the next video frame corresponding to the inactive tracking track;

[0023] Calculating a third intersection-and-union ratio between the predicted bounding box corresponding to the inactive state tracking trajectory and each of the second high-score bounding boxes according to the bounding box prediction information corresponding to the inactive state tracking trajectory and the bounding box information of all the second high-score bounding boxes;

[0024] Determining a third matching degree value between the predicted bounding box corresponding to the inactive state tracking trajectory and each of the second high-score bounding boxes according to a third intersection-and-union ratio between the predicted bounding box corresponding to the inactive state tracking trajectory and each of the second high-score bounding boxes;

[0025] According to the third matching degree value between the predicted bounding box corresponding to the inactivated tracking trajectory and each of the second high-score bounding boxes, a third target tracking trajectory whose third matching degree value is greater than or equal to a preset third matching degree threshold and its corresponding third high-score bounding box are determined from the inactivated tracking trajectory and all the second high-score bounding boxes; the matching relationship between the third target tracking trajectory and its corresponding third high-score bounding box is used to update the target tracking result corresponding to the second target tracking operation of the current video frame, as the target tracking result corresponding to the third target tracking operation of the current video frame;

[0026] According to the third target tracking trajectory and its corresponding third high-score bounding box, all fourth high-score bounding boxes are determined from all the high-score bounding boxes, and based on all the fourth high-score bounding boxes and all the third tracking trajectories, a fourth target tracking operation is performed on the current video frame to obtain a basic target tracking result of the current video frame; all the fourth high-score bounding boxes are high-score bounding boxes whose third matching degree values ​​are greater than or equal to the third matching degree threshold.

[0027] As an optional implementation, in the first aspect of the present invention, performing a fourth target tracking operation on the current video frame based on all the fourth high-score bounding boxes and all the third tracking trajectories to obtain a basic target tracking result of the current video frame includes:

[0028] Obtaining a screenshot of a rectangular frame of the target outside the third tracking track corresponding to each of the third tracking tracks, and determining first feature information corresponding to each of the third tracking tracks according to the screenshot of the rectangular frame of the target outside the third tracking track corresponding to each of the third tracking tracks;

[0029] Extracting the second feature information corresponding to each of the fourth high-score bounding boxes, and determining all the fourth tracking tracks and their corresponding fifth high-score bounding boxes from all the third tracking tracks and all the fourth high-score bounding boxes according to the first feature information corresponding to all the third tracking tracks and the second feature information corresponding to all the fourth high-score bounding boxes;

[0030] According to all the fifth high-scoring bounding boxes, determining all the sixth high-scoring bounding boxes from all the fourth high-scoring bounding boxes, and establishing new tracking tracks for all the sixth high-scoring bounding boxes;

[0031] According to the matching relationship between all the fourth tracking trajectories and their corresponding fifth high-score bounding boxes, and the matching relationship between all the sixth high-score bounding boxes and their corresponding new tracking trajectories, the target tracking result corresponding to the third target tracking operation of the current video frame is updated as the basic target tracking result corresponding to the fourth target tracking operation of the current video frame.

[0032] As an optional implementation, in the first aspect of the present invention, the determining, from all the third tracking trajectories and all the fourth high-score bounding boxes, all the fourth tracking trajectories and their corresponding fifth high-score bounding boxes according to the first feature information corresponding to all the third tracking trajectories and the second feature information corresponding to all the fourth high-score bounding boxes, comprises:

[0033] Calculating a Euclidean distance parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes according to the first feature information corresponding to all of the third tracking trajectories and the second feature information corresponding to all of the fourth high-score bounding boxes;

[0034] Calculating a cosine similarity parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes according to the first feature information corresponding to all of the third tracking trajectories and the second feature information corresponding to all of the fourth high-score bounding boxes;

[0035] According to the Euclidean distance parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes, and the cosine similarity parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes, all the fourth tracking trajectories whose Euclidean distance parameter is less than a preset Euclidean distance parameter threshold and whose cosine similarity parameter is less than a preset cosine similarity parameter threshold and their corresponding fifth high-score bounding boxes are determined from all the third tracking trajectories and all the fourth high-score bounding boxes.

[0036] As an optional implementation, in the first aspect of the present invention, the basic target tracking result of the current video frame includes a matching relationship between each basic tracking track and its corresponding basic bounding box, and the matching relationship between all the basic tracking tracks and their corresponding basic bounding boxes includes a matching relationship between the first target tracking track and its corresponding first high-score bounding box, a matching relationship between the second target tracking track and its corresponding target low-score bounding box, a matching relationship between the third target tracking track and its corresponding third high-score bounding box, a matching relationship between all the fourth tracking tracks and their corresponding fifth high-score bounding boxes, and a matching relationship between all the sixth high-score bounding boxes and their corresponding new tracking tracks;

[0037] The step of performing trajectory overlap analysis on the basic target tracking result according to the basic target tracking result of the current video frame to obtain the final target tracking result includes:

[0038] According to the matching relationship between all the basic tracking tracks and their corresponding basic bounding boxes, all overlapping tracking tracks whose corresponding target external rectangular box screenshots have target overlap are determined from all the basic tracking tracks, and non-overlapping external rectangular box screenshots corresponding to each overlapping tracking track are obtained in the previous video frame;

[0039] According to the target external rectangular frame screenshots and the corresponding non-overlapping external rectangular frame screenshots corresponding to all the overlapping tracking trajectories, calculating the similarity between the target external rectangular frame screenshot and the corresponding non-overlapping external rectangular frame screenshot corresponding to each overlapping tracking trajectory;

[0040] According to the similarity between the target external rectangular frame screenshots corresponding to all the overlapping tracking trajectories and the corresponding non-overlapping external rectangular frame screenshots, determining the tracking trajectory to be adjusted whose similarity is less than or equal to a preset similarity threshold from all the overlapping tracking trajectories, and adjusting the basic bounding box corresponding to the tracking trajectory to be adjusted to obtain the adjusted basic bounding box corresponding to the tracking trajectory to be adjusted;

[0041] Based on the adjusted basic bounding box corresponding to the tracking trajectory to be adjusted, the basic target tracking result is updated to obtain a final target tracking result.

[0042] The second aspect of the present invention discloses a target tracking device based on Byte Track, the device comprising:

[0043] A determination module, configured to determine, for a current video frame to be tracked, all high-score bounding boxes and all low-score bounding boxes from all the bounding boxes according to the scores of all the bounding boxes in the current video frame; all the high-score bounding boxes are bounding boxes whose scores are greater than or equal to a preset score threshold, and all the low-score bounding boxes are bounding boxes whose scores are less than the preset score threshold; and determine all first tracking tracks from all tracking tracks in the previous video frames according to the previous video frames corresponding to the current video frame;

[0044] A target tracking module is used to perform multiple target tracking operations on the current video frame based on all the first tracking trajectories, all the high-score bounding boxes, and all the low-score bounding boxes to obtain a basic target tracking result of the current video frame; all the tracking trajectories at least include active tracking trajectories that have been successfully tracked for two or more frames in the previous video frames, and all the first tracking trajectories include trajectories that have been successfully tracked in a previous video frame corresponding to the current video frame and trajectories that have lost tracking in the previous n video frames corresponding to the current video frame, where n is a preset positive integer;

[0045] The analysis module is used to perform trajectory overlap analysis on the basic target tracking result according to the basic target tracking result of the current video frame to obtain a final target tracking result.

[0046] As an optional implementation, in the second aspect of the present invention, the target tracking module performs multiple target tracking operations on the current video frame based on all the first tracking trajectories, all the high-score bounding boxes, and all the low-score bounding boxes, and a method for obtaining a basic target tracking result of the current video frame specifically includes:

[0047] Performing a bounding box prediction operation on a next video frame corresponding to each of the first tracking trajectories to obtain bounding box prediction information corresponding to each of the first tracking trajectories; the bounding box prediction information corresponding to each of the first tracking trajectories includes bounding box position information and / or bounding box size information of a predicted bounding box in the next video frame corresponding to the first tracking trajectory;

[0048] Calculating a first intersection-and-union ratio between a predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes according to the bounding box prediction information corresponding to all of the first tracking trajectories and the bounding box information of all of the high-score bounding boxes;

[0049] Determining a first matching degree value between the predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes according to a first intersection-and-union ratio between the predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes;

[0050] According to the first matching degree value between the predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes, a first target tracking trajectory whose first matching degree value is greater than or equal to a preset first matching degree threshold and its corresponding first high-score bounding box are determined from all the first tracking trajectories and all the high-score bounding boxes; the matching relationship between the first target tracking trajectory and its corresponding first high-score bounding box is used to indicate the target tracking result corresponding to the first target tracking operation of the current video frame;

[0051] According to the first target tracking track and the first high-score bounding box corresponding to it, all second tracking tracks are determined from all the first tracking tracks, and all second high-score bounding boxes are determined from all the high-score bounding boxes, and based on all the second tracking tracks and all the second high-score bounding boxes, second, third and fourth target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame; all the second tracking tracks are tracking tracks whose first matching degree value is greater than or equal to the first matching degree threshold, and all the second high-score bounding boxes are high-score bounding boxes whose first matching degree value is greater than or equal to the first matching degree threshold.

[0052] As an optional implementation, in the second aspect of the present invention, the target tracking module performs a second, third, and fourth target tracking operation on the current video frame based on all the second tracking trajectories and all the second high-score bounding boxes, and a method for obtaining a basic target tracking result of the current video frame specifically includes:

[0053] Calculating a second intersection-and-union ratio between each predicted bounding box corresponding to the second tracking trajectory and each low-score bounding box according to the bounding box prediction information corresponding to all the second tracking trajectories and the bounding box information of all the low-score bounding boxes;

[0054] Determining a second matching degree value between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box according to a second intersection-and-union ratio between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box;

[0055] According to the second matching degree value between the predicted bounding box corresponding to each second tracking trajectory and each of the low-score bounding boxes, a second target tracking trajectory whose second matching degree value is greater than or equal to a preset second matching degree threshold and its corresponding target low-score bounding box are determined from all the second tracking trajectories and all the low-score bounding boxes; the matching relationship between the second target tracking trajectory and its corresponding target low-score bounding box is used to update the target tracking result corresponding to the first target tracking operation of the current video frame, as the target tracking result corresponding to the second target tracking operation of the current video frame;

[0056] According to the second target tracking trajectory, all third tracking trajectories are determined from all the second tracking trajectories, and based on all the second high-score bounding boxes and all the third tracking trajectories, third and fourth target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame; all the third tracking trajectories are tracking trajectories whose second matching degree value is greater than or equal to the second matching degree threshold.

[0057] As an optional implementation, in the second aspect of the present invention, all of the tracking trajectories also include inactive tracking trajectories that have not been successfully tracked in the previous video frames;

[0058] The target tracking module performs the third and fourth target tracking operations on the current video frame based on all the second high-score bounding boxes and all the third tracking trajectories to obtain the basic target tracking result of the current video frame, specifically including:

[0059] Performing a bounding box prediction operation on a next video frame corresponding to the inactive tracking track to obtain bounding box prediction information corresponding to the inactive tracking track; the bounding box prediction information corresponding to the inactive tracking track includes bounding box position information and / or bounding box size information of a predicted bounding box in the next video frame corresponding to the inactive tracking track;

[0060] Calculating a third intersection-and-union ratio between the predicted bounding box corresponding to the inactive state tracking trajectory and each of the second high-score bounding boxes according to the bounding box prediction information corresponding to the inactive state tracking trajectory and the bounding box information of all the second high-score bounding boxes;

[0061] Determining a third matching degree value between the predicted bounding box corresponding to the inactive state tracking trajectory and each of the second high-score bounding boxes according to a third intersection-and-union ratio between the predicted bounding box corresponding to the inactive state tracking trajectory and each of the second high-score bounding boxes;

[0062] According to the third matching degree value between the predicted bounding box corresponding to the inactivated tracking trajectory and each of the second high-score bounding boxes, a third target tracking trajectory whose third matching degree value is greater than or equal to a preset third matching degree threshold and its corresponding third high-score bounding box are determined from the inactivated tracking trajectory and all the second high-score bounding boxes; the matching relationship between the third target tracking trajectory and its corresponding third high-score bounding box is used to update the target tracking result corresponding to the second target tracking operation of the current video frame, as the target tracking result corresponding to the third target tracking operation of the current video frame;

[0063] According to the third target tracking trajectory and its corresponding third high-score bounding box, all fourth high-score bounding boxes are determined from all the high-score bounding boxes, and based on all the fourth high-score bounding boxes and all the third tracking trajectories, a fourth target tracking operation is performed on the current video frame to obtain a basic target tracking result of the current video frame; all the fourth high-score bounding boxes are high-score bounding boxes whose third matching degree values ​​are greater than or equal to the third matching degree threshold.

[0064] As an optional implementation, in the second aspect of the present invention, the target tracking module performs a fourth target tracking operation on the current video frame based on all the fourth high-score bounding boxes and all the third tracking trajectories, and a method for obtaining a basic target tracking result of the current video frame specifically includes:

[0065] Obtaining a screenshot of a rectangular frame of the target outside the third tracking track corresponding to each of the third tracking tracks, and determining first feature information corresponding to each of the third tracking tracks according to the screenshot of the rectangular frame of the target outside the third tracking track corresponding to each of the third tracking tracks;

[0066] Extracting the second feature information corresponding to each of the fourth high-score bounding boxes, and determining all the fourth tracking tracks and their corresponding fifth high-score bounding boxes from all the third tracking tracks and all the fourth high-score bounding boxes according to the first feature information corresponding to all the third tracking tracks and the second feature information corresponding to all the fourth high-score bounding boxes;

[0067] According to all the fifth high-scoring bounding boxes, determining all the sixth high-scoring bounding boxes from all the fourth high-scoring bounding boxes, and establishing new tracking tracks for all the sixth high-scoring bounding boxes;

[0068] According to the matching relationship between all the fourth tracking trajectories and their corresponding fifth high-score bounding boxes, and the matching relationship between all the sixth high-score bounding boxes and their corresponding new tracking trajectories, the target tracking result corresponding to the third target tracking operation of the current video frame is updated as the basic target tracking result corresponding to the fourth target tracking operation of the current video frame.

[0069] As an optional implementation, in the second aspect of the present invention, the target tracking module determines all fourth tracking tracks and their corresponding fifth high-score bounding boxes from all the third tracking tracks and all the fourth high-score bounding boxes according to the first feature information corresponding to all the third tracking tracks and the second feature information corresponding to all the fourth high-score bounding boxes, specifically including:

[0070] Calculating a Euclidean distance parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes according to the first feature information corresponding to all of the third tracking trajectories and the second feature information corresponding to all of the fourth high-score bounding boxes;

[0071] Calculating a cosine similarity parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes according to the first feature information corresponding to all of the third tracking trajectories and the second feature information corresponding to all of the fourth high-score bounding boxes;

[0072] According to the Euclidean distance parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes, and the cosine similarity parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes, all the fourth tracking trajectories whose Euclidean distance parameter is less than a preset Euclidean distance parameter threshold and whose cosine similarity parameter is less than a preset cosine similarity parameter threshold and their corresponding fifth high-score bounding boxes are determined from all the third tracking trajectories and all the fourth high-score bounding boxes.

[0073] As an optional implementation, in the second aspect of the present invention, the basic target tracking result of the current video frame includes a matching relationship between each basic tracking track and its corresponding basic bounding box, and the matching relationship between all the basic tracking tracks and their corresponding basic bounding boxes includes a matching relationship between the first target tracking track and its corresponding first high-score bounding box, a matching relationship between the second target tracking track and its corresponding target low-score bounding box, a matching relationship between the third target tracking track and its corresponding third high-score bounding box, a matching relationship between all the fourth tracking tracks and their corresponding fifth high-score bounding boxes, and a matching relationship between all the sixth high-score bounding boxes and their corresponding new tracking tracks;

[0074] The analysis module performs trajectory overlap analysis on the basic target tracking result according to the basic target tracking result of the current video frame to obtain the final target tracking result in the following manner:

[0075] According to the matching relationship between all the basic tracking tracks and their corresponding basic bounding boxes, all overlapping tracking tracks whose corresponding target external rectangular box screenshots have target overlap are determined from all the basic tracking tracks, and non-overlapping external rectangular box screenshots corresponding to each overlapping tracking track are obtained in the previous video frame;

[0076] According to the target external rectangular frame screenshots and the corresponding non-overlapping external rectangular frame screenshots corresponding to all the overlapping tracking trajectories, calculating the similarity between the target external rectangular frame screenshot and the corresponding non-overlapping external rectangular frame screenshot corresponding to each overlapping tracking trajectory;

[0077] According to the similarity between the target external rectangular frame screenshots corresponding to all the overlapping tracking trajectories and the corresponding non-overlapping external rectangular frame screenshots, determining the tracking trajectory to be adjusted whose similarity is less than or equal to a preset similarity threshold from all the overlapping tracking trajectories, and adjusting the basic bounding box corresponding to the tracking trajectory to be adjusted to obtain the adjusted basic bounding box corresponding to the tracking trajectory to be adjusted;

[0078] Based on the adjusted basic bounding box corresponding to the tracking trajectory to be adjusted, the basic target tracking result is updated to obtain a final target tracking result.

[0079] The third aspect of the present invention discloses another target tracking device based on Byte Track, the device comprising:

[0080] A memory storing executable program code;

[0081] a processor coupled to the memory;

[0082] The processor calls the executable program code stored in the memory to execute the target tracking method based on Byte Track disclosed in the first aspect of the present invention.

[0083] The fourth aspect of the present invention discloses a computer storage medium, wherein the computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the target tracking method based on Byte Track disclosed in the first aspect of the present invention.

[0084] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0085] In an embodiment of the present invention, for the current video frame to be tracked, all high-score bounding boxes and all low-score bounding boxes are determined from all bounding boxes according to the scores of all bounding boxes in the current video frame; all first tracking tracks are determined from all tracking tracks in the previous video frames according to the previous video frames corresponding to the current video frame, and based on all first tracking tracks, all high-score bounding boxes and all low-score bounding boxes, multiple target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame; and track overlap analysis is performed on the basic target tracking result to obtain a final target tracking result. It can be seen that the implementation of the present invention can reduce the occurrence of target loss due to occlusion or unstable target detection, thereby improving the target tracking accuracy of the current video frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0087] Figure 1 It is a flowchart of a target tracking method based on Byte Track disclosed in an embodiment of the present invention;

[0088] Figure 2 is a flow chart of another target tracking method based on Byte Track disclosed in an embodiment of the present invention;

[0089] Figure 3 It is a structural schematic diagram of a target tracking device based on Byte Track disclosed in an embodiment of the present invention;

[0090] Figure 4 It is a schematic diagram of the structure of another target tracking device based on Byte Track disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0091] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0092] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0093] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0094] The present invention discloses a target tracking method and device based on Byte Track, which can reduce the occurrence of target loss due to occlusion or unstable target detection, thereby improving the target tracking accuracy of the current video frame.

[0095] Embodiment 1

[0096] See also Figure 1 , Figure 1 is a flowchart of a target tracking method based on Byte Track disclosed in an embodiment of the present invention. Optionally, the method can be implemented by a target tracking device, which can be integrated in smart devices such as smart computers, smart phones, cameras, etc., or can be a local server or cloud server for processing the target tracking process of the current video frame, etc., which is not limited in the embodiment of the present invention. Figure 1 As shown, the target tracking method based on Byte Track may include the following operations:

[0097] 101. For a current video frame to be tracked, determine all high-score bounding boxes and all low-score bounding boxes from all bounding boxes according to the scores of all bounding boxes in the current video frame.

[0098] In the embodiment of the present invention, specifically, all high-score bounding boxes are bounding boxes with scores greater than or equal to a preset score threshold, and all low-score bounding boxes are bounding boxes with scores less than the preset score threshold, wherein the score can be understood as confidence.

[0099] 102. According to the previous video frame corresponding to the current video frame, all first tracking tracks are determined from all tracking tracks in the previous video frames, and based on all the first tracking tracks, all the high-score bounding boxes, and all the low-score bounding boxes, multiple target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame.

[0100] In an embodiment of the present invention, all tracking trajectories include at least activated tracking trajectories (which may include newly created trajectories of the target frame in the first video frame) that have been successfully tracked for two frames or more in previous video frames (i.e., all video frames with frame numbers before the current video frame), and all first tracking trajectories include trajectories that have been successfully tracked in the previous video frame corresponding to the current video frame and trajectories that have lost tracking in the previous n video frames corresponding to the current video frame, where n is a preset positive integer.

[0101] It should be noted that the tracking trajectory is created by the code starting from the first frame, which at least includes all trajectories of continuous tracking and can also include trajectories of interrupted tracking; while the bounding box of the current video frame refers to the bounding box obtained only from the current video frame and does not contain any information of previous video frames.

[0102] 103. According to the basic target tracking result of the current video frame, trajectory overlap analysis is performed on the basic target tracking result to obtain the final target tracking result.

[0103] In an embodiment of the present invention, each successfully matched basic tracking track in the current video frame and its corresponding basic bounding box are analyzed to see whether there is an overlapping tracking track with a target overlapped. If so, the basic bounding box corresponding to the overlapping tracking track is adjusted (i.e., ID adjustment), such as adjusting it to the bounding box corresponding to the target before the overlap, so as to obtain the final target tracking result.

[0104] It can be seen that the implementation of the embodiment of the present invention can first classify the bounding boxes in the current video frame, and then perform multiple target tracking operations on the current video frame based on all the first tracking trajectories, high-score bounding boxes and low-score bounding boxes determined from previous video frames to obtain the basic target tracking result of the current video frame, and perform trajectory overlap analysis on the basic target tracking result to obtain the final target tracking result. In this way, the occurrence of target loss due to occlusion or unstable target detection can be reduced, thereby improving the target tracking accuracy of the current video frame.

[0105] Embodiment 2

[0106] See also Figure 2 , Figure 2 is a flowchart of another target tracking method based on Byte Track disclosed in an embodiment of the present invention. Optionally, the method can be implemented by a target tracking device, which can be integrated in smart devices such as smart computers, smart phones, cameras, etc., or can be a local server or cloud server for processing the target tracking process of the current video frame, etc., which is not limited in the embodiment of the present invention. Figure 2 As shown, the target tracking method based on Byte Track may include the following operations:

[0107] 201. For a current video frame to be tracked, determine all high-score bounding boxes and all low-score bounding boxes from all bounding boxes according to the scores of all bounding boxes in the current video frame.

[0108] 202. According to the previous video frame corresponding to the current video frame, determine all first tracking trajectories from all tracking trajectories in the previous video frame.

[0109] 203. Perform a bounding box prediction operation on the next video frame corresponding to each first tracking trajectory to obtain bounding box prediction information corresponding to each first tracking trajectory.

[0110] In the embodiment of the present invention, optionally, the bounding box prediction information corresponding to each first tracking trajectory includes bounding box position information and / or bounding box size information of the predicted bounding box in the next video frame corresponding to the first tracking trajectory.

[0111] 204 . Calculate a first intersection-over-union ratio between the predicted bounding box corresponding to each first tracking trajectory and each high-score bounding box according to the bounding box prediction information corresponding to all first tracking trajectories and the bounding box information of all high-score bounding boxes.

[0112] 205 . Determine a first matching degree value between the predicted bounding box corresponding to each first tracking trajectory and each high-score bounding box according to a first intersection-and-union ratio between the predicted bounding box corresponding to each first tracking trajectory and each high-score bounding box.

[0113] 206. According to the first matching degree value between the predicted bounding box corresponding to each first tracking trajectory and each high-score bounding box, determine, from all the first tracking trajectories and all the high-score bounding boxes, a first target tracking trajectory having a first matching degree value greater than or equal to a preset first matching degree threshold and its corresponding first high-score bounding box.

[0114] In the embodiment of the present invention, the matching relationship between the first target tracking trajectory and its corresponding first high-score bounding box is used to indicate the target tracking result corresponding to the first target tracking operation of the current video frame, so as to update the first tracking trajectory.

[0115] For example, it can be understood as: (1) all tracked trajectories (Tracked, trajectories successfully tracked in the previous frame) and lost trajectories (Lost, trajectories lost in the previous n frames) are merged and called the first tracked trajectory (strack_pool);

[0116] (2) predicting the possible position and size of the next frame bounding box of the first tracking trajectory (the Kalman filter can be used to predict the bounding box);

[0117] (3) Calculate the IoU (intersection over union) value between the next frame bounding box predicted by the first tracking trajectory and the current frame high-resolution bounding box (detections), and obtain a pairwise IoU relationship loss matrix (the smaller the IoU, the greater the degree of association, and the maximum IoU value is 1, indicating that there is no intersection between the bounding boxes);

[0118] (4) According to the IoU loss matrix, the Hungarian algorithm is used to match the first tracking trajectory and the current frame high-score bounding box (detections), and three results are obtained: the matched trajectory (activated_stracks and refind_stracks) and bounding box, the unsuccessfully matched trajectory (r_tracked_stracks), and the unsuccessfully matched current frame bounding box (detections_cp) (the Hungarian algorithm can perform a one-to-one match between two pairs according to the loss matrix and return the results of successful and unsuccessful matching);

[0119] (5) Use the successfully matched current frame bounding box to update the first tracking trajectory (change the box in the preliminary tracking trajectory to the current frame bounding box, and the id remains the original id).

[0120] 207. According to the first target tracking track and the first high-score bounding box corresponding thereto, all second tracking tracks are determined from all first tracking tracks, and all second high-score bounding boxes are determined from all high-score bounding boxes, and based on all second tracking tracks and all second high-score bounding boxes, a second, a third, and a fourth target tracking operation are performed on the current video frame to obtain a basic target tracking result of the current video frame.

[0121] In the embodiment of the present invention, all second tracking trajectories are tracking trajectories without a first matching degree value greater than or equal to the first matching degree threshold, and all second high-score bounding boxes are high-score bounding boxes without a first matching degree value greater than or equal to the first matching degree threshold.

[0122] 208. According to the basic target tracking result of the current video frame, trajectory overlap analysis is performed on the basic target tracking result to obtain a final target tracking result.

[0123] In the embodiment of the present invention, for other descriptions of step 201, step 202 and step 208, please refer to the detailed description of step 101 to step 103 in the first embodiment, and the embodiment of the present invention will not be repeated.

[0124] It can be seen that the implementation of the embodiment of the present invention can improve the adaptability and flexibility of the target tracking system to various tracking situations through multi-level matching and tracking strategies (including the first target tracking operation and the subsequent second, third, and fourth target tracking operations). At the same time, the target tracking result is finally determined through trajectory overlap analysis, which can further improve the accuracy and reliability of the result.

[0125] In an optional embodiment, in the above step 207, based on all the second tracking tracks and all the second high-score bounding boxes, the second, third and fourth target tracking operations are performed on the current video frame to obtain the basic target tracking result of the current video frame, including:

[0126] Calculate a second intersection-and-union ratio between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box according to the bounding box prediction information corresponding to all second tracking trajectories and the bounding box information of all low-score bounding boxes;

[0127] Determining a second matching degree value between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box according to a second intersection-and-union ratio between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box;

[0128] According to the second matching degree value between the predicted bounding box corresponding to each second tracking track and each low-score bounding box, a second target tracking track having a second matching degree value greater than or equal to a preset second matching degree threshold and its corresponding target low-score bounding box are determined from all the second tracking tracks and all the low-score bounding boxes;

[0129] According to the second target tracking track, all third tracking tracks are determined from all second tracking tracks, and based on all second high-score bounding boxes and all third tracking tracks, third and fourth target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame.

[0130] In this optional embodiment, the matching relationship between the second target tracking trajectory and its corresponding target low-score bounding box is used to update the target tracking result corresponding to the first target tracking operation of the current video frame as the target tracking result corresponding to the second target tracking operation of the current video frame; all third tracking trajectories are tracking trajectories in which there is no second matching degree value greater than or equal to the second matching degree threshold.

[0131] Further, before calculating a second intersection-over-union ratio between the predicted bounding box corresponding to each second tracking track and each low-score bounding box according to the bounding box prediction information corresponding to all second tracking tracks and the bounding box information of all low-score bounding boxes, the method further includes:

[0132] Trajectories that have lost tracking in the first n video frames corresponding to the current video frame are eliminated from all second tracking trajectories to update all second tracking trajectories, and an operation of calculating a second intersection-and-union ratio between a predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box is triggered based on bounding box prediction information corresponding to all second tracking trajectories and bounding box information of all low-score bounding boxes.

[0133] For example, it can be understood as: (1) finding the tracks that were not matched in the first matching, and filtering out the tracked tracks (in the low-score matching process, those tracks that have been lost may not be matched), that is, all the second tracked tracks updated in the above;

[0134] (2) Because these trajectories have already predicted the bounding box of the next frame, there is no need to predict it here;

[0135] (3) Calculate the IoU between all the updated second tracking trajectories and the low-score bounding box of the current video frame;

[0136] (4) Use the Hungarian algorithm to match all the updated second tracking tracks (r_tracked_stracks) with the low-score bounding box (detections_low) of the current video frame;

[0137] (5) using the successfully matched low-score bounding box of the current video frame (i.e., the second target tracking track and its corresponding target low-score bounding box) to update the above tracking track;

[0138] (6) The tracks that have not been successfully tracked at this time can be marked as lost tracks.

[0139] It can be seen that this optional embodiment can further increase the coverage of target tracking by introducing the matching and processing of low-score bounding boxes. After the initial matching of the high-score bounding box, the remaining second tracking trajectory is matched with the low-score bounding box, which can effectively reduce the situation of missed tracking, especially those target objects with low scores due to occlusion, perspective change or detection error, thereby improving the integrity and accuracy of the overall tracking.

[0140] In another optional embodiment, all tracking trajectories also include inactive tracking trajectories that have not been successfully tracked in previous video frames;

[0141] In the above steps, based on all the second highest score bounding boxes and all the third tracking tracks, the third and fourth target tracking operations are performed on the current video frame to obtain the basic target tracking result of the current video frame, including:

[0142] Performing a bounding box prediction operation on the next video frame corresponding to the inactive tracking track to obtain bounding box prediction information corresponding to the inactive tracking track;

[0143] Calculate a third intersection-and-union ratio between the predicted bounding box corresponding to the inactive tracking track and each second highest score bounding box according to the bounding box prediction information corresponding to the inactive tracking track and the bounding box information of all second highest score bounding boxes;

[0144] Determining a third matching degree value between the predicted bounding box corresponding to the inactive state tracking trajectory and each second high score bounding box according to a third intersection-union ratio between the predicted bounding box corresponding to the inactive state tracking trajectory and each second high score bounding box;

[0145] According to the third matching degree value between the predicted bounding box corresponding to the inactive tracking track and each second high-scoring bounding box, determine, from the inactive tracking track and all the second high-scoring bounding boxes, a third target tracking track having a third matching degree value greater than or equal to a preset third matching degree threshold and its corresponding third high-scoring bounding box;

[0146] According to the third target tracking trajectory and its corresponding third high-score bounding box, all fourth high-score bounding boxes are determined from all high-score bounding boxes, and based on all fourth high-score bounding boxes and all third tracking trajectories, a fourth target tracking operation is performed on the current video frame to obtain a basic target tracking result of the current video frame.

[0147] In this optional embodiment, optionally, the bounding box prediction information corresponding to the inactive state tracking track includes the bounding box position information and / or bounding box size information of the predicted bounding box in the next video frame corresponding to the inactive state tracking track. The matching relationship between the third target tracking track and its corresponding third high-scoring bounding box is used to update the target tracking result corresponding to the second target tracking operation of the current video frame as the target tracking result corresponding to the third target tracking operation of the current video frame, and all fourth high-scoring bounding boxes are high-scoring bounding boxes without a third matching degree value greater than or equal to the third matching degree threshold.

[0148] For example, it can be understood as: (1) finding the bounding box (detections_cp) of the current video frame that was not successfully matched in the first target tracking operation (no high-scoring bounding box that was successfully matched), and finding the inactivated track (unconfirmed);

[0149] (2) Calculate the IoU between the above trajectory and the bounding box of the current video frame;

[0150] (3) Use the Hungarian algorithm to match the above tracking trajectory and bounding box;

[0151] (4) Update the above tracking tracks (activated_stracks) using the successfully matched current frame bounding box;

[0152] (5) At this time, the inactivated tracks that have not been successfully tracked can be directly marked as deleted tracks (removed_stracks).

[0153] It can be seen that this optional embodiment can effectively restore the target trajectory temporarily lost due to occlusion, motion blur and other factors by predicting the bounding box position and size of the inactivated tracking trajectory in the next frame and matching it with the high-resolution bounding box of the current frame, thereby enhancing the continuity and stability of target tracking; at the same time, by setting a matching degree threshold, it effectively distinguishes between tracking trajectories and bounding boxes with high and low matching degrees, and promptly marks the inactivated trajectories that have not been successfully matched as deleted, thereby avoiding the waste of invalid tracking resources and improving computational efficiency.

[0154] In yet another optional embodiment, in the above step, based on all fourth high-scoring bounding boxes and all third tracking tracks, a fourth target tracking operation is performed on the current video frame to obtain a basic target tracking result of the current video frame, including:

[0155] Obtaining a screenshot of a rectangular frame of the target outside the third tracking track corresponding to each third tracking track, and determining first feature information corresponding to each third tracking track according to the screenshot of the rectangular frame of the target outside the third tracking track corresponding to each third tracking track;

[0156] Extracting the second feature information corresponding to each fourth high-score bounding box, and determining all fourth tracking tracks and their corresponding fifth high-score bounding boxes from all third tracking tracks and all fourth high-score bounding boxes according to the first feature information corresponding to all third tracking tracks and the second feature information corresponding to all fourth high-score bounding boxes;

[0157] Based on all fifth-highest scoring bounding boxes, determine all sixth-highest scoring bounding boxes from all fourth-highest scoring bounding boxes, and establish new tracking tracks for all sixth-highest scoring bounding boxes;

[0158] According to the matching relationship between all fourth tracking trajectories and their corresponding fifth high-score bounding boxes, and the matching relationship between all sixth high-score bounding boxes and their corresponding new tracking trajectories, the target tracking result corresponding to the third target tracking operation of the current video frame is updated as the basic target tracking result corresponding to the fourth target tracking operation of the current video frame.

[0159] In this optional embodiment, further, according to the first feature information corresponding to all the third tracking tracks and the second feature information corresponding to all the fourth high-score bounding boxes, all the fourth tracking tracks and their corresponding fifth high-score bounding boxes are determined from all the third tracking tracks and all the fourth high-score bounding boxes, including:

[0160] Calculating a Euclidean distance parameter between each third tracking track and each fourth high-score bounding box according to the first feature information corresponding to all third tracking tracks and the second feature information corresponding to all fourth high-score bounding boxes;

[0161] Calculating a cosine similarity parameter between each third tracking track and each fourth high-score bounding box according to the first feature information corresponding to all third tracking tracks and the second feature information corresponding to all fourth high-score bounding boxes;

[0162] According to the Euclidean distance parameter between each third tracking trajectory and each fourth high-score bounding box, and the cosine similarity parameter between each third tracking trajectory and each fourth high-score bounding box, all fourth tracking trajectories having a Euclidean distance parameter less than a preset Euclidean distance parameter threshold and a cosine similarity parameter less than a preset cosine similarity parameter threshold and their corresponding fifth high-score bounding boxes are determined from all third tracking trajectories and all fourth high-score bounding boxes.

[0163] For example, it can be understood as: (1) finding the high-scoring bounding boxes (u_detection) that have not been successfully matched, and finding the unmatched tracks (including: tracks that have been successfully tracked in the previous video frame corresponding to the current video frame that have not been successfully matched, and tracks that have lost tracking in the previous n video frames corresponding to the current video frame that have not been successfully matched);

[0164] (2) Extract the feature F1 of the unmatched high-score bounding box and the feature Fi of the unmatched trajectory (the unmatched high-score bounding box directly uses the ReID feature extraction model to extract the feature F1; for all trajectories, the ROI image of the most recent / latest frame will be updated and saved. Therefore, the feature Fi can be extracted based on the ROI image of the unmatched trajectory in combination with the ReID model);

[0165] (3) Feature matching between the high-score detection frame and the unmatched trajectory is performed. The matching principle is:

[0166] Calculate the Euclidean distance Ei between the high-resolution frame and the latest frame of the trajectory;

[0167] <ii>Calculate the cosine similarity Si between the high-score frame and the latest frame of the trajectory;

[0168] <iii>When Ei < T1 (T1 is the threshold of Euclidean distance) and Si < T2 (T2 is the threshold of cosine similarity) respectively meet the threshold requirements, the high-score bounding box and the trajectory are successfully matched.

[0169] (4) After traversing and completing the matching, output the successfully matched trajectories (activated_stracks), the unsuccessfully matched trajectories (removed_stracks), and the unsuccessfully matched high-score bounding boxes (u_detection).

[0170] (5) Establish new trajectories for the unsuccessfully matched high-score bounding boxes (u_detection).

[0171] It should be noted that by reconfirming the matching between the new target and the trajectory based on the ReID feature, it is determined whether the target is a previously disappeared target. If it is a disappeared target, the original ID information is retained; if it is not a previously disappeared target, a new ID and trajectory are created. This optimization scheme effectively solves the problem of reassigning IDs and trajectories when the target reappears in the case of target loss caused by occlusion and unstable target detection, and can ensure that the ID remains consistent when the target reappears after being lost.

[0172] It can be seen that this optional embodiment can combine two feature matching strategies of Euclidean distance and cosine similarity to match the remaining unmatched tracking trajectories and the remaining unmatched high-score bounding boxes, effectively reducing the cases of false matching and missed matching, thereby improving the accuracy and stability of target tracking.

[0173] In another optional embodiment, the basic target tracking result of the current video frame includes the matching relationship between each basic tracking trajectory and its corresponding basic bounding box. The matching relationships between all basic tracking trajectories and their corresponding basic bounding boxes include the matching relationship between the first target tracking trajectory and its corresponding first high-score bounding box, the matching relationship between the second target tracking trajectory and its corresponding low-score bounding box, the matching relationship between the third target tracking trajectory and its corresponding third high-score bounding box, the matching relationship between all fourth tracking trajectories and their corresponding fifth high-score bounding boxes, and the matching relationship between all sixth high-score bounding boxes and their corresponding new tracking trajectories;

[0174] Among them, according to the basic target tracking result of the current video frame, performing trajectory overlap analysis on the basic target tracking result to obtain the final target tracking result includes:

[0175] According to the matching relationship between all basic tracking tracks and their corresponding basic bounding boxes, all overlapping tracking tracks whose corresponding target external rectangular box screenshots have target overlap are determined from all basic tracking tracks, and non-overlapping external rectangular box screenshots corresponding to each overlapping tracking track are obtained in previous video frames;

[0176] According to the target external rectangular frame screenshots corresponding to all overlapping tracking trajectories and the corresponding non-overlapping external rectangular frame screenshots, the similarity between the target external rectangular frame screenshot corresponding to each overlapping tracking trajectory and the corresponding non-overlapping external rectangular frame screenshot is calculated;

[0177] According to the similarity between the target outer rectangular frame screenshots corresponding to all overlapping tracking tracks and the corresponding non-overlapping outer rectangular frame screenshots, the tracking track to be adjusted whose similarity is less than or equal to a preset similarity threshold is determined from all overlapping tracking tracks, and the basic bounding box corresponding to the tracking track to be adjusted is adjusted to obtain the adjusted basic bounding box corresponding to the tracking track to be adjusted;

[0178] Based on the adjusted basic bounding box corresponding to the tracking trajectory to be adjusted, the basic target tracking result is updated to obtain the final target tracking result.

[0179] In this optional embodiment, for example, it can be understood as follows: (1) performing a trajectory overlap analysis on all successfully matched trajectories, and saving ROI image information of the corresponding targets before overlapping and after the overlapping state is changed to separation;

[0180] (2) For multiple targets whose trajectories intersect, feature comparison is performed, that is, the ROI image feature F1 before the overlap and the ROI image feature F2 after the overlap is separated are extracted. If the features before and after the overlap are similar, it proves that the tracking before and after the overlap is accurate, and there is no need to update the trajectory information. If the features before and after the overlap are not similar, it proves that the tracking before and after the overlap is inaccurate, and the ID exchange has occurred, and the ID of the overlapping target needs to be switched.

[0181] It should be noted that this step focuses on optimizing the situation where IDs are swapped after targets overlap. First, the ROI image before the target overlaps and the ROI image after the overlap changes to a separated state are saved. ReID is used to extract the features of the targets before and after the overlap, and feature matching is performed to confirm whether the ID switches after the targets overlap. This solves the problem of being unable to associate the ID when the target appears again (matching unsuccessful) when the target is lost due to occlusion and unstable target detection; it also ensures the accuracy of the trajectory before and after the target overlaps.

[0182] It can be seen that this optional embodiment can deeply analyze and process the trajectory of the target after overlapping. This strategy effectively solves the problem of target ID interchange caused by occlusion, unstable detection, etc., and by saving the ROI images before and after the overlap and performing feature comparison, it can accurately identify and correct erroneous tracking trajectories, thereby ensuring the continuity and accuracy of target tracking. In this way, when facing complex and changeable video scenes, such as frequent target overlap, occlusion, and rapid movement, the stability and robustness of the system can be ensured, as well as maintaining high tracking accuracy and reliability.

[0183] Embodiment 3

[0184] See also Figure 3 , Figure 3 FIG. 1 is a schematic diagram of a target tracking device based on Byte Track disclosed in an embodiment of the present invention. Figure 3 As shown, the target tracking device based on Byte Track may include:

[0185] The determination module 301 is used to determine, for a current video frame to be tracked, all high-scoring bounding boxes and all low-scoring bounding boxes from all bounding boxes according to the scores of all bounding boxes in the current video frame; and to determine all first tracking trajectories from all tracking trajectories in the previous video frames according to the previous video frames corresponding to the current video frame;

[0186] The target tracking module 302 is used to perform multiple target tracking operations on the current video frame based on all the first tracking tracks, all the high-score bounding boxes, and all the low-score bounding boxes to obtain a basic target tracking result of the current video frame;

[0187] The analysis module 303 is used to perform trajectory overlap analysis on the basic target tracking result according to the basic target tracking result of the current video frame to obtain the final target tracking result.

[0188] In an embodiment of the present invention, all high-score boundary boxes are boundary boxes with scores greater than or equal to a preset score threshold, and all low-score boundary boxes are boundary boxes with scores less than the preset score threshold; all tracking trajectories at least include activated tracking trajectories that have been successfully tracked for two frames or more in previous video frames, and all first tracking trajectories include trajectories that have been successfully tracked in the previous video frame corresponding to the current video frame and trajectories that have lost tracking in the previous n video frames corresponding to the current video frame, where n is a preset positive integer.

[0189] It can be seen that implementation Figure 3 The described Byte Track-based target tracking device can first classify the bounding boxes in the current video frame, and then perform multiple target tracking operations on the current video frame based on all first tracking tracks, high-score bounding boxes and low-score bounding boxes determined from previous video frames to obtain a basic target tracking result of the current video frame, and perform track overlap analysis on the basic target tracking result to obtain a final target tracking result. In this way, the occurrence of target loss due to occlusion or unstable target detection can be reduced, thereby improving the target tracking accuracy of the current video frame.

[0190] In an optional embodiment, the target tracking module 302 performs multiple target tracking operations on the current video frame based on all the first tracking tracks, all the high-score bounding boxes, and all the low-score bounding boxes, and obtains the basic target tracking result of the current video frame in a manner that specifically includes:

[0191] Performing a bounding box prediction operation on a next video frame corresponding to each first tracking trajectory to obtain bounding box prediction information corresponding to each first tracking trajectory;

[0192] Calculate a first intersection-and-union ratio between the predicted bounding box corresponding to each first tracking trajectory and each high-score bounding box according to the bounding box prediction information corresponding to all first tracking trajectories and the bounding box information of all high-score bounding boxes;

[0193] Determining a first matching degree value between the predicted bounding box corresponding to each first tracking trajectory and each high-score bounding box according to a first intersection-and-union ratio between the predicted bounding box corresponding to each first tracking trajectory and each high-score bounding box;

[0194] According to the first matching degree value between the predicted bounding box corresponding to each first tracking track and each high-scoring bounding box, a first target tracking track having a first matching degree value greater than or equal to a preset first matching degree threshold and a first high-scoring bounding box corresponding to the first tracking track and all the high-scoring bounding boxes are determined;

[0195] According to the first target tracking track and its corresponding first high-score bounding box, all second tracking tracks are determined from all first tracking tracks, and all second high-score bounding boxes are determined from all high-score bounding boxes, and based on all second tracking tracks and all second high-score bounding boxes, the second, third and fourth target tracking operations are performed on the current video frame to obtain the basic target tracking result of the current video frame.

[0196] In this optional embodiment, the bounding box prediction information corresponding to each first tracking trajectory includes the bounding box position information and / or bounding box size information of the predicted bounding box in the next video frame corresponding to the first tracking trajectory; the matching relationship between the first target tracking trajectory and the first high-score bounding box corresponding to it is used to indicate the target tracking result corresponding to the first target tracking operation of the current video frame; all second tracking trajectories are tracking trajectories whose first matching degree value is greater than or equal to the first matching degree threshold, and all second high-score bounding boxes are high-score bounding boxes whose first matching degree value is greater than or equal to the first matching degree threshold.

[0197] It can be seen that implementation Figure 3 The described Byte Track-based target tracking device can improve the adaptability and flexibility of the target tracking system to various tracking situations through a multi-level matching and tracking strategy (including the first target tracking operation and the subsequent second, third, and fourth target tracking operations). At the same time, the target tracking result is finally determined through trajectory overlap analysis, which can further improve the accuracy and reliability of the result.

[0198] In another optional embodiment, the target tracking module 302 performs the second, third and fourth target tracking operations on the current video frame based on all the second tracking trajectories and all the second high-score bounding boxes, and obtains the basic target tracking result of the current video frame in a manner that specifically includes:

[0199] Calculate a second intersection-and-union ratio between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box according to the bounding box prediction information corresponding to all second tracking trajectories and the bounding box information of all low-score bounding boxes;

[0200] Determining a second matching degree value between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box according to a second intersection-and-union ratio between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box;

[0201] According to the second matching degree value between the predicted bounding box corresponding to each second tracking track and each low-score bounding box, a second target tracking track having a second matching degree value greater than or equal to a preset second matching degree threshold and its corresponding target low-score bounding box are determined from all the second tracking tracks and all the low-score bounding boxes;

[0202] According to the second target tracking track, all third tracking tracks are determined from all second tracking tracks, and based on all second high-score bounding boxes and all third tracking tracks, third and fourth target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame.

[0203] In this optional embodiment, the matching relationship between the second target tracking trajectory and its corresponding target low-score bounding box is used to update the target tracking result corresponding to the first target tracking operation of the current video frame as the target tracking result corresponding to the second target tracking operation of the current video frame; all third tracking trajectories are tracking trajectories in which there is no second matching degree value greater than or equal to the second matching degree threshold.

[0204] It can be seen that implementation Figure 3 The described Byte Track-based target tracking device can further increase the coverage of target tracking by introducing the matching and processing of low-score bounding boxes. After the initial matching of the high-score bounding box, the remaining second tracking trajectory is matched with the low-score bounding box, which can effectively reduce the missed tracking, especially those target objects with low scores due to occlusion, perspective change or detection error, thereby improving the integrity and accuracy of the overall tracking.

[0205] In yet another optional embodiment, all tracking trajectories also include inactive tracking trajectories that have not been successfully tracked in previous video frames;

[0206] The target tracking module 302 performs the third and fourth target tracking operations on the current video frame based on all the second high-scoring bounding boxes and all the third tracking tracks, and obtains the basic target tracking result of the current video frame in the following manner:

[0207] Performing a bounding box prediction operation on the next video frame corresponding to the inactive tracking track to obtain bounding box prediction information corresponding to the inactive tracking track;

[0208] Calculate a third intersection-and-union ratio between the predicted bounding box corresponding to the inactive tracking track and each second highest score bounding box according to the bounding box prediction information corresponding to the inactive tracking track and the bounding box information of all second highest score bounding boxes;

[0209] Determining a third matching degree value between the predicted bounding box corresponding to the inactive state tracking trajectory and each second high score bounding box according to a third intersection-union ratio between the predicted bounding box corresponding to the inactive state tracking trajectory and each second high score bounding box;

[0210] According to the third matching degree value between the predicted bounding box corresponding to the inactive tracking track and each second high-scoring bounding box, determine, from the inactive tracking track and all the second high-scoring bounding boxes, a third target tracking track having a third matching degree value greater than or equal to a preset third matching degree threshold and its corresponding third high-scoring bounding box;

[0211] According to the third target tracking trajectory and its corresponding third high-score bounding box, all fourth high-score bounding boxes are determined from all high-score bounding boxes, and based on all fourth high-score bounding boxes and all third tracking trajectories, a fourth target tracking operation is performed on the current video frame to obtain a basic target tracking result of the current video frame.

[0212] In this optional embodiment, the bounding box prediction information corresponding to the inactivated tracking trajectory includes the bounding box position information and / or bounding box size information of the predicted bounding box in the next video frame corresponding to the inactivated tracking trajectory; the matching relationship between the third target tracking trajectory and its corresponding third high-score bounding box is used to update the target tracking result corresponding to the second target tracking operation of the current video frame as the target tracking result corresponding to the third target tracking operation of the current video frame; all fourth high-score bounding boxes are high-score bounding boxes whose third matching degree value is greater than or equal to the third matching degree threshold.

[0213] It can be seen that implementation Figure 3 The described Byte Track-based target tracking device can effectively restore the target trajectory temporarily lost due to occlusion, motion blur and other factors by predicting the bounding box position and size of the inactivated tracking track in the next frame and matching it with the high-resolution bounding box of the current frame, thereby enhancing the continuity and stability of target tracking. At the same time, by setting a matching degree threshold, it effectively distinguishes between tracking tracks and bounding boxes with high and low matching degrees, and promptly marks the inactivated tracks that have not been successfully matched as deleted, thereby avoiding the waste of invalid tracking resources and improving computational efficiency.

[0214] In another optional embodiment, the target tracking module 302 performs a fourth target tracking operation on the current video frame based on all fourth high-scoring bounding boxes and all third tracking tracks, and obtains a basic target tracking result of the current video frame in a manner that specifically includes:

[0215] Obtaining a screenshot of a rectangular frame of the target outside the third tracking track corresponding to each third tracking track, and determining first feature information corresponding to each third tracking track according to the screenshot of the rectangular frame of the target outside the third tracking track corresponding to each third tracking track;

[0216] Extracting the second feature information corresponding to each fourth high-score bounding box, and determining all fourth tracking tracks and their corresponding fifth high-score bounding boxes from all third tracking tracks and all fourth high-score bounding boxes according to the first feature information corresponding to all third tracking tracks and the second feature information corresponding to all fourth high-score bounding boxes;

[0217] Based on all fifth-highest scoring bounding boxes, determine all sixth-highest scoring bounding boxes from all fourth-highest scoring bounding boxes, and establish new tracking tracks for all sixth-highest scoring bounding boxes;

[0218] According to the matching relationship between all fourth tracking trajectories and their corresponding fifth high-score bounding boxes, and the matching relationship between all sixth high-score bounding boxes and their corresponding new tracking trajectories, the target tracking result corresponding to the third target tracking operation of the current video frame is updated as the basic target tracking result corresponding to the fourth target tracking operation of the current video frame.

[0219] In this optional embodiment, further, the target tracking module 302 determines all fourth tracking tracks and their corresponding fifth high-score bounding boxes from all third tracking tracks and all fourth high-score bounding boxes according to the first feature information corresponding to all third tracking tracks and the second feature information corresponding to all fourth high-score bounding boxes, specifically including:

[0220] Calculating a Euclidean distance parameter between each third tracking track and each fourth high-score bounding box according to the first feature information corresponding to all third tracking tracks and the second feature information corresponding to all fourth high-score bounding boxes;

[0221] Calculating a cosine similarity parameter between each third tracking track and each fourth high-score bounding box according to the first feature information corresponding to all third tracking tracks and the second feature information corresponding to all fourth high-score bounding boxes;

[0222] According to the Euclidean distance parameter between each third tracking trajectory and each fourth high-score bounding box, and the cosine similarity parameter between each third tracking trajectory and each fourth high-score bounding box, all fourth tracking trajectories having a Euclidean distance parameter less than a preset Euclidean distance parameter threshold and a cosine similarity parameter less than a preset cosine similarity parameter threshold and their corresponding fifth high-score bounding boxes are determined from all third tracking trajectories and all fourth high-score bounding boxes.

[0223] It can be seen that implementation Figure 3 The described Byte Track-based target tracking device can match the remaining unmatched tracking tracks with the remaining unmatched high-score bounding boxes by combining two feature matching strategies, namely Euclidean distance and cosine similarity, effectively reducing the situations of false matching and missed matching, thereby improving the accuracy and stability of target tracking.

[0224] In yet another optional embodiment, the basic target tracking result of the current video frame includes a matching relationship between each basic tracking track and its corresponding basic bounding box, and the matching relationship between all basic tracking tracks and their corresponding basic bounding boxes includes a matching relationship between a first target tracking track and its corresponding first high-score bounding box, a matching relationship between a second target tracking track and its corresponding target low-score bounding box, a matching relationship between a third target tracking track and its corresponding third high-score bounding box, a matching relationship between all fourth tracking tracks and their corresponding fifth high-score bounding boxes, and a matching relationship between all sixth high-score bounding boxes and their corresponding new tracking tracks;

[0225] The analysis module 303 performs trajectory overlap analysis on the basic target tracking result according to the basic target tracking result of the current video frame, and obtains the final target tracking result in the following manner:

[0226] According to the matching relationship between all basic tracking tracks and their corresponding basic bounding boxes, all overlapping tracking tracks whose corresponding target external rectangular box screenshots have target overlap are determined from all basic tracking tracks, and non-overlapping external rectangular box screenshots corresponding to each overlapping tracking track are obtained in previous video frames;

[0227] According to the target external rectangular frame screenshots corresponding to all overlapping tracking trajectories and the corresponding non-overlapping external rectangular frame screenshots, the similarity between the target external rectangular frame screenshot corresponding to each overlapping tracking trajectory and the corresponding non-overlapping external rectangular frame screenshot is calculated;

[0228] According to the similarity between the target outer rectangular frame screenshots corresponding to all overlapping tracking tracks and the corresponding non-overlapping outer rectangular frame screenshots, the tracking track to be adjusted whose similarity is less than or equal to a preset similarity threshold is determined from all overlapping tracking tracks, and the basic bounding box corresponding to the tracking track to be adjusted is adjusted to obtain the adjusted basic bounding box corresponding to the tracking track to be adjusted;

[0229] Based on the adjusted basic bounding box corresponding to the tracking trajectory to be adjusted, the basic target tracking result is updated to obtain the final target tracking result.

[0230] It can be seen that implementation Figure 3 The described target tracking device based on Byte Track can deeply analyze and process the trajectory after target overlap. This strategy effectively solves the problem of target ID interchange caused by occlusion, unstable detection, etc., and can accurately identify and correct erroneous tracking trajectories by saving ROI images before and after overlap and performing feature comparison, thereby ensuring the continuity and accuracy of target tracking. In this way, in the face of complex and changeable video scenes, such as frequent target overlap, occlusion, and rapid movement, the stability and robustness of the system can be ensured, and high tracking accuracy and reliability can be maintained.

[0231] Embodiment 4

[0232] See also Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the structure of another target tracking device based on Byte Track disclosed in an embodiment of the present invention. Figure 4 As shown, the target tracking device based on Byte Track may include:

[0233] A memory 401 storing executable program codes;

[0234] a processor 402 coupled to the memory 401;

[0235] The processor 402 calls the executable program code stored in the memory 401 to execute the steps of the target tracking method based on Byte Track described in the first embodiment of the present invention or the second embodiment of the present invention.

[0236] Embodiment 5

[0237] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps of the target tracking method based on Byte Track described in Embodiment 1 or Embodiment 2 of the present invention.

[0238] Embodiment 6

[0239] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the target tracking method based on Byte Track described in Embodiment 1 or Embodiment 2.

[0240] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0241] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0242] Finally, it should be noted that the target tracking method and device based on Byte Track disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.< / iii> < / ii>

Claims

1. A target tracking method based on Byte Track, characterized in that: The method comprises: For a current video frame to be tracked, according to the scores of all bounding boxes in the current video frame, all high-score bounding boxes and all low-score bounding boxes are determined from all the bounding boxes; all the high-score bounding boxes are bounding boxes whose scores are greater than or equal to a preset score threshold, and all the low-score bounding boxes are bounding boxes whose scores are less than the preset score threshold; According to the previous video frame corresponding to the current video frame, all first tracking trajectories are determined from all tracking trajectories in the previous video frame, and based on all the first tracking trajectories, all the high-score bounding boxes and all the low-score bounding boxes, multiple target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame; all the tracking trajectories at least include active tracking trajectories that have been successfully tracked for two frames or more in the previous video frames, and all the first tracking trajectories include trajectories that have been successfully tracked in a previous video frame corresponding to the current video frame and trajectories that have lost tracking in the previous n video frames corresponding to the current video frame, where n is a preset positive integer; According to the basic target tracking result of the current video frame, trajectory overlap analysis is performed on the basic target tracking result to obtain a final target tracking result.

2. The target tracking method based on Byte Track according to claim 1, characterized in that: The performing multiple target tracking operations on the current video frame based on all the first tracking trajectories, all the high-score bounding boxes, and all the low-score bounding boxes to obtain a basic target tracking result of the current video frame includes: Performing a bounding box prediction operation on a next video frame corresponding to each of the first tracking trajectories to obtain bounding box prediction information corresponding to each of the first tracking trajectories; the bounding box prediction information corresponding to each of the first tracking trajectories includes bounding box position information and / or bounding box size information of a predicted bounding box in the next video frame corresponding to the first tracking trajectory; Calculating a first intersection-and-union ratio between a predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes according to the bounding box prediction information corresponding to all of the first tracking trajectories and the bounding box information of all of the high-score bounding boxes; Determining a first matching degree value between the predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes according to a first intersection-and-union ratio between the predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes; According to the first matching degree value between the predicted bounding box corresponding to each of the first tracking trajectories and each of the high-score bounding boxes, a first target tracking trajectory whose first matching degree value is greater than or equal to a preset first matching degree threshold and its corresponding first high-score bounding box are determined from all the first tracking trajectories and all the high-score bounding boxes; the matching relationship between the first target tracking trajectory and its corresponding first high-score bounding box is used to indicate the target tracking result corresponding to the first target tracking operation of the current video frame; According to the first target tracking track and the first high-score bounding box corresponding to it, all second tracking tracks are determined from all the first tracking tracks, and all second high-score bounding boxes are determined from all the high-score bounding boxes, and based on all the second tracking tracks and all the second high-score bounding boxes, second, third and fourth target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame; all the second tracking tracks are tracking tracks whose first matching degree value is greater than or equal to the first matching degree threshold, and all the second high-score bounding boxes are high-score bounding boxes whose first matching degree value is greater than or equal to the first matching degree threshold.

3. The target tracking method based on Byte Track according to claim 2, characterized in that: The performing second, third and fourth target tracking operations on the current video frame based on all the second tracking trajectories and all the second high-score bounding boxes to obtain a basic target tracking result of the current video frame includes: Calculating a second intersection-and-union ratio between each predicted bounding box corresponding to the second tracking trajectory and each low-score bounding box according to the bounding box prediction information corresponding to all the second tracking trajectories and the bounding box information of all the low-score bounding boxes; Determining a second matching degree value between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box according to a second intersection-and-union ratio between the predicted bounding box corresponding to each second tracking trajectory and each low-score bounding box; According to the second matching degree value between the predicted bounding box corresponding to each second tracking trajectory and each of the low-score bounding boxes, a second target tracking trajectory whose second matching degree value is greater than or equal to a preset second matching degree threshold and its corresponding target low-score bounding box are determined from all the second tracking trajectories and all the low-score bounding boxes; the matching relationship between the second target tracking trajectory and its corresponding target low-score bounding box is used to update the target tracking result corresponding to the first target tracking operation of the current video frame, as the target tracking result corresponding to the second target tracking operation of the current video frame; According to the second target tracking trajectory, all third tracking trajectories are determined from all the second tracking trajectories, and based on all the second high-score bounding boxes and all the third tracking trajectories, third and fourth target tracking operations are performed on the current video frame to obtain a basic target tracking result of the current video frame; all the third tracking trajectories are tracking trajectories whose second matching degree value is greater than or equal to the second matching degree threshold.

4. The target tracking method based on Byte Track according to claim 3, characterized in that: All of the tracking trajectories also include inactive tracking trajectories that were not successfully tracked in the previous video frames; The performing of the third and fourth target tracking operations on the current video frame based on all the second high-resolution bounding boxes and all the third tracking trajectories to obtain a basic target tracking result of the current video frame includes: Performing a bounding box prediction operation on a next video frame corresponding to the inactive tracking track to obtain bounding box prediction information corresponding to the inactive tracking track; the bounding box prediction information corresponding to the inactive tracking track includes bounding box position information and / or bounding box size information of a predicted bounding box in the next video frame corresponding to the inactive tracking track; Calculating a third intersection-and-union ratio between the predicted bounding box corresponding to the inactive state tracking trajectory and each of the second high-score bounding boxes according to the bounding box prediction information corresponding to the inactive state tracking trajectory and the bounding box information of all the second high-score bounding boxes; Determining a third matching degree value between the predicted bounding box corresponding to the inactive state tracking trajectory and each of the second high-score bounding boxes according to a third intersection-and-union ratio between the predicted bounding box corresponding to the inactive state tracking trajectory and each of the second high-score bounding boxes; According to the third matching degree value between the predicted bounding box corresponding to the inactivated tracking trajectory and each of the second high-score bounding boxes, a third target tracking trajectory whose third matching degree value is greater than or equal to a preset third matching degree threshold and its corresponding third high-score bounding box are determined from the inactivated tracking trajectory and all the second high-score bounding boxes; the matching relationship between the third target tracking trajectory and its corresponding third high-score bounding box is used to update the target tracking result corresponding to the second target tracking operation of the current video frame, as the target tracking result corresponding to the third target tracking operation of the current video frame; According to the third target tracking trajectory and its corresponding third high-score bounding box, all fourth high-score bounding boxes are determined from all the high-score bounding boxes, and based on all the fourth high-score bounding boxes and all the third tracking trajectories, a fourth target tracking operation is performed on the current video frame to obtain a basic target tracking result of the current video frame; all the fourth high-score bounding boxes are high-score bounding boxes whose third matching degree values ​​are greater than or equal to the third matching degree threshold.

5. The target tracking method based on Byte Track according to claim 4, characterized in that: The performing a fourth target tracking operation on the current video frame based on all the fourth high-score bounding boxes and all the third tracking tracks to obtain a basic target tracking result of the current video frame includes: Obtaining a screenshot of a rectangular frame of the target outside the third tracking track corresponding to each of the third tracking tracks, and determining first feature information corresponding to each of the third tracking tracks according to the screenshot of the rectangular frame of the target outside the third tracking track corresponding to each of the third tracking tracks; Extracting the second feature information corresponding to each of the fourth high-score bounding boxes, and determining all the fourth tracking tracks and their corresponding fifth high-score bounding boxes from all the third tracking tracks and all the fourth high-score bounding boxes according to the first feature information corresponding to all the third tracking tracks and the second feature information corresponding to all the fourth high-score bounding boxes; According to all the fifth high-scoring bounding boxes, determining all the sixth high-scoring bounding boxes from all the fourth high-scoring bounding boxes, and establishing new tracking tracks for all the sixth high-scoring bounding boxes; According to the matching relationship between all the fourth tracking trajectories and their corresponding fifth high-score bounding boxes, and the matching relationship between all the sixth high-score bounding boxes and their corresponding new tracking trajectories, the target tracking result corresponding to the third target tracking operation of the current video frame is updated as the basic target tracking result corresponding to the fourth target tracking operation of the current video frame.

6. The target tracking method based on Byte Track according to claim 5, characterized in that: The determining, based on the first feature information corresponding to all the third tracking tracks and the second feature information corresponding to all the fourth high-score bounding boxes, from all the third tracking tracks and all the fourth high-score bounding boxes, all the fourth tracking tracks and their corresponding fifth high-score bounding boxes comprises: Calculating a Euclidean distance parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes according to the first feature information corresponding to all of the third tracking trajectories and the second feature information corresponding to all of the fourth high-score bounding boxes; Calculating a cosine similarity parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes according to the first feature information corresponding to all of the third tracking trajectories and the second feature information corresponding to all of the fourth high-score bounding boxes; According to the Euclidean distance parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes, and the cosine similarity parameter between each of the third tracking trajectories and each of the fourth high-score bounding boxes, all the fourth tracking trajectories whose Euclidean distance parameter is less than a preset Euclidean distance parameter threshold and whose cosine similarity parameter is less than a preset cosine similarity parameter threshold and their corresponding fifth high-score bounding boxes are determined from all the third tracking trajectories and all the fourth high-score bounding boxes.

7. The target tracking method based on Byte Track according to claim 6, characterized in that: The basic target tracking result of the current video frame includes a matching relationship between each basic tracking track and its corresponding basic bounding box, and the matching relationship between all the basic tracking tracks and their corresponding basic bounding boxes includes a matching relationship between the first target tracking track and its corresponding first high-score bounding box, a matching relationship between the second target tracking track and its corresponding target low-score bounding box, a matching relationship between the third target tracking track and its corresponding third high-score bounding box, a matching relationship between all the fourth tracking tracks and their corresponding fifth high-score bounding boxes, and a matching relationship between all the sixth high-score bounding boxes and their corresponding new tracking tracks; The step of performing trajectory overlap analysis on the basic target tracking result according to the basic target tracking result of the current video frame to obtain the final target tracking result includes: According to the matching relationship between all the basic tracking tracks and their corresponding basic bounding boxes, all overlapping tracking tracks whose corresponding target external rectangular box screenshots have target overlap are determined from all the basic tracking tracks, and non-overlapping external rectangular box screenshots corresponding to each overlapping tracking track are obtained in the previous video frame; According to the target external rectangular frame screenshots and the corresponding non-overlapping external rectangular frame screenshots corresponding to all the overlapping tracking trajectories, calculating the similarity between the target external rectangular frame screenshot and the corresponding non-overlapping external rectangular frame screenshot corresponding to each overlapping tracking trajectory; According to the similarity between the target external rectangular frame screenshots corresponding to all the overlapping tracking trajectories and the corresponding non-overlapping external rectangular frame screenshots, determining the tracking trajectory to be adjusted whose similarity is less than or equal to a preset similarity threshold from all the overlapping tracking trajectories, and adjusting the basic bounding box corresponding to the tracking trajectory to be adjusted to obtain the adjusted basic bounding box corresponding to the tracking trajectory to be adjusted; Based on the adjusted basic bounding box corresponding to the tracking trajectory to be adjusted, the basic target tracking result is updated to obtain a final target tracking result.

8. A target tracking device based on Byte Track, characterized in that: The device comprises: A determination module, configured to determine, for a current video frame to be tracked, all high-score bounding boxes and all low-score bounding boxes from all the bounding boxes according to the scores of all the bounding boxes in the current video frame; all the high-score bounding boxes are bounding boxes whose scores are greater than or equal to a preset score threshold, and all the low-score bounding boxes are bounding boxes whose scores are less than the preset score threshold; and determine all first tracking tracks from all tracking tracks in the previous video frames according to the previous video frames corresponding to the current video frame; A target tracking module is used to perform multiple target tracking operations on the current video frame based on all the first tracking trajectories, all the high-score bounding boxes, and all the low-score bounding boxes to obtain a basic target tracking result of the current video frame; all the tracking trajectories at least include active tracking trajectories that have been successfully tracked for two or more frames in the previous video frames, and all the first tracking trajectories include trajectories that have been successfully tracked in a previous video frame corresponding to the current video frame and trajectories that have lost tracking in the previous n video frames corresponding to the current video frame, where n is a preset positive integer; The analysis module is used to perform trajectory overlap analysis on the basic target tracking result according to the basic target tracking result of the current video frame to obtain a final target tracking result.

9. A target tracking device based on Byte Track, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the target tracking method based on Byte Track according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the target tracking method based on Byte Track according to any one of claims 1 to 7.

Images