Occlusion-aware multi-object tracking

By employing an unsupervised re-identification and occlusion estimation subsystem, the problems of occlusion and vanishing objects in multi-object tracking systems are solved, enabling efficient and low-cost object tracking and improving the system's accuracy and scalability.

CN117425916BActive Publication Date: 2026-11-17MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202280039430.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-06-04
Filing Date
2022-05-11
Publication Date
2026-11-17
Estimated Expiration
2042-05-11

AI Technical Summary

Technical Problem

Existing multi-object tracking systems perform poorly in the face of object occlusion, disappearance and high computational costs, especially in the absence of supervised labeled data, making it difficult to effectively track objects.

Method used

An unsupervised relabeling subsystem and an occlusion estimation subsystem are adopted. The relabeled object is trained by a loss function based on unsupervised matching, and the occlusion estimation subsystem is combined to predict the occlusion position, thereby reducing the dependence on supervised labels and improving tracking performance.

Benefits of technology

It enables efficient object tracking in the absence of supervised labeled data, reduces computational costs, and improves the accuracy and scalability of multi-object tracking systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117425916B_ABST
    Figure CN117425916B_ABST
Patent Text Reader

Abstract

A system for tracking a target object across multiple image frames. The system includes a logic machine and a storage machine. The storage machine stores instructions executable by the logic machine to compute a trajectory for the target object on one or more previous frames preceding the target frame. In response to an evaluation that no target object is detected in the target frame, the instructions are executable to predict an estimated region for the target object based on the trajectory, predict an occlusion center based on a set of candidate occlusion locations of other objects within a threshold distance of the estimated region, each of which overlaps with the estimated region, and automatically estimate a bounding box for the target object in the target frame based on the occlusion center.
Need to check novelty before this filing date? Find Prior Art

Description

Background Technology

[0001] Computer vision systems can track multiple objects to evaluate a scene. For example, tracking multiple objects can be a key component of tasks including autonomous driving, robot navigation, and video analytics. Summary of the Invention

[0002] This summary is provided to introduce some concepts in a simplified form, which will be further described below in the detailed description. This summary is neither intended to identify key or essential features of the claimed subject matter nor to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to the implementation of solutions to any or all the shortcomings mentioned in any part of this disclosure.

[0003] A system for tracking a target object across multiple image frames includes a logic machine and a memory. The memory machine stores instructions executable by the logic machine to compute a trajectory for the target object on one or more previous frames preceding the target frame. In response to an evaluation that no target object is detected in the target frame, the instructions are executable to predict an estimated region for the target object based on the trajectory, predict an occlusion center based on a set of candidate occlusion locations of other objects within a threshold distance of the estimated region, each of which overlaps with the estimated region, and automatically estimate a bounding box for the target object in the target frame based on the occlusion center. Attached Figure Description

[0004] Figure 1 A multi-object tracking system is shown.

[0005] Figures 2A-2B A method for training a multi-object tracking system is shown.

[0006] Figures 3A-3D A method for detecting occluded objects is shown.

[0007] Figure 4 A state machine for detecting occluded objects is shown.

[0008] Figure 5 An exemplary computing system is shown. Detailed Implementation

[0009] Multi-object tracking (MOT) systems can be used to detect, track, and / or identify objects in image data (e.g., still image data, video data, or any other spatial, color, brightness, and / or other optical data obtained using any suitable camera and / or sensor). Typically, MOT systems combine artificial intelligence (AI), machine learning (ML), computer vision, and / or other computer subsystems configured to perform detection, tracking, and / or identification based on processing image data.

[0010] Typically, an MOT system can utilize various features of the input image (e.g., pixel features, high-level image features, and / or semantic features determined by AI and / or ML subsystems) to evaluate what objects are present in each frame and track one or more of the object's identifier, location, and / or trajectory. In addition to object detection, existing and / or future detection subsystems can be configured to output additional information about the detected objects, such as object location and extent (e.g., bounding boxes), object classification (e.g., people vs. animals vs. vehicles, or any other suitable classification), object features (e.g., semantic features, visual features), and object identification (e.g., distinguishing a specific person, distinguishing a specific vehicle). In some examples, an object detector subsystem can be used to track objects based on detecting objects in image frames (e.g., object detectors based on convolutional neural networks, multilayer neural networks, or any other suitable AI, ML, computer vision, or other object detector models).

[0011] Even when the object detection subsystem successfully detects a target, the MOT system typically cannot track objects frame by frame. The object detection subsystem may not always be able to detect objects in every frame. For example, due to adverse conditions affecting object visibility (e.g., strong light, insufficient light), the object detection subsystem may fail to detect objects in one or more frames. In other examples, objects may be posed, deformed, or otherwise configured such that the object detection subsystem fails to detect the object, or detects the object but the generated object features may be useless for effectively tracking objects visible in another frame (e.g., the object detection subsystem may fail to recognize that a car viewed from the front in one frame is the same car viewed from the side in another frame).

[0012] In some examples, an object detected in one frame may be occluded in subsequent image frames. Detecting highly occluded objects can be a challenging task for both detection and tracking. A single input frame may not provide enough information to perceive a highly occluded object. Anchor-based methods can handle occlusion in detection, including predicting multiple instances for a single proposal or detecting a pedestrian using a pair of anchors (one for the head and the other for the whole body). However, both may require carefully designed non-maximum suppression (NMS) algorithms for post-processing. In some examples, a mask guidance subsystem can be configured to force the detector to pay attention to the more visible head portion, thereby detecting the entire body of the pedestrian. Some examples could attempt to utilize an attention model during the tracking phase instead of handling occlusion during the detection phase. Given a detected object, the attention model focuses on the visible region of the object.

[0013] Another challenge in object detection is when an object disappears from a frame. In some examples, an object may be partially or completely occluded by another object in the frame. Thus, even if the object is within the scene represented in the image, there may be a limited or zero visible image features corresponding to the object. In other examples, an object may temporarily move out of the frame while remaining near the scene represented in the image, and may return to a later image frame. In these examples, the object detector subsystem cannot detect the object in the frame at all because the object does not exist in the frame. Therefore, if an object disappears from some of those frames or is not detected in those frames, a MOT system that specifically tracks objects by utilizing the object detector subsystem may be unable to track objects across multiple frames.

[0014] Therefore, occlusion due to missing objects can lead to poor tracking results. As described in this paper, a re-identification (Re-ID) subsystem can be used to re-identify missing objects. Some re-identification subsystems may require large amounts of labeled data for supervised training, which may make their implementation costly or infeasible. Furthermore, without a suitable compensation strategy, the re-identification subsystem may struggle to track highly occluded objects that were initially undetectable by the object detector.

[0015] In some examples, multi-object trackers can incorporate independent re-identification models to learn discriminative representations of objects. However, such discriminative representations can lead to high computational costs, for example, due to cropping objects from an image and independently feeding the cropped image into a pre-trained re-identification model. In tracking, it may be necessary to learn discriminative representations of objects to identify associations. This representation can be used to re-identify lost objects after a period of time. In some examples, multi-object trackers can crop image patches of detected objects, resize them, and feed the patches into separate Re-ID models. However, this process can be time-consuming because the feature representations of different objects must be computed independently. To reduce computation, in some cases, Re-ID feature computation can be shared with the backbone of the object detector subsystem by introducing an additional Re-ID branch parallel to the detection branch, e.g., in anchor-based or point-based detectors. In some cases, these costs can be mitigated by developing a model that shares re-identification feature computation with the model backbone, by introducing an additional re-identification branch parallel to the object detection branch of the model, as in anchor-based or point-based detectors.

[0016] A Re-ID subsystem can be trained to classify each identifier into a category. However, the resulting Re-ID subsystem may not be very scalable, especially when the number of identifiers is huge (e.g., the Re-ID system may need to perform hundreds of thousands of classification tasks to train the Re-ID subsystem). For example, the original FairMOT... TM Also in CrowdHuman TM They pre-train their models. However, the Re-ID subsystem is trained using pseudo-labels, that is, a unique label is assigned to each annotated box, and the Re-ID subsystem is trained in a classification manner. (CrowdHuman) TM There are approximately 339K bounding boxes, resulting in a massive number of pseudo-labels. This leads to a number of parameters in the classifier that is even greater than the total number of parameters in other subsystems (56.6M vs. 20.3M).

[0017] As the number of identifiers increases, the classifier task may require increasingly more memory. Furthermore, training the Re-ID subsystem using previous methods may require supervised training with identifier information. However, obtaining well-annotated data can be very expensive. In some examples, the Re-ID representation can be trained by leveraging the cyclic association consistency between two adjacent frames. However, other existing Re-ID systems do not handle object births and deaths. In some examples, pseudo-identifier information can be used to train the Re-ID subsystem. This pseudo-identifier information is obtained using a simple MOT tracking method that uses tracklets, where tracklets are segments being tracked (e.g., the trajectory or path followed by a moving object). Each tracklet is assigned a pseudo-identifier. However, the pseudo-identifier information is noisy because objects whose trajectories are divided into short trajectories can be assigned multiple identifiers.

[0018] However, even with such improvements, relabeling using supervised machine learning can be difficult to scale because it may be challenging to generate enough supervised labeled data. Furthermore, supervised relabeling learning uses fully connected neural network layers (e.g., sibling classification layers), the size of which is linearly proportional to the number of distinct labels in the dataset. Therefore, supervised relabeling learning can introduce costs and / or challenges during training.

[0019] To address this issue, the Re-ID subsystem learning mechanism disclosed in this paper utilizes an unsupervised matching-based loss between two adjacent frames, e.g., instead of a supervised classification loss. The matching-based loss is formulated based on the observations that: 1) objects with the same identifier in adjacent frames share a similar appearance (e.g., a particular object does not suddenly and completely change its appearance between frames), and 2) objects in different videos (or within a single frame of a video) may have different identifiers and therefore may have different appearances (this is because a particular object cannot appear in two different places simultaneously, and because different videos often have different themes). Compared to other methods, the Re-ID learning mechanism disclosed in this paper has at least three advantages: 1) it does not require any identifier annotation and is therefore more scalable; 2) it is applicable not only to video-based data but also benefits from pre-training on still image-based data; and 3) the matching-based loss is independent of the number of identifiers and can therefore be trained directly on massive amounts of video / image-based data with a large number of identifiers.

[0020] Figure 1An MOT system 100 is illustrated, configured to achieve improved performance in tracking multiple objects across multiple image frames, at least in part, by utilizing unsupervised labeling of objects detected in different image frames. The MOT system 100 includes an object detection subsystem 110, which is operated to detect objects during tracking. The MOT system 100 is able to track objects even when the object detection subsystem 110 fails to detect an object in one or more image frames (e.g., due to complete or partial occlusion or complete or partial removal from the frame). The MOT system 100 mitigates fundamental challenges in object tracking by using an unsupervised re-identification subsystem 120 to identify when the same object appears in multiple frames, and by using an occlusion estimation subsystem 140 to detect when an occluded object is present.

[0021] The relabeling subsystem 120 is configured to relabel objects without relying on label information or pseudo-label information. It can be trained in an unsupervised, matching-based manner on image-based and video-based data without requiring any label information to distinguish different objects, and it also has better scalability for datasets with large numbers of labels. Using unsupervised training data including image or video data representing scenes with multiple objects, the unsupervised relabeling subsystem 120 is trained to track objects and consider object tracking during temporary object disappearance. Since unsupervised training data does not require manual labeling, training data for the MOT system 100 can be obtained cheaply and easily by leveraging existing machine learning training datasets (e.g., those used for object detectors or other multi-object trackers).

[0022] The unsupervised Re-ID subsystem 120 utilizes an unsupervised matching-based loss between different (e.g., adjacent) frames. The unsupervised Re-ID formula is inspired by the observation that objects with the same identifier in adjacent frames share a similar appearance, and that objects in different videos / images have different identifiers. Unsupervised Re-ID training is based on supervisory signals, including: 1) a strong "single object identifier" supervisory signal indicating that objects within a frame should not match each other; and 2) a weak "object persistence" supervisory signal indicating that an object within one frame may match an object in another frame. For example, the single object identifier signal indicates that objects in different videos (or, within a frame) are assumed to have different identifiers. Because the lack of matching is independent of the number of identifiers, it has no scalability issues and can be trained directly on massive datasets.

[0023] The ability to train the re-identification subsystem 120 using unsupervised data can mitigate the potentially substantial costs of implementing the re-identification subsystem. It is believed that the performance of the unsupervised re-identification subsystem 120 can surpass that of the supervised re-identification model, thereby improving tracking performance while reducing the costs associated with acquiring training data. Compared to supervised classification-based Re-ID learning, the proposed unsupervised Re-ID subsystem does not suffer from the dimensionality explosion problem with a large number of identifiers, and therefore may be more suitable for practical large-scale applications. No additional parameters need to be introduced in the disclosed unsupervised Re-ID learning method.

[0024] Figure 2A A method 200 is illustrated, for example, using an unsupervised Re-ID subsystem to generate training data for multi-object tracking in video data. In step 202, method 200 includes: operating a machine learning system to detect a first set of target objects in a first image frame of the video. Any suitable object detection model can be used (e.g., Figure 1 The object detection subsystem 110. For example, the object detection model may have a convolutional neural network (e.g., a dynamic convolutional neural network, a convolutional neural network with attention, a convolutional variable autoencoder, a pyramidal convolutional neural network, and / or any other neural network model, such as a fully connected neural network). In some examples, the object detection model may include one or more pre-trained domain-specific models, such as a "human detector" (e.g., a convolutional neural network previously trained to detect people) or a "car detector". In some examples, the object detection model may be configured to detect one or more different types of entities (e.g., a convolutional neural network with output nodes for classifying detected entities as cars or people). In still other examples, the object detection model may include multiple different object detection models, each configured to detect the presence of one or more types of objects. More generally, the object detection model may be configured to detect objects (e.g., whether an object is definitively classified as a particular type of object) and output feature data for the detected objects, wherein the feature data includes any suitable computer-readable data that defines the detected features of the detected objects. For example, the feature data may include vector representations of the detected objects, such as intermediate values ​​computed by the object detection model (e.g., vector values ​​obtained from intermediate layers of a convolutional neural network).

[0025] In 204, method 200 includes operating a machine learning system to detect a second set of test objects in a second subsequent image frame of the video.

[0026] At 206, method 200 includes a series of steps performed on each target object in the first set of target objects. Specifically, at 208, method 200 includes comparing the object feature data of the target objects with the object feature data of each test object in the second set of test objects. Any suitable comparison can be used. As a non-limiting example, each detected object can be represented by a feature vector of a set of computed features indicating the image (e.g., image data, neural network activation data), and the object feature data can be compared by any suitable means (e.g., by cosine similarity or other geometric similarity functions, and / or by a previously trained machine learning comparison function).

[0027] Based on this comparison, in step 210, method 200 further includes selecting one or more test objects from the second group of test objects as candidate objects based on comparison values ​​of the object feature data of the target object and the object feature data of the test objects that are above a threshold. Alternatively or additionally, a single test object may be selected as a candidate object based on the comparison value between the candidate object and the target object, which is the highest comparison value among the test objects in that group.

[0028] In step 212, method 200 further includes automatically providing the same unsupervised label to both the target object and the candidate object if only one candidate object has a comparison value higher than a threshold. The unsupervised label indicates the correspondence between objects that can be used to train the re-identification subsystem. The unsupervised label can be generated without any supervision, e.g., without requiring a human annotator to identify and label the objects. The unsupervised label can be any identifier, e.g., a string, integer, or GUID. The unsupervised label can be associated with the target object and the candidate object in any suitable data representation; for example, a training file can be saved for each of the target object and the candidate object, each training file being indexed by the unsupervised label and / or including the unsupervised label as a header.

[0029] In 214, method 200 further includes automatically providing the same unsupervised label to the target object and the best candidate among the two or more candidate objects with the highest comparison value if two or more candidate objects have comparison values ​​higher than a threshold.

[0030] In 216, method 200 further includes automatically providing a placeholder unsupervised label to the target object if no candidate object has a comparison value higher than a threshold, for example, a label indicating that no target object was found. By using placeholder labels for any candidate object that does not have a match higher than the threshold, false matches can be avoided when candidate objects lack true matches.

[0031] In some examples, the similarity matrix tracks comparison values ​​of object feature data. In some examples, the similarity matrix includes rows for each object in a union of a first set of target objects and a second set of test objects, and columns for each object in the union, with each matrix element representing a comparison value between the pair of objects obtained from the union. In some examples, the similarity matrix includes placeholder columns for unsupervised labels.

[0032] Figure 2B It shows the result of frame I t-1 and I t An exemplary description of a similarity matrix 2000 (e.g., representing a similarity matrix S) is provided, with each object corresponding to a row 2006, each corresponding to a column 2004, and an additional placeholder column 2002 indexed. The matrix can be broadly divided into four quadrants 2010, 2020, 2030, 2040, and a placeholder column 2002 (outside the four quadrants). The top-left quadrant 2010 and bottom-right quadrant 2040 are controlled by strong single object identification supervision signals. The top-right quadrant 2020 and bottom-left quadrant 2030 are controlled by weak object persistence supervision signals. Elements on the diagonal (marked with “o”) are ignored because these elements correspond to objects identified as themselves (e.g., as identical objects in the same frame). Objects match other objects within quadrants 2020 and 2030, as indicated by checkmarks. In some cases, objects match cells in the placeholder column, as indicated by “x”.

[0033] In some examples, at 218, method 200 further includes training a machine learning system to track objects based on unsupervised labels and one or more automatic supervision signals. In some examples, the one or more automatic supervision signals further include a strong single object identifier signal indicating a constraint that a first target object from a first image frame should not have a comparison value higher than a threshold relative to any other object from the first image frame. In some examples, the one or more automatic supervision signals include a weak object persistence signal indicating a constraint that each target object in the first image frame should have a comparison value higher than a threshold relative to at most one test object in a second image frame. In some examples, the weak object persistence signal further indicates that if a target object from the first group has a comparison value higher than a threshold relative to a test object in the second group, then the test object should also, in turn, have a comparison value higher than a threshold relative to the target object. In some examples, object feature data and comparison values ​​are generated by one or more differentiable functions. Therefore, training the machine learning system to track objects includes configuring one or more differentiable functions based on unsupervised labels and one or more automatic supervision signals. In some examples, one or more differentiable functions include a neural network configured to compute object feature data based on cropped image patches.

[0034] In some examples, one or more differentiable functions include an object detector subsystem of a machine learning system, and the object feature data is an intermediate representation of the objects detected by the object detector subsystem.

[0035] Back Figure 1 In a non-limiting example, the re-identification system 120 can be trained using video-based data as described above. For example, suppose... It is from the t-th frame of a video, and It is frame I t The truth bounding box of object i in the table. Where D is the dimension of the appearance feature vector. The unsupervised Re-ID module learning mechanism can calculate... As long as it is differentiable. Possible solutions include cropping image patches based on a given bounding box and feeding the cropped image patches into an additional Re-ID network to extract appearance features based on regions of interest (ROIs) by sharing the same backbone with the object detector network, as well as extracting appearance features based on center points.

[0036] Given two adjacent frames I t-1 I t Let i ∈ {0, ..., N} t-1 -1, ..., N t-1 +N t -1} is the index of all objects in the two frames, where N t It is frame I t The number of objects in the dataset. To learn a Re-ID representation with supervised signals, a similarity matrix can be used to measure the similarity between each pair of objects, where...

[0037] The values ​​on the diagonal of S are set to negative infinity to avoid assigning an object to itself. Typically, if objects i and j have the same identity, S... i,j >0, otherwise, S i,j <0. Can be achieved via the row-by-row softmax function. Obtain the allocation matrix Where T is the softmax temperature. Since objects in a frame have different identifiers, this can be achieved by using an intra-frame loss function corresponding to the individual object identifier signal. Supervised training is used to configure the values ​​in the upper left and lower right portions of M. The object persistence signal is defined based on the ideal case where all objects appear in both frames. In this case, frame I... t-1 All objects should be associated with the next frame in a one-to-one manner. tObject matching in M. Then, for each row in M, by using the inter-marginal marginal loss. Each object is encouraged to match with another object with high confidence, where The definition of marginal loss between margins is similar to the popular triple loss, such as the maximum matching probability. The marginal value is greater than the second-largest value by a predefined margin m (e.g., 0.5 or any other suitable predefined marginal value). In addition to the marginal loss, M adds another cyclic constraint, meaning that forward and backward assignments should be consistent with each other. If frame I... t-1 object i and frame I t-1 If object j matches, then the cyclic constraint loss can be defined:

[0038]

[0039] Since two adjacent frames in video-based data often share some objects with the same identifier, such two adjacent frames can be used as positive samples for Re-ID training. Unsupervised Re-ID learning on positive samples has a total loss.

[0040] The proposed Re-ID representation introduces placeholders to handle object births and deaths. In some examples, an object in one frame may disappear in the next (object death), or an object invisible in one frame may appear in the next (object birth). To address this, a new similarity matrix is ​​created by padding the placeholder column into S. All values ​​in the placeholder are the same value p. Let... This is achieved by applying a row-wise softmax function to the assignment matrix S'. Then, in the above equation, M can be replaced by M'. By using p, it is encouraged to learn that the distance between objects that disappear or reappear is less than p.

[0041] The introduction of the placeholder p is to handle the birth and death of objects, that is, objects that appear in I<t-1> but disappear in I<t>, as well as objects that newly appear in I<t>, should be assigned to the placeholder. Let Si,j be the cosine similarity between the Re-ID features of object i and object j. For perceptible matching, if object i and object j have the same identity, Si,j should be greater than p, otherwise Si,j < p. If two objects have the same identity, the cosine similarity between their Re-ID features should be positive, otherwise it should be negative, so p can optionally be set to 0. However, it is believed that in the early training stage, the variance of the values in S may be very small (about 0.015), and the cosine similarity between any pair of objects is about 0.75. Therefore, if p = 0, it may be difficult for the model to properly handle the birth and death of objects at the beginning. Thus, by default, p can be better set as the dynamic average of the values in S (excluding the diagonal values). Interestingly, with this training strategy, the average value of the values in S is about 0 after convergence. When two different static images are sampled as a negative sample pair, the placeholder p will be set to 0, because all objects in these two images have different identities.

[0042] The Re-ID subsystem can also be trained on image-based data, for example, by using two augmentations of one image as adjacent frames. However, in this case, the loss function based on positive samples may not perform well, because the objects in the two augmentations usually may have very similar appearances, which does not provide strong incentive to learn discriminative Re-ID features. Since objects in two different static images are usually different entities, a loss function based on negative samples is introduced by treating two different static images as a negative sample pair: In this formula, the additional placeholder p from the above is utilized. The cosine distance between objects in the negative pair is encouraged to be less than p, which means all objects should be assigned to the placeholder. Therefore, the total unsupervised Re-ID learning loss for image-based data is:

[0043] where N pos , N neg are the numbers of positive and negative samples in the batch. As a non-limiting example, the number of positive samples may be 4 times that of negative samples.

[0044] In some examples, while tracking an object, the machine learning system is further trained to automatically detect when the target object is occluded by another object in a second frame and to estimate the bounding box of the occluded target object in the second frame. For example, this detection can be via an occlusion estimation subsystem. In some examples, detecting when a target object is occluded by another object in a second frame includes computing a trajectory for the target object based at least on a first frame and / or zero to multiple previous frames; and evaluating whether the target object was detected in the second frame. In some examples, estimating the bounding box of the target object in the second frame includes predicting an estimated region for the target object based on the trajectory. Therefore, an occlusion center can be predicted based on a set of candidate occlusion locations of other objects within a threshold distance of the estimated region, each of which overlaps with the estimated region. The bounding box can be estimated based on the predicted occlusion center.

[0045] Figure 3A A method 300 for tracking a target object across multiple image frames is illustrated. In 302, method 300 includes calculating a trajectory 3021 for the target object on one or more previous frames that occur before the target frame. For example, Figure 3B A previous frame of a view with three entities is shown: entity 3010 (with trajectory 3011), entity 3020 (with trajectory 3021), and entity 3030 (with trajectory 3031). The trajectory of an object in the frame can be calculated in any suitable manner, such as based on one or more displacement vectors, each indicating the direction and magnitude of displacement of the object between two previous frames. For example, the trajectory of the object can be predicted as a displacement based on the displacement vectors of two previous frames. Alternatively or additionally, a previously trained machine learning model can be used to predict the trajectory of the object, which is configured to predict the trajectory based on the previous motion indicated in the displacement vectors.

[0046] In 304, method 300 includes evaluating whether a target object was detected in the target frame. For example, in Figure 3C No source was detected in the middle. Figure 3B Entity 3020.

[0047] In 306, method 300 includes predicting an estimated region 3222 for a target object based on a trajectory. The estimated region is predicted by shifting the bounding box of the target object in a previous frame to a new bounding box in the target frame having the size of the target object and the position determined by the calculated trajectory, according to the calculated trajectory.

[0048] In 308, method 300 includes predicting an occlusion center 3024 based on a set of candidate occlusion locations of a set of other objects within a threshold distance for the estimated region, each of the set of candidate occlusion locations overlapping the estimated region. Figure 3CThe estimated region 3022 of entity 3020 is shown (which is in Figure 3C (Not detected in the image), via motion from previous frames (e.g., the trajectory of entity 3020 in frame 3C placing its possible location in frame 3C before the estimated region 3022). For example, occlusion center 3024 can be predicted based on this set of candidate occlusion locations by finding the center of the intersection region of bounding boxes 3012 and 3022; occlusion center 3026 can be predicted in a similar manner based on finding the center of the intersection region of bounding boxes 3022 and 3032. The candidate occlusion location is selected as the location of occlusion center 3024 and occlusion center 3026 that is closer to the center point consistent with the predicted trajectory 3021. In this example, candidate occlusion location 3026 is selected as the occlusion center point (e.g., as defined below). ).

[0049] In 310, method 300 includes automatically estimating the bounding box of the target object in the target frame based on the occlusion center and the size and / or bounding box of the target object in the previous frame. For example, Figure 3D The estimated bounding box 3040 is shown (e.g., the bounding box defined below). Compared to bounding box 3022, according to this disclosure, a state machine is used to estimate bounding box 3040 based on a selected occlusion center, thereby locating the target object in the target frame with higher accuracy compared to the estimated region based solely on the trajectory.

[0050] In some examples, method 300 further includes computing a heatmap of the occlusion center by operating one or more convolutional neural network units. In some examples, estimating the bounding box includes operating a state machine configured to record tracking states of multiple objects and state information of the tracked motion, and estimating the bounding box based on such recorded state information. In some examples, estimating the bounding box includes operating a Kalman filter. In some examples, estimating the bounding box includes training a machine learning system to automatically estimate the bounding box via supervised learning using image data and ground-value bounding boxes.

[0051] return Figure 1 The occlusion estimation subsystem 140 is configured to identify and track occluded objects by estimating the occlusion location and the motion of existing tracking segments, for example, using method 300. While the Re-ID subsystem can help re-identify re-emerging objects after a short period of disappearance, it may be unable to track occluded objects if they are not detected by the detector. To mitigate the problems caused by missing detections, the occlusion estimation subsystem 140 is configured to predict whether any occlusion has occurred and locate the missing objects by considering temporal motion information and the tracking state of existing tracking segments.

[0052] The occlusion estimation subsystem 140 may include an attention subsystem that focuses on the occluded (or overlapping) region where occlusion occurs, rather than the visible region of the object.

[0053] The occlusion estimation subsystem 140 can perform motion estimation on objects in an existing tracking segment and predict whether occlusion will occur between two objects in the future. The occlusion estimation subsystem 140 can further estimate an occlusion map showing all possible occlusion locations. By further combining the state of the existing tracking segment with these two components, a lost object re-finding mechanism can be configured to locate occluded objects.

[0054] Occlusion estimation mechanisms can be based on keypoint-based detection, which detects the location of occlusion and discovers objects missing from the detector by combining temporal information from existing tracking segments. The occlusion estimation subsystem can mitigate tracking gaps caused by missing detection. The occlusion estimation subsystem is configured to locate occluded objects by simultaneously estimating an occlusion map showing all possible occlusion locations and utilizing motion estimation information.

[0055] Occlusion locations can be treated as keypoints and detected through keypoint estimation. Unlike the Re-ID subsystem, the occlusion estimation module can be designed in a supervised manner, where occlusion annotations are generated based on the bounding boxes of the target objects, for example, from existing tracking datasets such as MOT. TM .

[0056] First, occlusion can be defined based on the bounding box coordinates of two objects i and j within an image, and their overlapping region is defined as... in If the corresponding overlapping region occupies most of an object, then occlusion between objects i and j is detected: Where A is a function for calculating the area of ​​the bounding box, and τ is a hyperparameter that can be set to any suitable value (e.g., the default is 0.7). To re-locate the occluded object, the occlusion position of two overlapping objects is defined as the overlap region o. ij The center point. The ground truth occlusion map Y is rendered by a 2D Gaussian kernel function based on all valid occlusions (e.g., H(o ij Defined in ), making H(o ij ) = 1, where in It's an obstruction. ij The center point, and This is the standard deviation of the Gaussian kernel relative to the occlusion size. In some examples, an additional convolutional neural network (CNN) head can be introduced to obtain a predicted occlusion center heatmap. In some examples, the CNN head can be parallel to the detector head and share the same backbone network. R is the downsampling factor of the backbone network. Let represent the probability of being located at the center of occlusion in (x, y), and be derived from... Supervision, where L is any suitable loss function, for example, a focal loss function with two hyperparameters α and β, which can take any suitable values, such as 2 and 4, respectively, such that Considering that R may be greater than 1, additional CNN heads can be added to generate offset heatmaps. This can help compensate for quantization errors when generating the occlusion center heatmap Y. The L1 loss function can be used to regress the center offset. Offset supervision can only be specified at the center location. Total occlusion estimation loss can be defined.

[0057] While the aforementioned occlusion estimation subsystem can predict all possible occlusion locations, i.e., the center point of the overlapping region, it is not equivalent to detecting missing objects near these locations. For example, some occluded objects may have already been detected by the detector. On the other hand, even if a missing object is detected, the estimated occlusion location alone may not be sufficient to infer the size and precise location of the occluded object. To determine whether a missing object has been detected, the state machine is configured to record the tracking state of existing tracking segments and match the latest detection results against these tracking segments. If some tracking segments cannot find a matching detection box, then a missing object may have been detected. Figure 4 The specific tracking logic is explained in detail. Figure 4 The matching strategy between the tracking fragment and the detection box in step 2 can follow the original strategy used in the target MOT system or any suitable strategy based on union-intersection (IOU) and Re-ID feature matching. Once a missing object is detected, it can be found by using the motion information of the corresponding tracking fragment, which can be estimated using a Kalman filter. As an example, to find I... t The missing object i, I t-1 The bounding box in the image is represented as Predicting I via Kalman filter t The position in the middle, and is represented as Then one can consider proximity. The estimated occlusion center is used to search for detected objects that may occlude i. For objects that may occlude i... Each overlapping box Target occlusion position, according to The distance between and located All predicted occlusion centers within To calculate the target occlusion position Select the best matching pair as if The default value is (τ) o =0.7), the estimated bounding box of missing object I. Based on Know To estimate, such as and in

[0058] In some examples, the occlusion center and occlusion offset can be computed in the occlusion estimation subsystem, for example, via one or more convolutional layers (e.g., a 3x3 convolutional layer configured to output a 256-channel feature map, and / or via a 1x1 convolutional layer configured to compute a task-specific heatmap). In some examples, the one or more convolutional layers can be connected via ReLU, sigmoid, a labeling function, or any other suitable activation function. In some examples, the occlusion center branch can be configured to compute the output heatmap and / or occlusion offset heatmap using ReLU, sigmoid, or a labeling function.

[0059] The unsupervised Re-ID learning subsystem and occlusion detection subsystem disclosed in this paper can be applied to any suitable MOT method. In some examples, in addition to the detection loss function in the MOT framework, an occlusion loss function computed by the occlusion estimator, for example, can be used as a weighted sum. It is believed that MOT performance is further improved due to the occlusion estimation subsystem 140 (e.g., beneficial for object tracking even when the object is partially or completely occluded by one or more frames). The methods described in this paper can be used with any suitable existing technology and / or future MOT framework, for example, in frameworks using the detection-tracking paradigm.

[0060] It is believed that the method described in this paper can achieve excellent performance in MOT scenarios. For example, the method described in this paper has already been used in ablation research with MOT Halllenge. TM The dataset was evaluated, assessing performance improvements from various techniques described in this paper, involving one or more standard metrics of MOT performance, such as MOT accuracy (MOTA), MOT precision (MOTP), ID F1 score (IDF1), main tracked object (MT), main lost object (ML), false positives (FP), false negatives (FN), number of identifier switching (IDS), and number of fragments (Frag). It is believed that unsupervised re-identification may achieve similar or better performance compared to supervised re-identification. Furthermore, it is believed that adding image-based pre-training and / or adding occlusion estimation leads to further improvements in MOT performance.

[0061] In some examples, there are supervised relabeling learning systems (e.g., FairMOT). TM The re-identification loss function can be replaced by unsupervised re-identification learning, for example, by directly replacing the re-identification loss function with the unsupervised re-identification learning loss function as defined in this disclosure. When the methods disclosed herein are integrated into another MOT framework, the detection subsystem of that framework can remain unchanged, and the occlusion estimation subsystem can be added in parallel with the detection subsystem. The Re-ID subsystem is trained using Re-ID features from two frames. For example, all objects of interest in each frame can be detected using an object detector, and then subsequent trackers can be used to associate objects across different frames (e.g., online or offline). Online methods process video sequences frame by frame and track objects by using only information from the current frame. In contrast, offline methods process video sequences in batches and utilize information from the entire video. Individual or joint modeling methods can be used. In individual modeling methods, the trackers are trained independently, and it is assumed that the detection results are available in advance. In joint modeling methods, the trackers are jointly trained with the detector by sharing the same feature extractor backbone. Therefore, in some examples, joint modeling methods may be computationally more efficient than individual modeling methods. The newly proposed Re-ID subsystem and occlusion estimation subsystem can both be integrated into the online detection-tracking MOT system and learned jointly with the detector.

[0062] The method described in this paper can be used with any MOT system that can be configured to use a differentiable re-identification feature learning subsystem, which can be replaced by the unsupervised re-identification subsystem of this disclosure to achieve large-scale unsupervised training. Furthermore, the method described in this paper can be used in any MOT system that utilizes convolutional neural networks for object detection. When the method described in this paper is applied to existing methods, the existing object detection subsystem can remain unchanged, and the re-identification and / or occlusion estimation subsystem described in this paper can be added in parallel to the existing detection subsystem. For example, the method described in this paper can be integrated into FairMOT. TM CenterTrack TM Any other method that utilizes a keypoint-based detector (such as CenterNet) TM In some examples, the subsystems described herein can be optimized via an optimization subsystem, for example, via Adam. TM Optimizer. In some examples, the subsystems described in this paper can be pre-trained on supplementary datasets. For example, the unsupervised re-identification subsystem can be pre-trained on image-based data as described above, for example, using CrowdHuman. TMDataset. Unsupervised relabeling and occlusion estimation can be used alone or simultaneously. For example, both unsupervised relabeling and occlusion estimation can be used in FairMOT. TM In this context, this is a re-identification-based approach. Conversely, even CenterTrack... TM Not based on re-identification, occlusion estimation can also be used with CenterTrack. TM Use them together.

[0063] In some implementations, the methods and processes described herein can be attached to a computing system of one or more computing devices. Specifically, these methods and processes can be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0064] Figure 5 A non-limiting embodiment of a computing system 500 that can implement one or more of the methods and processes described above is illustrated schematically. The computing system 500 is shown in a simplified form. The computing system 500 may take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones), and / or other computing devices.

[0065] The computing system 500 includes a logic machine 502 and a storage machine 504. The computing system 500 may optionally include a display subsystem 506, an input subsystem 508, a communication subsystem 510, and / or... Figure 5 Other components not shown.

[0066] Logic machine 502 includes one or more physical devices configured to execute instructions. For example, a logic machine may be configured to execute instructions that are part of one or more application programs, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform tasks, implement data types, transition the state of one or more components, achieve technical effects, or otherwise achieve desired results.

[0067] A logical machine may include one or more processors configured to execute software instructions. Additionally or alternatively, a logical machine may include one or more hardware or firmware logical machines configured to execute hardware or firmware instructions. The processor of the logical machine may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. Components of the logical machine may optionally be distributed across two or more independent devices that can be remotely located and / or configured for coordinated processing. Aspects of the logical machine may be virtualized and executed by remotely accessible network computing devices configured in a cloud computing configuration.

[0068] Storage machine 504 includes one or more physical devices configured to store instructions executable by a logical machine to implement the methods and processes described herein. When such methods and processes are implemented, the state of storage machine 504 can be transformed—for example, storing different data.

[0069] Storage machine 504 may include removable and / or built-in devices. Storage machine 504 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and / or magnetic memory (e.g., hard disk drive, floppy disk drive, tape drive, MRAM, etc.), etc. Storage machine 504 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location-addressable, file-addressable, and / or content-addressable devices.

[0070] It should be understood that storage machine 504 includes one or more physical devices. However, alternatively, aspects of the instructions described herein may be propagated via a communication medium (e.g., electromagnetic signals, optical signals, etc.) and not maintained by a physical device for a finite duration.

[0071] The aspects of logic machine 502 and storage machine 504 can be integrated together into one or more hardware logic components. For example, such hardware logic components may include field-programmable gate arrays (FPGAs), application-specific integrated circuits (PASICs / ASICs), application-specific standard products (PSSPs / ASSPs), systems-on-a-chip (SoCs), and complex programmable logic devices (CPLDs). In some implementations, computing system 500 may include two or more independently operable computing devices; for example, a dedicated back-end training device and a dedicated runtime device that utilizes a model trained using the dedicated back-end training device. In some implementations, training and runtime may be performed by the same computing device. The dedicated back-end training device and / or runtime device may include a training storage device that holds executable instructions for training the machine learning system described herein.

[0072] The terms "module," "program," and "engine" can be used to describe aspects of a computing system 500 implemented to perform specific functions. In some cases, a module, program, or engine can be instantiated via a logical machine 502 that executes instructions stored in a storage machine 504. It should be understood that different modules, programs, and / or engines can be instantiated from the same applications, services, code blocks, objects, libraries, routines, APIs, functions, etc. Similarly, the same module, program, and / or engine can be instantiated from different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module," "program," and "engine" can include single or multiple sets of executable files, data files, libraries, drivers, scripts, database records, etc.

[0073] It should be understood that, as used herein, a "service" is an application that can execute across multiple user sessions. A service can be used for one or more system components, programs, and / or other services. In some implementations, a service can run on one or more server computing devices.

[0074] When included, the display subsystem 506 can be used to present a visual representation of the data held by the storage machine 504. This visual representation may take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data held by the storage machine and thus transform the state of the storage machine, the state of the display subsystem 506 can also be transformed to visually represent changes in the underlying data. The display subsystem 506 may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with the logic machine 502 and / or the storage machine 504 in a shared enclosure, or such display devices may be peripheral display devices.

[0075] When included, the input subsystem 508 may include or interface with one or more user input devices, such as a keyboard, mouse, touchscreen, or game controller. In some embodiments, the input subsystem may include or interface with a selected Natural User Input (NUI) component. Such a component may be integrated or peripheral, and the translation and / or processing of input actions may be handled on-board or off-board. Example NUI components may include a microphone for speech and / or voice recognition; an infrared, color, stereo, and / or depth camera for machine vision and / or gesture recognition; a head tracker, eye tracker, accelerometer, and / or gyroscope for motion detection and / or intent recognition; and an electric field sensing component for assessing brain activity.

[0076] When included, the communication subsystem 510 can be configured to communicatively couple the computing system 500 to one or more other computing devices. The communication subsystem 500 may include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem can be configured to communicate via a wireless telephone network, or a wired or wireless local area network or wide area network. In some embodiments, the communication subsystem may allow the computing system 500 to send and / or receive messages from other devices via a network such as the Internet.

[0077] In one example, a system for tracking a target object across multiple image frames includes a logic machine and a storage machine. In this example, or any example, the storage machine stores executable instructions to: compute a trajectory for the target object on one or more previous frames preceding the target frame; in response to an evaluation that no target object is detected in the target frame: predict an estimated region for the target object based on the trajectory; predict an occlusion center based on a set of candidate occlusion locations of a set of other objects within a threshold distance of the estimated region, each location of the set of candidate occlusion locations overlapping the estimated region; and automatically estimate a bounding box for the target object in the target frame based on the occlusion center. In this example, or any example, the instructions are further executable to compute a heatmap of the occlusion center by operating one or more convolutional neural network units. In this example, or any example, estimating the bounding box includes operating a state machine configured to record state information including the tracking states of multiple objects and the tracked motion, and estimating the bounding box based on this recorded state information. In this example, or any example, estimating the bounding box includes operating a Kalman filter. In this example, or any example, the instructions are further executable to train a machine learning system to automatically estimate bounding boxes via supervised learning using image data and ground truth bounding boxes. In this example or any other example, the instructions held on the training storage device are executable to train a machine learning system to track objects based on one or more unsupervised labels, which are based on comparison values ​​indicating the similarity between object feature data for an object in a first video frame and object feature data for an object in a second video frame. In this example or any other example, one or more comparison values ​​of the object feature data are tracked in a similarity matrix comprising: rows for each object in a union of both a first set of target objects and a second set of test objects, columns for each object in the union, and placeholder columns for unsupervised labels, wherein each matrix element represents a comparison value between a pair of objects obtained from the union. In this example or any other example, the object feature data and comparison values ​​are generated by one or more differentiable functions, and training the machine learning system to track objects includes configuring one or more differentiable functions based on unsupervised labels and one or more automatic supervision signals.

[0078] In one example, a method for tracking a target object across multiple image frames includes: computing a trajectory for the target object on one or more previous frames preceding the target frame; evaluating if no target object is detected in the target frame; predicting an estimated region for the target object based on the trajectory; predicting an occlusion center based on a set of candidate occlusion locations of a set of other objects within a threshold distance of the estimated region, each location of the set of candidate occlusion locations overlapping the estimated region; and automatically estimating a bounding box for the target object in the target frame based on the occlusion center. In this example or any other example, the method further includes computing a heatmap of the occlusion center by operating one or more convolutional neural network units. In this example or any other example, estimating the bounding box includes operating a state machine configured to record state information including the tracking states of multiple objects and the tracked motion, and estimating the bounding box based on this recorded state information. In this example or any other example, estimating the bounding box includes operating a Kalman filter. In this example or any other example, the method includes training a machine learning system to automatically estimate bounding boxes via supervised learning using image data and ground-value bounding boxes.

[0079] In one example, a method for generating training data for multi-object tracking in video data includes: operating a machine learning system to detect a first group of target objects in a first image frame of the video; operating the machine learning system to detect a second group of test objects in a second subsequent image frame of the video; for each target object in the first group of target objects: comparing object feature data of the target object with object feature data of each test object in the second group of test objects; selecting one or more test objects from the second group of test objects as candidate objects based on comparison values ​​of the object feature data of the target objects and the object feature data of the test objects that are above a threshold; automatically providing the same unsupervised label to the target object and the candidate objects if only one candidate object has a comparison value above the threshold; automatically providing the same unsupervised label to the target object and the best candidate object with the highest comparison value among the two or more candidate objects if two or more candidate objects have comparison values ​​above the threshold; and automatically providing a placeholder unsupervised label to the target object if no candidate object has a comparison value above the threshold. In this example or any other example, comparison values ​​of object feature data are tracked in a similarity matrix comprising: rows for each object in a union of a first set of target objects and a second set of test objects, columns for each object in the union, and placeholder columns for unsupervised labels, wherein each matrix element represents a comparison value between a pair of objects obtained from the union. In this example or any other example, the method further includes training a machine learning system to track objects based on unsupervised labels and one or more automatic supervision signals. In this example or any other example, the one or more automatic supervision signals further include a strong single object identification signal indicating a constraint that a target object from a first image frame should not have a comparison value higher than a threshold for any other object from the first image frame. In this example or any other example, the one or more automatic supervision signals include a weak object persistence signal indicating a constraint that each target object in the first image frame should have a comparison value higher than a threshold for at most one test object in the second image frame. In this example, or any other example, the weak object persistence signal further indicates that if a target object from the first group has a comparison value higher than a threshold relative to a test object in the second group, then the test object should also, conversely, have a comparison value higher than a threshold relative to the target object. In this example, or any other example, the object feature data and comparison values ​​are generated by one or more differentiable functions, and training the machine learning system to track objects involves configuring one or more differentiable functions based on unsupervised labels and one or more automatically supervised signals.

[0080] It should be understood that the configurations and / or methods described herein are exemplary in nature, and these particular implementations or examples are not intended to be limiting, as many variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Therefore, the various actions shown and / or described may be performed in the order shown and / or described, in a different order, in parallel, or omitted. Similarly, the order of the above processes may be changed.

[0081] The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations disclosed herein, as well as other features, functions, actions and / or properties, and any and all equivalents thereof.

Claims

1. A system for tracking a target object across multiple image frames, comprising: Logic machine; as well as A storage machine that stores instructions executable by the logical machine to: Calculate the trajectory for the target object on one or more previous frames that occur before the target frame; In response to the assessment that the target object was not detected in the target frame: Once it is determined that the target object is not detected in the target frame due to occlusion by one or more other objects, an estimated region for the target object is predicted based on the trajectory. The occlusion center is predicted based on a set of candidate occlusion locations of a set of other objects within a threshold distance of the estimated region, each of the set of candidate occlusion locations overlapping the estimated region; as well as The bounding box of the target object in the target frame is automatically estimated based on the occlusion center, wherein the bounding box is estimated by a trained machine learning system trained through supervised learning of image data and ground truth bounding boxes.

2. The system of claim 1, wherein the instructions are further executable to compute a heatmap for the occlusion center by operating one or more convolutional neural network units.

3. The system of claim 1, wherein estimating the bounding box includes operating a state machine configured to record tracking states of a plurality of objects and state information of tracked motions, and to estimate the bounding box based on such recorded state information.

4. The system of claim 1, wherein estimating the bounding box comprises operating a Kalman filter.

5. The system of claim 1, further comprising a training storage device storing instructions executable to train a machine learning system to track objects based on one or more unsupervised labels, the unsupervised labels being based on comparison values ​​indicating similarity between object feature data for an object in a first video frame and object feature data for an object in a second video frame.

6. The system of claim 5, wherein one or more comparison values ​​for object feature data are tracked in a similarity matrix, the similarity matrix comprising: For each row of objects in the union of the first set of target objects and the second set of test objects, For each column of the object in the union, and For placeholder columns with unsupervised labels, Each matrix element represents a comparison value between a pair of objects obtained from the union.

7. The system of claim 5, wherein the object feature data and the comparison value are generated by one or more differentiable functions, and wherein training the machine learning system to track objects includes configuring the one or more differentiable functions based on the unsupervised labels and one or more automatic supervision signals.

8. A method for tracking a target object across multiple image frames, the method comprising: Calculate the trajectory for the target object on one or more previous frames that occur before the target frame; The assessment indicates that the target object was not detected in the target frame because it was occluded by one or more other objects. Based on the trajectory, predict the estimated region for the target object; The occlusion center is predicted based on a set of candidate occlusion locations of a set of other objects within a threshold distance of the estimated region, each of the set of candidate occlusion locations overlapping the estimated region; as well as The bounding box of the target object in the target frame is automatically estimated based on the occlusion center, wherein the bounding box is estimated by a trained machine learning system trained through supervised learning of image data and ground truth bounding boxes.

9. The method of claim 8, further comprising computing a heatmap for the occlusion center by operating one or more convolutional neural network units.

10. The method of claim 8, wherein estimating the bounding box comprises operating a state machine configured to record tracking states of a plurality of objects and state information of tracked motions, and estimating the bounding box based on such recorded state information.

11. The method of claim 8, wherein estimating the bounding box comprises operating a Kalman filter.

Citation Information

Patent Citations

  • Multi-target tracking method based on multi-track fusion

    CN111815682A

  • Real-time object detection, tracking and occlusion reasoning

    US20140347475A1