Controlling a list of inactive tracks used for re-identification in an object tracking system

Abstract

The present disclosure relates to a method (600), a system, and software for controlling a list of inactive tracks in an object tracking system. The described technique includes the step of obtaining a false positive rate (FPR) of a re-identification model using a judgment threshold (S604) and the step of obtaining an acceptable FPR of the tracking system (S606). The re-identification model is used to match new detections with inactive tracks. The technique includes the step of determining one or more termination conditions for deleting inactive tracks based on the acceptable FPR and the model FPR (S608). If a first inactive track satisfies the termination condition, the first inactive track is deleted from the list (S610).

Description

Technology Field

[0001] The present invention relates to object tracking, and more particularly to a method, system, and software for controlling an inactive track list used for re-identification in an object tracking system. Background Technology

[0002] Object tracking systems play a key role in applications requiring continuous monitoring of objects within a scene, such as surveillance and autonomous vehicles. These systems utilize video feeds and algorithms to track objects across multiple frames, thereby forming a so-called "object track." This object track is constructed from a series of associated detection results for the same object.

[0003] However, tracking may be interrupted due to reasons such as occlusion, the object moving out of sight, or missing detection. In such cases, the object track becomes inactive, and these inactive tracks are stored in memory or an “inactive track gallery.” When the object reappears, it can be re-identified using the inactive track gallery, thereby preventing the creation of duplicate tracks for the same object. The IEEE ICIP 2017 paper “Simple online and realtime tracking with a deep association metric” by Wojke N. et al. discloses a technique for object tracking in occlusion situations.

[0004] Generally, “re-identification (ReID)” refers to the process of accurately identifying and associating a previously detected object with a new detection result for the same object. Using re-identification, a new detection result can be matched with an inactive track representing a previously detected object; this is accomplished by comparing feature vectors that represent the appearance or visual characteristics of the detected object. These feature vectors are typically generated by a convolutional neural network (CNN) trained to output similar vectors for images containing the same object and different vectors for images containing different objects. A decision model evaluates the feature similarity between the new detection result and the inactive track by comparing the feature vector associated with the new detection result with the feature vector associated with the inactive track. At this stage, a similarity score (e.g., Euclidean distance or cosine similarity) may be calculated, and this similarity score can be used to determine whether to match the new detection result with the inactive track.

[0005] However, as the number of inactive tracks increases, the accuracy of the decision model decreases. For example, a CNN may have an accuracy of about 70% in matching a single detection result in a gallery containing 100 inactive tracks, but the accuracy can be improved to 90% when there are only 5 inactive tracks. To manage this, a method can be used to keep the size of the inactive track gallery small and maintain accuracy by discarding inactive tracks that have remained inactive for a predetermined time limit (e.g., 5 seconds).

[0006] This method works effectively in scenes with medium object density, as the number of inactive tracks is maintained at a manageable level. However, in scenes with very few objects, the aforementioned time limit may be unnecessarily strict, and maintaining inactive tracks for a longer period can improve the probability of successful re-identification without overloading the system. Conversely, in scenes with many objects, it may be necessary to discard inactive tracks more quickly to prevent the inactive track gallery from becoming excessively large, otherwise the accuracy of the decision model may be degraded.

[0007] Therefore, improvement is required in this context. The problem to be solved

[0008] Considering the above, it is desirable to resolve or at least mitigate one or more of the aforementioned disadvantages as described in the appended independent patent claims. means of solving the problem

[0009] According to a first aspect of the present disclosure, a method for controlling a list of inactive tracks used for re-identification in an object tracking system comprising an object detector is provided, wherein the object tracking system tracks objects in a scene, and the object tracking system includes a re-identification model used when attempting to associate object detection data received from an object detector with an inactive track from the list, and the re-identification model is a metric learning model learned on the object detection data, the method comprises: obtaining a false positive rate of the re-identification model using a decision threshold of the re-identification model; obtaining an acceptable false positive rate of the object tracking system; determining one or more termination conditions for removing an inactive track from the list using the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model; determining that a first inactive track satisfies one of the one or more termination conditions; and removing the inactive track from the list.

[0010] Advantageously, by using the techniques described herein, the static time limit for the time an inactive track may be maintained in the list of inactive tracks used for re-identification is replaced by one or more termination conditions for determining when to discard the inactive track. The one or more termination conditions are determined based on the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model (ReID model). Accordingly, the time an inactive track is maintained in the list depends on the scene congestion and / or the complexity of the scene being tracked. By using the techniques of the present disclosure, the minimum matching precision between the object detection result and the inactive track (i.e., the acceptable false positive rate of the object tracking system) is maintained. Accordingly, the one or more termination conditions are determined by considering the performance value of the re-identification model (i.e., the false positive rate of the model) to maintain the minimum probability that the feature similarity result output from the re-identification model will be correct. This consequently ensures that the number of inactive tracks included in the list of inactive tracks is maintained at a manageable level to achieve an acceptable false positive rate.

[0011] In the context of the present disclosure, a “decision threshold” refers to a reference point, such as a distance metric between the feature vector of object detection data and the feature vector of an inactive track, at which the ReID model determines whether to classify the detection result as a true match or a false match with the inactive track. The threshold is set based on the point where the probability distribution for a true match and the probability distribution for a false match intersect. The false positive rate of the ReID model is influenced by this threshold, and the trade-off between true matches and false positives is controlled by adjusting the threshold.

[0012] In the context of the present disclosure, a “metric learning model” refers to a machine learning model designed to learn a similarity function between object data, typically using a distance metric between feature vectors. Examples of such models may include convolutional neural networks (CNNs), transformer-based models, and Siamese networks. As another example, non-neural network-based models such as Support Vector Machines (SVMs) may be used. These models may be trained using pairs, triplets, or larger sets of object data. In pair-based learning, the model is trained to minimize the distance between similar objects and maximize the distance between different objects. In triplet-based learning, anchor samples, positive samples (same object), and negative samples (different objects) are used. Additionally, larger object datasets may be used, which can further enhance the model’s ability to accurately distinguish objects.

[0013] In some examples, each inactive track in the list is associated with a counter indicating the number of failed association attempts for the corresponding inactive track, and the method further comprises the steps of: determining an association attempt threshold for each inactive track in the list using the acceptable false positive rate of an object tracking system and the false positive rate of a re-identification model; obtaining first object detection data from an object detector; and evaluating whether the first object detection data is associated with a first inactive track from the list, wherein if the object detection data is not associated with a first inactive track from the list, the counter associated with the first inactive track is increased, and the first termination condition among the one or more termination conditions includes the counter exceeding the association attempt threshold.

[0014] Advantageously, by systematically removing inactive tracks after repeated association failure attempts, the list of inactive tracks used for re-identification is prevented from being excessively filled with irrelevant data, thereby reducing memory and computational burdens. Using a counter indicating the number of times an inactive track is evaluated consecutively without matching with object detection can be a low-complexity method that better balances the trade-off between track retention and accurate re-identification performance. The aforementioned association attempt threshold is set to maintain the minimum matching precision between the object detection result and the inactive track.

[0015] In some examples, the method further comprises the step of acquiring a plurality of object detection data from an object detector, wherein the plurality of object detection data are associated with the same image frame of a video stream representing a scene, and for each of the plurality of object detection data, the method comprises the step of evaluating whether the object detection data is associated with a first inactive track from the list, and if the object detection data is not associated with a first inactive track from the list, incrementing a counter associated with the first inactive track, and the step of determining whether the first inactive track satisfies a first termination condition among one or more termination conditions is performed after the evaluation of all of the plurality of object detection data is completed.

[0016] Advantageously, by delaying the discarding of inactive tracks until all object detections within a single frame have been evaluated, the risk of tracks being deleted prematurely can be reduced. By considering all detections within a frame, this example minimizes the possibility of discarding inactive tracks that could subsequently be re-identified within the same frame, and ensures that potential matches are not overlooked. This approach can increase the chance of re-identification by attempting to associate each object detection within a frame with the corresponding inactive track before deciding to discard the inactive track.

[0017] In some examples, when the first object detection data is associated with the first inactive track from the list, the first inactive track is removed from the list. When the inactive track is associated with object detection, the track becomes active, and accordingly, the counter is no longer associated.

[0018] In some examples, the above threshold number is It is determined by solving the equation, where represents the true positive rate of the re-identification model, and represents the difference from 1 of the acceptable false positive rate of the object tracking system (i.e., 1 - acceptable false positive rate), and the threshold count is determined using N. Accordingly, the variable N adjusts the performance of the ReID model to meet system requirements and ensures that the ReID model achieves a minimum probability of outputting correct feature similarity results. For example, the threshold count can be determined by applying a ceiling function or a floor function to N.

[0019] In some examples, each inactive track in the list is associated with location data representing a location within the scene at a time when it is determined to be inactive by an object tracking system, and the method further comprises: determining a first location data from acquired object detection data; selecting a subset of inactive tracks from the list based on the location data of the first object detection data and the location data associated with each inactive track in the list; and for each inactive track included in the subset of inactive tracks, evaluating whether the object detection data is associated with the inactive track, and if the object detection data is not associated with the inactive track, incrementing a counter associated with the inactive track.

[0020] Advantageously, in this example, the counter is updated only for relevant inactive tracks (i.e., tracks within a threshold distance from the location associated with the inactive track). This prevents the deletion of unnecessary tracks. By evaluating only relevant tracks close to the object's location, the system avoids incrementing the counter for tracks that are clearly irrelevant to the detected object. This prevents valid inactive tracks from being deleted prematurely due to irrelevant object data, which can contribute to maintaining a more accurate and reliable tracking system over time.

[0021] In some examples, the method further comprises the steps of: obtaining a maximum number of inactive tracks that may be included in the list; sorting the list of inactive tracks according to a counter associated with each inactive track; and truncate the list of inactive tracks to include the maximum number of inactive tracks. Accordingly, constraints on computational resources can be satisfied. For example, if processing performance degrades, the truncation can temporarily reduce the load on the re-identification system without changing the threshold number of re-identification attempts for each inactive track. This can contribute to maintaining performance efficiency even under fluctuating computational resources and balancing system requirements with operational capacity.

[0022] In some examples, each inactive track in the list is associated with a timer indicating the time interval since the inactive track was added to the list, and the method further comprises: determining the current number of objects p in the scene; estimating the number of attempts r, indicating how many times an attempt to associate object detection data received from an object detector with an inactive track from the list is performed per unit time for each inactive track by an object tracking system; and determining a threshold time interval based on unit time using p, r, the acceptable false positive rate of the object tracking system, and the false positive rate of the re-identification model, wherein the second termination condition among the one or more termination conditions comprises the time interval of the inactive track exceeding the threshold time interval.

[0023] Unlike the preceding examples that used the number of association attempts, the second termination condition described above does not directly depend on the number of association failure attempts. Instead, in this example, the number of re-identification attempts per inactive track and per unit of time (e.g., second) is estimated, which can be calculated, for example, using historical data and / or the number of objects within the scene. Therefore, the number of re-identification attempts may depend not only on factors such as the number of objects within the scene, but also on the quality of the tracking system and the complexity of the scene. For example, in environments with many obstacles, such as areas where objects can be easily obscured or movement paths frequently intersect, tracks may be lost frequently, resulting in an increase in the number of re-identification attempts per inactive track and per unit of time. Conversely, in simple scenes with few obstacles and excellent visibility, the number of re-identification attempts per inactive track and per unit of time may be lower.

[0024] These estimates are subsequently used to determine critical time intervals, which allow for effective control of the number of inactive tracks while maintaining system accuracy in both crowded and low-object density environments, regardless of scene complexity.

[0025] In some examples, the critical time interval t based on unit time is It is determined by solving the equation. In other examples, the critical time interval t based on unit time is It is determined by solving the equation, where R corresponds to the number of total association attempts performed per unit time by the object tracking system and is a function of p.

[0026] In some examples, the method includes the step of estimating the change in the number of objects in a scene p′ per unit time using historical data representing the number of objects in a scene p and the number of objects in a scene at multiple time points, and the critical time interval t based on unit time is It is determined by solving the equation, where R corresponds to the number of total association attempts performed per unit time by the object tracking system and is a function of p′.

[0027] In this example, the number of new objects entering (and exiting) the scene per unit of time, i.e., the “track flow” into the scene, is estimated. The track flow p′ can be estimated as a continuously updated running average. Advantageously, this allows the system to respond more effectively to changes in scene complexity caused by the dynamic movement of objects.

[0028] In some examples, the method further comprises the steps of: obtaining a maximum number of inactive tracks that may be included in the list; sorting the list of inactive tracks according to a time interval indicated by a timer associated with each inactive track; and truncate the list of inactive tracks to include the maximum number of inactive tracks.

[0029] Accordingly, constraints on computational resources can be met. For example, if processing power is degraded, the aforementioned truncation can temporarily reduce the load on the Re-Identification (ReID) system without changing the critical time interval. This can contribute to maintaining performance efficiency even under fluctuating computational resources and balancing system requirements with available operational capacity.

[0030] In some examples, the method further comprises the steps of: obtaining an updated acceptable false positive rate of the object tracking system from a user of the object tracking system; and updating at least one of an association attempt threshold number and a threshold time interval using the updated acceptable false positive rate of the object tracking system.

[0031] Advantageously, flexibility to adapt to user-defined system requirements can be achieved. By allowing the user of the object tracking system to provide an updated acceptable false positive rate, this method enables the system to remain customized according to specific operational requirements. Accordingly, the system can dynamically adjust the association attempt threshold and / or threshold time interval based on the updated acceptable false positive rate.

[0032] According to a second aspect of the present disclosure, the objective is achieved by a non-transient computer-readable storage medium storing instructions for implementing a method according to a first aspect when executed on a device having processing capability.

[0033] According to a third aspect of the present disclosure, the objective is achieved by an object tracking system comprising an object detector and tracking objects within a scene, wherein the object tracking system comprises a re-identification model used when attempting to associate object detection data received from the object detector with an inactive track from a list, wherein the re-identification model is a metric learning model learned on the object detection data, and wherein the object tracking system is configured to control a list of inactive tracks used for re-identification, thereby obtaining a false positive rate of the re-identification model using a decision threshold of the re-identification model; obtaining an acceptable false positive rate of the object tracking system; determining one or more termination conditions for removing an inactive track from the list using the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model; determining that a first inactive track satisfies one of the one or more termination conditions; and wherein the inactive track is removed from the list.

[0034] In some examples, the object tracking system of the third aspect is connected to a camera that captures a video stream representing the scene.

[0035] The second and third aspects can generally have the same features and advantages as the first aspect. Brief explanation of the drawing

[0036] The foregoing and additional purposes, features, and advantages will be more clearly understood through the following exemplary and non-limiting detailed description of embodiments of the present disclosure with reference to the accompanying drawings, wherein like reference numerals indicate similar elements, FIG. 1 illustrates an object tracking system according to examples; FIG. 2 illustrates the distribution of matches and non-matches in a metric learning model used for re-identification in object tracking according to examples; FIG. 3 illustrates a tracked scene and a list of inactive tracks used for re-identification, wherein each inactive track is associated with a counter indicating the number of association failure attempts for the corresponding inactive track; FIG. 4 illustrates a list of inactive tracks used for tracking scenes and re-identification of FIG. 3, wherein each inactive track is additionally associated with location data; FIG. 5 illustrates a list of inactive tracks used for tracking scenes and re-identification according to examples, wherein each inactive track is associated with a timer representing the time interval since the inactive track was added to the list; and FIGS. 6 to 8 each illustrate a flowchart of a method for controlling an inactive track list used for re-identification in an object tracking system according to examples. Specific details for implementing the invention

[0037] Object tracking systems are essential in applications such as surveillance and autonomous driving, forming “object tracks” by continuously monitoring objects within a scene through video feeds and algorithms. These tracks are constructed from a sequence of associated object detections and may become inactive if an object is temporarily lost due to occlusion or out of sight. To prevent the creation of duplicate tracks, inactive tracks are stored in an “inactive track gallery” so that they can be re-identified when an object reappears. The Re-identification (ReID) process uses feature vectors and decision models, such as Convolutional Neural Networks (CNNs), to match new detection results with inactive tracks. The techniques described herein optimize track management by replacing the static time limits for inactive tracks described in the prior art with dynamic limits based on the performance values ​​of the decision model, thereby maintaining a desired minimum probability of the correctness of the feature similarity results output from the decision model. By using these techniques, the retention time of inactive tracks can be adaptively adjusted according to scene density, thereby maintaining high matching precision. In sparse scenes with relatively low object density, inactive tracks can be retained in the inactive track gallery for a longer period, improving re-identification performance without overloading the system. Conversely, in crowded scenes with relatively high object density, inactive tracks are discarded more quickly to maintain accuracy. The methods described herein utilize various termination conditions, such as counters that track the number of association failures and time limits that are adjusted according to scene density and track flow. Such dynamic control can contribute to maintaining desired performance levels and enables efficient and precise object tracking across various environments. Detailed examples and illustrations are provided in the following sections with reference to FIGS. 1 through 8.

[0038] FIG. 1 illustrates, by example, an overview of the components and functions of an object tracking system (100). The object tracking system (100) receives a video stream (104), that is, a sequence of video frames, as input. For example, the object tracking system may be connected to a camera that captures the video stream (104) representing a scene. In some examples, the object tracking system (100) is included in the camera.

[0039] The object tracking system (100) includes an object detector (102) configured to detect an object in each image frame of a video stream (104). The object detector (102) outputs object detection data (106).

[0040] The object detection data (106) may include information such as the spatial coordinates, size, and class label of each object detected within each image frame. The object detection data (106) may also include a confidence score indicating the likelihood that the detection corresponds to an actual object or classification result. The detection data may also include appearance features (feature vectors) extracted from the detected area within the image frame (e.g., extracted using a Convolutional Neural Network (CNN)), which facilitates object tracking and re-identification (ReID) by enabling the object tracker (122) to distinguish similar objects. This data may be formatted in the form of a bounding box or region of interest (ROI) around each object, which is used as input to the object tracker (122) to associate detection results across consecutive frames and generate a continuous object track. An object tracking system can implement “tracking-by-detection,” which is an object tracking approach that detects objects in each frame of a video sequence and then connects the detected instances between frames to form a track.

[0041] Object detection data (106) is provided to an object tracker (122) including a decision model (112). The decision model (112) evaluates the feature similarity between a new detection result and an inactive track and / or an active track by comparing a feature vector associated with a new detection result (i.e., an object in the object detection data (106)) with a feature vector associated with a track. A similarity score (e.g., Euclidean distance, cosine similarity, or a score using a CNN trained for that purpose) may be calculated, and said similarity score is used to determine whether to match the new detection result to an active track (AT), to an inactive track (IAT), or to define a new object track. In some embodiments, if the new detection result does not match to any active track, a new track may be initiated. Over time, as these new tracks accumulate a series of detection results, the feature vector representing the new track can be compared with the feature vector of the inactive track (IAT), thereby evaluating whether the new track should be associated with the inactive track.

[0042] In some configurations, the object tracker (122) may use different decision models (112) to evaluate the active track (AT) and the inactive track (IAT), respectively. Each AT or IAT may include multiple feature vectors, each feature vector representing an individual object detection associated with the corresponding track over time. The decision model may include a Kalman filter. The Kalman filter may utilize the predicted trajectory of the object being tracked, along with spatial coordinates and motion information of the object detection data. For re-identification (Re-ID), that is, matching with the inactive track (IAT) (110), the state of the Kalman filter may be used to estimate the location where the object may have moved after becoming inactive, and an attempt to reactivate the IAT may be made only within the estimated area, for which a similarity score is calculated as described above. More specifically, there may be various methods for implementing re-identification. For example, as an implementation method, potential matching candidates can be associated based on Kalman filters, etc., and a "matching cost" can be assigned to those candidates. Then, an association algorithm can be executed to match the detection result with the track based on said cost. The matching cost can be calculated based on feature distances, motion information, etc. The association algorithm can be, for example, a greedy approach, which performs matching starting from the lowest cost. Additionally, the association algorithm can be set as a bipartite graph problem and can be solved, for example, using the Hungarian algorithm.

[0043] The object tracker (122) maintains and updates information regarding active tracks and inactive tracks in memory. The active tracks (AT) and inactive tracks (IAT) may be stored in separate data structures, referred to as the active track list (108) and the inactive track list (110), respectively, depending on the embodiment. The active track list (108) contains information regarding objects currently being tracked, including the feature vector, spatial coordinates, and motion state of each object. These tracks are continuously updated with new object detection data while the object remains visible in the acquired images of the scene.

[0044] If an object temporarily disappears from the camera's field of view due to occlusion, out of sight, or out of the object detection area, the corresponding active track (AT) is moved to the inactive track list (110). This move may be triggered, for example, when the AT has not matched any detected object for a predetermined number of frames. The inactive track list (110) stores feature vectors and history data of tracks that are not currently active but are maintained so that they can be re-identified if an object reappears. Similarly, if object detection data (106) is associated with an inactive track (IAT) from the inactive track list (110), the IAT is moved to the active track list (108), and accordingly, the IAT is removed from the inactive track list (110).

[0045] The object tracker (122) also includes an inactive track metadata handler (IAT metadata handler) (120). Each inactive track (IAT) is associated with metadata used to evaluate whether it satisfies termination conditions, such as whether it exceeds a predetermined number of consecutive matching failure attempts or whether it is maintained in the inactive track list (110) beyond a predefined time limit. The inactive track metadata handler (120) is configured to process and update this data and perform the function of, for example, incrementing a counter that records the number of associated failure attempts for each IAT when a new detection result does not match the corresponding track. Additionally, or alternatively, the inactive track metadata handler (120) handles a timer for each IAT that tracks the time elapsed since the IAT was added to the inactive track list (110).

[0046] The inactive track metadata handler (120) may operate based on the output of the decision model (112), for example, if a counter is implemented as described above. For example, if the decision model (112) evaluates that no match is established between the object detection result and any inactive track, the inactive track metadata handler (120) may directly update the metadata corresponding to the inactive track (IAT). Specifically, a counter tracking the number of consecutive association failure attempts for the IAT may be updated (e.g., incremented). As described above, if the IAT is determined to be irrelevant with respect to location data, for example, if the Kalman filter predicts that the spatial and motion characteristics of the object detection data (106) do not match the inactive track, the inactive track metadata handler (120) may not increment the counter.

[0047] The object tracker (122) also includes an inactive track evaluator (IAT evaluator) (116). The inactive track evaluator (116) operates based on metadata provided by the inactive track metadata handler (120) and corresponding inactive tracks stored in the inactive track list (110). For each inactive track included in the inactive track list (110), the inactive track evaluator (116) uses the associated metadata to determine whether the inactive track satisfies one or more predefined termination conditions. If the inactive track satisfies a termination condition, the inactive track is removed from the inactive track list (110).

[0048] As illustrated in FIG. 1, the separation of functions for managing the list of inactive tracks used for re-identification in an object tracking system is provided for illustrative purposes only. Components such as the inactive track metadata handler (120) and the inactive track evaluator (116) are depicted as separate entities to clearly convey the roles and procedures involved in processing inactive tracks. However, it should be understood that the techniques discussed herein may be implemented in various ways, and the specific organizational form of the components may vary depending on the system architecture and design choices. For example, specific functions may be combined into a single module, distributed across multiple systems, or implemented using alternative methods to achieve the same purpose. Accordingly, the structure illustrated in FIG. 1 is not intended to be limiting, and any configuration that controls the list of inactive tracks and supports the re-identification process is included within the scope of this disclosure.

[0049] FIG. 2 illustrates the decision-making process of a re-identification model in an object tracking system. The graph shows the distribution of feature vector distances for two different object comparison types. The x-axis of the graph (200) represents the feature vector distance, and the y-axis represents the probability or measured frequency.

[0050] The two curves (202 and 204) correspond to comparisons for true match and false match, respectively. Curve (202) (solid line) represents the distance distribution for true re-identification cases where feature vectors derived from different instances of the same object are compared. On the other hand, curve (204) (dotted line) represents the distance distribution for false match cases where feature vectors derived from instances of different objects are compared.

[0051] The decision threshold (206) is a value used by the re-identification model to determine whether to identify a new object as a lost track. Distance values ​​located to the left of this threshold are classified as true re-identification, and distance values ​​located to the right are considered false re-identification. The false positive rate of the re-identification model can be calculated directly using this decision threshold. Specifically, the false positive rate is calculated as the proportion of cases located to the left of the threshold within the distribution corresponding to the curve (204) (false matching), which represents instances where the model incorrectly identifies a new object as a lost track.

[0052] The false positive rate (FPR) can be obtained from an object tracking system from internal or external sources, such as when derived from offline data like historical information or external guidelines. Once the system is deployed, the FPR can be dynamically calculated in an online environment. For example, in a scenario where an algorithm is running and there are multiple active tracks within a scene that are spatially sufficiently separated (i.e., far apart from each other in the camera view), it is certain that these tracks correspond to different objects or individuals. In such cases, by analyzing the ReID distance between different tracks (representing a 'false positive' distribution) and comparing it with the ReID distance within the same track (self-to-self distance, i.e., representing a 'true positive' distribution), the system can estimate the relevant distributions within the scene over time. This can contribute to ensuring sufficient statistical confidence in those distributions. Using these online observations, the object tracking system can dynamically set a decision threshold (206) to effectively separate the true positive distribution from the false positive distribution, even if the threshold may differ from the threshold set during offline learning. The newly set threshold in this way may have a different false positive rate for the re-identification model compared to the existing threshold, which also causes the termination conditions used to remove inactive tracks from the inactive track list to change, as explained later.

[0053] FIG. 3 illustrates a tracked scene (an image frame of such a scene) containing a plurality of objects (310, 312). FIG. 3 also illustrates a representation of a list (110) of inactive tracks (302, 304, 306, 308). In some embodiments, each inactive track (302, 304, 306, 308) included in the list (110) is associated with a counter (303, 305, 307, 309) indicating the number of failed attempts associated with the inactive track (302, 304, 306, 308). For example, an inactive track (308) was evaluated against object detection four times in succession but did not match. When a match is confirmed, as previously explained, the inactive track is moved from the inactive track list (110) to the active track list (108) (see FIG. 1).

[0054] In some examples, the termination condition for the inactive track (302, 304, 306, 308) includes the associated counter (303, 305, 307, 309) reaching a predetermined value.

[0055] The above predetermined value, i.e., the number of association attempts threshold, can be determined using the false positive rate of the re-identification model calculated using the determination threshold as described above in relation to FIG. 2. For example, the false positive rate of the re-identification model may be 0.025 (2.5%).

[0056] The association attempt threshold can also be determined using the acceptable false positive rate of the object tracking system. The acceptable false positive rate of the object tracking system can be obtained from the user of the object detection system. The acceptable false positive rate can be updated during the use of the object tracking system, and accordingly, the association attempt threshold can also be updated using the updated acceptable false positive rate of the object tracking system.

[0057] The acceptable false positive rate can vary depending on the implementation and the requirements of the use case. For example, in high-security scenarios such as surveillance in sensitive areas like airports or military facilities, the acceptable false positive rate may be set very low to minimize false re-identifications. Conversely, in less critical applications such as monitoring population flow in shopping malls or tracking wildlife, a higher false positive rate may be acceptable, as intermittent misidentifications may not significantly impact overall system performance.

[0058] For example, the acceptable false positive rate can be set to 0.1 (10%).

[0059] In some examples, the association attempt threshold is determined by solving the following equation:

[0060]

[0062] Here represents the true positive rate of the re-identification model, and represents the difference from 1 of the acceptable false positive rate of the object tracking system (i.e., 1 - acceptable false positive rate), and the threshold number is determined using N.

[0063] = 0.975 (False Positive Rate (FPR) 2.5%), and In the case where = 0.9 (acceptable error 10%), N is obtained by taking the natural logarithm of both sides. It can be calculated as = 4.16. As can be understood from the above explanation, and can have an arbitrary appropriate value and may vary depending on system requirements and the accuracy of the model.

[0064] The above threshold number can be determined by applying a ceiling function to N, in which case N = 5. Also, the above threshold number can be determined by applying a floor function to N, in which case N = 4.

[0065] In the example of Figure 3, the number of associated attempts is set to 5.

[0066] In the example scenario described in relation to FIG. 3, if the object detection data received from the object detector (102) of FIG. 1 is related to the object (312) of FIG. 3, the object detection data is not associated with the inactive track (308) (different objects), so the counter (309) associated with the inactive track (308) increases by one from 4 to 5. Similarly, the counter (307) associated with the inactive track (306) increases by one from 5 to 6, and the counter (305) associated with the inactive track (304) increases by one from 2 to 3. On the other hand, if the object detection data related to the object (312) is associated with the inactive track (302) (re-identification matching as the same object is successful), the inactive track (302) is removed from the inactive track list (110) and becomes active.

[0067] Since the counter (307) associated with the inactive track (306) exceeds the association attempt threshold (6 > 5), the inactive track (306) can be removed from the list of inactive tracks (110).

[0068] In some embodiments, the deletion of an inactive track (IAT) is not performed until the evaluation of all object detection data associated with the same image frame of the video stream representing the scene is completed. For example, in the example of FIG. 3, a plurality of object detection data are received from the object detector, including, for instance, object detection data regarding object (312) and object (310). Accordingly, in these embodiments, for each of the plurality of object detection data, the object detection system evaluates whether the object detection data is associated with an inactive track(s) from the inactive track list (110). If a particular inactive track is not associated with object detection data, the counter associated with that inactive track is incremented by 1 as described above. In this example, the step of determining whether the inactive track satisfies a termination condition (the counter exceeding the threshold number of association attempts) is performed after all evaluations of the plurality of object detection data are completed. In another example, an inactive track that satisfies the termination condition is deleted as soon as the termination condition is satisfied. For example, in this example, the inactive track (306) can be deleted as soon as the object detection data is evaluated as not being associated with the inactive track (306).

[0069] In some examples, not all object detection data is evaluated for all inactive tracks (302, 304, 306, 308) included in the list (110). Such an example is illustrated in FIG. 4. In this example, each inactive track (302, 304, 306, 308) included in the list (110) is associated with location data, said location data indicates a location within the scene where the object tracking system determined the corresponding track to be inactive. This location data is indicated by arrows (402, 404, 406, 408) in FIG. 4. In the example of FIG. 4, inactive tracks (302) and (304) were both lost near a building within the scene, inactive track (306) was lost near a bush, and inactive track (308) was lost near a large tree within the scene.

[0070] As described above, object detection data output from an object detector may include position data, such as spatial coordinates, that indicate the location where an object is detected within a scene or image frame.

[0071] For each new object detection, a subset of inactive tracks included in the list (110) may be selected based on the proximity between the location data of the inactive tracks and the location data of the new detection. This selection process may be induced by a threshold distance, which determines whether an inactive track should be evaluated for the new object detection. The threshold distance may be a predefined maximum allowable distance between the location data of the new object detection and the location data of the lost tracks (302, 304, 306, 308). If the distance between the two locations is within the threshold distance, the inactive track is considered a potential matching target and further evaluation is performed using a re-identification model. Conversely, if the distance between the two locations exceeds the threshold distance, the inactive track is determined to be unlikely to correspond to the same object and is not evaluated for the current detection. Additionally, velocity data or similar information associated with the lost track may be used to estimate the location where the lost track object is expected to exist at the current time. The above threshold distance may increase over time. By considering the object's previous velocity, the system can predict the object's possible location and adjust the selection of inactive tracks accordingly. This can contribute to more precisely limiting the set of potential matches by ensuring that the correct inactive tracks are included for evaluation even if the object has moved since it was last tracked.

[0072] By using this selection process, the object detection system can efficiently reduce the number of inactive tracks to be evaluated, thereby reducing computational load and improving tracking performance. As a result, only the counters (303, 305, 307, 309) associated with the selected tracks are incremented based on the evaluation results. This approach prevents unnecessary incrementing of counters for irrelevant inactive tracks, thereby ensuring that tracks are removed from the list only for relevant failed association attempts. Consequently, inactive tracks are prevented from being deleted based on irrelevant data and can be retained within the system. This allows the object detection system to retain more potentially valid inactive tracks as matching candidates, thereby improving the likelihood of successful re-identification in future evaluations. In the example of FIG. 4, object detection data corresponding to an object (312) detected near a building can be evaluated only for inactive tracks (304 and 302) from the list (110), and accordingly, the counters of inactive tracks (306 and 308) included in the list are not incremented based on the object detection data.

[0073] In some examples, termination conditions regarding time limits are defined. These time limits may be defined in light of how crowded the scene is and the complexity of the scene in order to maintain an acceptable false positive rate of the object tracking system. FIG. 5 illustrates such an example. In this example, each inactive track (302, 304, 306, 308) included in the list (110) is associated with a timer (502, 504, 506, 508) indicating the time interval since each inactive track (302, 304, 306, 308) was added to the list (110). For example, in this example, 1.35 seconds have elapsed since the inactive track (304) was added to the list.

[0074] In these examples, the termination condition includes the time elapsed of an inactive track exceeding a critical time interval. The critical time interval can be determined using the current number of objects p in the scene, and can also be determined by estimating the number of attempts r performed per time unit by the object tracking system so that object detection data received from the object detector is associated with the inactive track from the list. The critical time interval can be determined using p, r, the acceptable false positive rate of the object tracking system, and the false positive rate of the re-identification model.

[0075] In one example, the critical time interval t, expressed in time units, is the equation It is determined by solving. Therefore, t is determined as follows:

[0076]

[0078] if = 0.9( ), = 0.99( ), when r = 2 and p = 3, this gives t = 1.75.

[0079] In another example, t expressed in units of time is the equation It is determined by solving for, where R corresponds to the total number of association attempts performed per unit of time by the object tracking system and is a function of p. Therefore, t is determined as follows:

[0080]

[0081] For example, in the case where R(p) = 3.5, the example above and If you use the values, t = 3.0.

[0082] Regarding the estimation of r, the number of attempts performed per inactive track per time unit by the object tracking system to correlate inactive tracks in the inactive track list with object detection data from the object detector, a default value for r may be used initially, but it is desirable to subsequently adjust it to suit the specific scene. To this end, the algorithm collects statistics on the number of revive attempts performed in the scene over time and the number of active tracks present at each revive attempt. By analyzing the correlation between the attempt frequency and the number of objects p within the scene, the system can dynamically adjust r as a function of p based on the observed data, thereby improving the accuracy of the object tracking system and its responsiveness to changes in scene conditions.

[0083] In some examples not illustrated in FIG. 5, the method includes the step of estimating the change in the number of objects in the scene p' per unit of time using historical data in which the number of objects in the scene is counted at multiple time points, and the critical time interval t is determined in units of time by solving the following equation:

[0084] ,

[0085] Here, R corresponds to the total number of association attempts performed by the object tracking system per unit of time and is a function of p′. This can further improve adaptability to changes in the number of tracked objects within a scene.

[0086] As described in relation to FIGS. 3 and 4, if the time interval (502, 504, 506, 508) of the inactive tracks (302, 304, 306, 308) in FIG. 5 exceeds a determined threshold time interval, the corresponding track is deleted from the list (110). For example, if the threshold time interval t is set to 1.75 as above, the inactive track (306) is deleted.

[0087] In any one example disclosed in connection with FIGS. 3 to 5, the maximum number of inactive tracks that can be included in the list (110) may be implemented, thereby allowing the tracking system of the system to operate within specified resource constraints and focus on the most promising re-identification candidates.

[0088] In the example of FIG. 5, the implementation may include the steps of obtaining a maximum number of inactive tracks to be included in a list, sorting a list of inactive tracks according to a time interval indicated by a timer associated with each inactive track, and cutting the list so that the list includes the maximum number of inactive tracks.

[0089] In the examples of FIGS. 3 and 4, the maximum number of inactive tracks to be included in the list can be similarly obtained. The implementation may also include the step of sorting the list of inactive tracks based on a counter associated with each inactive track, which indicates the number of failed re-identification attempts. Tracks with high counter values ​​indicating multiple failed attempts are less likely to correspond to newly detected objects, so they may be lowered in priority or removed. After sorting, the list is truncated to include only the maximum number of inactive tracks.

[0090] The maximum number of inactive tracks can be set based on the current processing capacity of the object tracking system and can be continuously updated (e.g., at predefined intervals). For systems with limited resources, it may be necessary to enforce a lower maximum number to maintain optimal performance and prevent overload.

[0091] In any example described in relation to FIGS. 3 through 5, the matching criteria for associating object detection with an inactive track may become stricter over time and / or as the value of the counter increases. This means that the minimum similarity score required to match an object detection with an older inactive track must be higher than the criterion for matching a new object detection with a more recently deactivated track. As a result, the system gradually increases the difficulty of reactivating an inactive track, which effectively implements a soft termination process. This approach optimizes the re-identification process by allowing inactive tracks that are unlikely to correspond with new detections to be naturally phased out over time.

[0092] FIG. 6 illustrates a flowchart of a method (600) for controlling a list of inactive tracks used for re-identification in an object tracking system. The object tracking system includes an object detector, and the object tracking system tracks objects within a scene. The object tracking system includes a re-identification model used when attempting to associate object detection data received from the object detector with an inactive track from a list, and the re-identification model is a metric learning model learned on the object detection data.

[0093] The method (600) includes the step (S602) of obtaining the false positive rate of a re-identification model using the decision threshold of the re-identification model. Additionally, the method further includes the step (S604) of obtaining an acceptable false positive rate of an object tracking system.

[0094] The method (600) also includes the step (S606) of determining one or more termination conditions for deleting inactive tracks from a list using the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model.

[0095] The method (600) also includes a step (S608) of determining that the first inactive track satisfies one or more of the termination conditions.

[0096] The method (600) also includes the step (S610) of deleting the inactive track from the list.

[0097] FIG. 7 illustrates an exemplary flowchart of a method (700) that extends the method (600) of FIG. 6, as further described in relation to FIG. 3 and FIG. 4.

[0098] The method (700) includes steps (S602) and (S604) of the method (600).

[0099] In the method (700), the step (S606) of the method (600) is implemented as a step (S702) of determining a threshold number of association attempts for each inactive track in the list using the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model, wherein the first termination condition among the one or more termination conditions includes the counter exceeding the threshold number of association attempts.

[0100] The method (700) also includes the step (S704) of obtaining first object detection data from an object detector.

[0101] The method (700) further includes a step (S706) of evaluating whether the first object detection data is associated with a first inactive track from a list, and if the object detection data is not associated with a first inactive track from a list, a step of increasing a counter associated with the first inactive track.

[0102] The method (700) also includes steps (S608) and (S610) of the method (600).

[0103] FIG. 8 illustrates, as an extension of the method (600) of FIG. 6, an exemplary flowchart of the method (800) described above in relation to FIG. 5.

[0104] The method (800) includes steps (S602) and (S604) of the method (600).

[0105] The method (800) also includes a step (S802) of determining the current number of objects p in the scene.

[0106] The method (800) further includes the step (S804) of estimating how many times an attempt to associate object detection data received from an object detector with an inactive track from a list is performed by the object tracking system per inactive track and per time unit.

[0107] In the method (800), the step (S606) of the method (600) is implemented as a step (S806) of determining a threshold time interval in time units using p, r, the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model, wherein the second termination condition among the one or more termination conditions includes the time interval of an inactive track exceeding the threshold time interval.

[0108] The method (800) also includes steps (S608) and (S610) of the method (600).

[0109] The methods of FIGS. 7 and FIGS. 8 can be combined into a single method in some embodiments, in which case two termination conditions are defined as described above.

[0110] In embodiments, the methods described herein, e.g., method (600), method (700), and method (800), may be implemented using a non-transient computer-readable storage medium in which instructions for executing said methods are stored when executed on one or more devices having processing capabilities. Processors suitable for executing said instructions include, for example, general-purpose and special-purpose microprocessors, and in any type of computer, a single processor or one of multiple processors or cores. said processors may be supplemented by or included in an application-specific integrated circuit (ASIC).

[0111] The above embodiments should be understood as exemplary embodiments of the present disclosure. Additional embodiments of the present disclosure are also anticipated. For example, cumulative matching characteristics (CMCs) of a Re-ID model may be used to determine termination conditions. Additionally, equivalents and variations not described above may be adopted without departing from the scope of the present disclosure as defined by the appended claims.

[0112] delete

Claims

Claim 1 A method (600, 700, 800) for controlling a list (110) of inactive tracks (302, 304, 306, 308) used for re-identification in an object tracking system (100) that includes an object detector (102) and tracks an object within a scene, wherein the object tracking system includes a re-identification model (112) used when attempting to associate object detection data (106) received from the object detector with an inactive track from the list, and the re-identification model is a metric learning model learned on the object detection data, wherein the method comprises: a step (S602) of obtaining a false positive rate of the re-identification model using a decision threshold (206) of the re-identification model; a step (S604) of obtaining an acceptable false positive rate of the object tracking system; a step (S606) of determining one or more termination conditions for deleting an inactive track from the list using the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model; a first A method comprising: determining that an inactive track satisfies one of the one or more termination conditions (S608); and deleting the inactive track from the list (S610). Claim 2 In claim 1, each inactive track in the list is associated with a counter (303, 305, 307, 309) indicating the number of association failure attempts for the corresponding inactive track, and the method further comprises: a step (S702) of determining a threshold number of association attempts for each inactive track in the list using the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model, wherein the first termination condition among the one or more termination conditions includes the counter exceeding the threshold number of association attempts; a step (S704) of obtaining first object detection data from the object detector; and a step (S706) of evaluating whether the first object detection data is associated with a first inactive track from the list, wherein if the object detection data is not associated with a first inactive track from the list, the counter associated with the first inactive track is increased. Claim 3 In paragraph 2, the method further comprises: a step of obtaining a plurality of object detection data associated with the same image frame of a video stream illustrating the scene from the object detector; a step of evaluating, for each of the plurality of object detection data, whether the object detection data is associated with a first inactive track from the list, wherein if the object detection data is not associated with the first inactive track from the list, a counter associated with the first inactive track is incremented; and a step of determining whether the first inactive track satisfies the first termination condition among the one or more termination conditions after the evaluation of all object detection data of the plurality of object detection data is completed. Claim 4 A method according to paragraph 2, wherein if the first object detection data is associated with the first inactive track from the list, the first inactive track is deleted from the list. Claim 5 In paragraph 2, the above threshold number is a mathematical formula It is determined by solving, and in the above mathematical formula represents the true positive rate of the above re-identification model, and A method in which represents the value obtained by subtracting the acceptable false positive rate of the object tracking system from 1, and the threshold number is determined using N. Claim 6 In paragraph 2, each inactive track in the list is associated with location data representing a location within a scene where the inactive track is determined to be inactive by the object tracking system, and the method further comprises: determining a first location data from the acquired object detection data; selecting a subset of inactive tracks from the list based on the location data from the first object detection data and the location data associated with each inactive track in the list; and for each inactive track in the subset of inactive tracks, evaluating whether the object detection data is associated with the inactive track, wherein if the object detection data is not associated with the inactive track, the method comprises increasing a counter associated with the inactive track. Claim 7 The method of paragraph 2 further comprises the steps of: obtaining a maximum number of inactive tracks in the list; sorting the list of inactive tracks according to a counter associated with each; and truncating the list so that the list contains the maximum number of inactive tracks. Claim 8 In claim 1, each inactive track in the list is associated with a timer (502, 504, 506, 508) representing a time interval after the inactive track is added to the list, and the method further comprises: a step (S802) of determining the current number p of objects in the scene; a step (S804) of estimating r, which is the number of times an attempt to associate object detection data received from the object detector with an inactive track from the list is performed by an object tracking system per inactive track and per time unit; and a step (S806) of determining a threshold time interval expressed in time units using p, r, the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model, wherein the second termination condition among the one or more termination conditions includes the time interval of the inactive track exceeding the threshold time interval. Claim 9 In paragraph 8, the critical time interval t expressed in time units is the mathematical formula The critical time interval t, determined by solving or expressed in time units, is a mathematical equation The method is determined by solving for, where R corresponds to the total number of association attempts performed by the object tracking system per unit of time and is a function of p. Claim 10 In claim 8, the method further comprises the step of estimating p′, the amount of change in the number of objects in a scene per time unit, using historical data in which p and the number of objects in a scene are recorded at multiple time points, and the critical time interval t expressed in time units is a mathematical formula The method is determined by solving for, where R corresponds to the total number of association attempts performed by the object tracking system per unit of time and is a function of p′. Claim 11 In claim 8, the method further comprises: obtaining a maximum number of inactive tracks in the list; sorting the list of inactive tracks according to a time interval indicated by a timer associated with each; and truncating the list so that the list contains the maximum number of inactive tracks. Claim 12 A method according to any one of claims 2 to 11, wherein the method further comprises: obtaining an updated acceptable false positive rate of the object tracking system from a user of the object detection system; and updating at least one of a threshold number of association attempts and a threshold time interval using the updated acceptable false positive rate of the object tracking system. Claim 13 A non-transient computer-readable storage medium storing instructions for implementing the method according to claim 1 when executed on a device having one or more processing capabilities. Claim 14 An object tracking system comprising an object detector and tracking an object within a scene, wherein the object tracking system comprises a re-identification model used when attempting to associate object detection data received from the object detector with an inactive track from a list, wherein the re-identification model is a metric learning model learned on the object detection data, and the object tracking system is configured to control a list of inactive tracks used for re-identification, and the object tracking system comprises: obtaining a false positive rate of the re-identification model using a decision threshold of the re-identification model; obtaining an acceptable false positive rate of the object tracking system; determining one or more termination conditions for deleting an inactive track from the list using the acceptable false positive rate of the object tracking system and the false positive rate of the re-identification model; determining that a first inactive track satisfies one of the one or more termination conditions; and being configured to delete the inactive track from the list. Claim 15 In paragraph 14, an object tracking system connected to a camera that captures a video stream representing a scene.

Images