How to track an object

Abstract

To provide a computer-implemented method of tracking objects in a video sequence of a scene.SOLUTION: A method comprises: determining a location of a sink in the scene where objects exit the scene and a location of a source where objects enter the scene; tracking a first object moving in the scene using a re-identification algorithm, wherein the first object is associated with a re-identification threshold of the re-identification algorithm; detecting that the first object has exited the scene at the sink; and, in response to detecting that the first object has exited the scene at the sink, adjusting the re-identification threshold associated with the first object such that a probability that the re-identification algorithm re-identifies a second object, which enters the scene at the source after the first object has exited the scene at the sink, as the first object is reduced.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention generally relates to a computer-implemented method for tracking an object in a video sequence of a scene. [Background technology]

[0002] A common application of video surveillance systems is tracking moving objects within a monitored scene. A computer vision-based technique for object tracking is re-identification (ReID), in which an object of interest (e.g., a person, vehicle, animal) is identified in one frame and then re-identified in successive frames. The tracking information of the object (e.g., the object's coordinates within the frames) can be recorded in a "tracklet" maintained for the object.

[0003] Re-identification has a wide range of applications, including surveillance, traffic monitoring, and crowd analysis. More generally, re-identification can be useful in situations where a tracked object may be temporarily occluded during tracking. If an object after (partial or complete) occlusion can be re-identified as being the same object as before the occlusion, the re-identified object (and its tracking information) can be associated after the occlusion with the object and tracking information identified before the occlusion. A corresponding technique can be used when tracking an object moving from a first camera view of a scene to a second camera view of the scene. Summary of the Invention

[0004] It has been recognized by the inventors that in tracking scenarios involving tracking objects with similar appearances (e.g., vehicles of similar make, model, and / or color, or people with similar clothing and / or appearance), the use of re-identification as conventionally implemented poses a risk of misidentifying similar but different objects as the same object when one of the similar objects exits the scene at the sink and a similar but different object (e.g., a different car of the same model and similar color) subsequently appears at the source. It is an object of the present invention to provide a method for tracking objects using re-identification that mitigates this risk.

[0005] Thus, according to a first aspect of the present invention, there is provided a computer-implemented method for tracking an object in a video sequence of a scene, the method comprising: determining a sink location in the scene where the object exits the scene and a source location where the object enters the scene; Tracking a first object moving within a scene using a re-identification algorithm, wherein the first object is associated with a re-identification threshold of the re-identification algorithm; Detecting that a first object has exited the scene at a sink; In response to detecting that the first object has exited the scene at the sink, adjusting a re-identification threshold associated with the first object such that the probability that the re-identification algorithm will re-identify a second object that enters the scene at the source after the first object has exited the scene at the sink as the first object is reduced; Includes.

[0006] By adjusting the re-identification threshold, a greater similarity between the second object and the first object is required for the re-identification algorithm to re-identify the second object as the first object, thereby enabling the method to reduce the risk of erroneously re-identifying as a tracked object an object that is visually similar to, but corresponds to a different physical object than, a tracked object that entered the scene at the source and previously exited the scene at the sink.

[0007] In the case of a temporary occlusion of an object moving across a scene, similar objects detected in the video sequence before and after the occlusion likely correspond to the same physical object. Therefore, in this case, re-identification is appropriate and desired. However, the likelihood of a physical object exiting a scene at the sink and re-entering it immediately thereafter is typically relatively low. However, some scenes include a source and a sink with relative positions such that it is still possible for the same physical object to re-enter the scene at the source some time after exiting the scene at the sink. Therefore, a simple approach of directly terminating tracking of an object upon detecting the exit of a tracked object at the sink does not allow for adequate handling of such exit and re-entry scenarios. That is, identifying the re-entering object as a new object could potentially result in the loss of relevant tracking information. In contrast, by adjusting a re-identification threshold associated with the first object rather than directly terminating tracking of the exiting object, the method of the present invention enables correct re-identification of an object that previously exited the scene at the sink and re-entered at the source.

[0008] The method thus enables object tracking with improved reliability, particularly in tracking scenarios that are likely to involve visually similar objects. Improved reliability is further provided without sacrificing the benefits associated with re-identification, i.e., handling temporary occlusions of tracked objects and movement of tracked objects between different camera views of a scene.

[0009] Further utility of the method of the present invention may be particularly evident when tracking objects in a scene that includes sources and sinks that are located adjacent to each other or that overlap.

[0010] The sinks and sources may be located along respective lanes of a road, at the exits and entrances of parking spaces, respectively, or at the exit and entrance ramps, respectively, along a road (eg, in a vehicle tracking application).

[0011] Sinks and sources may also be located on or adjacent portions of a walking path or pavement exiting a scene (e.g., in applications tracking individuals such as pedestrians and / or cyclists), or at the respective exits and entrances of a building, indoor space (e.g., a room), or outdoor space (e.g., a park).

[0012] "Re-identification threshold," as used herein, refers to a respective threshold associated with each respective tracked object for re-identifying the object detected in a frame of a video sequence as the respective tracked object by a re-identification algorithm. That is, for an object detected in a second frame to be re-identified as the object identified in a preceding first frame, the object features extracted from the second frame must match the object features extracted from the first frame to an extent defined by the re-identification threshold.

[0013] The term “object” (e.g., “first object,” “second object,” etc.) is used herein to refer to a representation in a video sequence (more specifically, in one or more frames of a video sequence) of a physical object in a scene (where the physical object has a type being tracked, such as a vehicle or an individual). Correspondingly, the term “physical object” refers to an actual physical object moving around in a monitored scene. Thus, as can be understood from the foregoing description, two “objects” (e.g., “first object” and “second object”) in a video sequence may depict the same or different physical objects in the scene, depending on the scenario.

[0014] In some embodiments, the method further comprises adjusting a re-identification threshold associated with the first object. Detecting the ingress of a second object at the source; re-identifying the second object as the first object using the adjusted re-identification threshold with a re-identification algorithm; Then, continue tracking the second object as the first object. Further includes:

[0015] Thereby, scenarios in which a second object is sufficiently similar to a first object currently being tracked (especially when the first and second objects actually correspond to the same physical object) may be resolved by re-identifying the second object as the first object so that tracking of the first object can continue.

[0016] In some embodiments, the method further includes, after re-identifying the second object as the first object, restoring the re-identification threshold and continuing tracking of the second object as the first object using the restored re-identification threshold. "Restoring" the re-identification threshold, as used herein, means restoring or returning the re-identification threshold to the value it had before it was adjusted, e.g., a predetermined default re-identification threshold. This approach is based on the concept that after resolving the exit and re-entry scenario (i.e., re-identifying the second object as the first object using the adjusted re-identification threshold), tracking of the first object may proceed using the original value of the re-identification threshold.

[0017] In some embodiments, re-identifying the second object as the first object includes comparing a set of second object features of the second object (i.e., second object features extracted from video frames including the second object) with a set of first object features of the first object (i.e., first object features extracted from video frames including the first object) to generate a matching score, and comparing the matching score to an adjusted re-identification threshold. Generating a matching score between the first object features and the second object features in this manner allows for re-identification to be evaluated using a threshold test.

[0018] The matching score may be defined such that the more similar the set of first and second object features are, the higher the matching score (e.g., the matching score increases as the distance between the set of first and second object features decreases). In this case, the re-identification threshold may be adjusted by being raised. Thus, the second object may be re-identified as the first object in response to the matching score meeting or exceeding the adjusted (raised) re-identification threshold.

[0019] Alternatively, the matching score may be defined such that the more similar the set of first and second object features are, the decreasing matching score (e.g., the matching score decreases as the distance between the set of first and second object features decreases). In this case, the re-identification threshold may be adjusted by lowering it. Thus, the second object may be re-identified as the first object in response to the matching score meeting or falling below the adjusted (lowered) re-identification threshold.

[0020] In some embodiments, tracking the first object includes maintaining a tracking context of the first object, and re-identifying the second object as the first object includes associating the second object with the tracking context of the first object.

[0021] Thus, "re-identification" of a second object as a first object may be achieved simply by associating the second object with the first object's existing tracking context, and thus the second object may, in a sense, inherit previously determined tracking data (e.g., tracklets) for the first object.

[0022] In some embodiments, the entry of the second object at the source is detected within a predetermined time of detecting the first object exiting the scene at the sink, i.e., a further condition for re-identifying the second object as the first object may be that the second object enters the scene at the source within a predetermined time of the exit of the first object at the sink.

[0023] In some embodiments, the method further comprises determining that more than a predetermined time has elapsed since detecting that the first object has left the scene at the sink: Detecting the ingress of a second object at the source; tracking the second object as a different object from the first object using a re-identification algorithm and a re-identification threshold associated with the second object; Further includes:

[0024] Thus, if it is determined that more than a predetermined time has elapsed since the first object was detected to have exited the scene at the sink, it may be inappropriate to attempt to re-identify a new object entering the scene at the source as the first object. In this case, the method may simply determine that the second object should be tracked as the new object. In embodiments in which tracking the first object includes maintaining a tracking context for the first object, the method may further include removing the tracking context for the first object from a set of active tracking contexts maintained by the re-identification algorithm in response to determining that more than a predetermined time has elapsed since the first object was detected to have exited the scene at the sink. Thus, by terminating tracking of the first object, both computational and memory resources may be conserved.

[0025] Furthermore, as noted above, scenarios may arise in which a second object is not sufficiently similar to a currently tracked first object to be re-identified as the first object. Thus, in some embodiments, the method, subsequent to adjusting the re-identification threshold associated with the first object, includes: Detecting the ingress of a second object at the source; determining, using a re-identification algorithm and an adjusted re-identification threshold, that the second object is different from the first object; Then, tracking the second object as a different object from the first object using a re-identification algorithm and a re-identification threshold associated with the second object. Further includes:

[0026] The second re-identification threshold may typically be set to the same value as the (unadjusted) re-identification threshold associated with the first object, eg, a predetermined default re-identification threshold.

[0027] In some embodiments, the method includes determining a location of a set of sources from which an object enters a scene, the source being one source in the set of sources, and adjusting a re-identification threshold to reduce the probability that a re-identification algorithm will re-identify an object entering the scene at any one of the set of sources as the first object.

[0028] That is, it is sufficient to adjust and maintain a single common re-identification threshold associated with the first object, which can be used by the re-identification algorithm regardless of at which of several sources in the scene the second object enters the scene.

[0029] In some embodiments, the method further comprises adjusting a re-identification threshold associated with the first object. Detecting the ingress of a second object in any one of the set of sources; re-identifying the second object as the first object using the adjusted re-identification threshold with a re-identification algorithm; Then, continue tracking the second object as the first object. Further includes:

[0030] This allows a scenario in which a second object that is sufficiently similar to a first object currently being tracked and enters the scene at any one of the set of sources to be resolved by re-identifying the second object as the first object so that tracking of the first object can continue.

[0031] In some embodiments, the sink is a first sink, and the method further comprises determining a position of a second sink at which the object exits the scene, the method comprising: Tracking a third object moving within the scene using a re-identification algorithm, wherein the third object is associated with a corresponding re-identification threshold of the re-identification algorithm; Detecting that a third object has exited the scene at a second sink; responsive to detecting that the third object has exited the scene at the second sink, terminating tracking of the third object such that a fourth object entering the scene at the source is prevented from being re-identified as the third object; Further includes:

[0032] Thus, the method recognizes that some scenes may include a sink (a "second sink") such that there is substantially zero probability that the third object will re-enter the scene at the source after the object (the "third object") has exited the scene at the second sink. Thus, computational resources may be conserved by terminating tracking of the third object.

[0033] In some embodiments, tracking the third object includes maintaining a tracking context for the third object, and terminating tracking of the third object includes removing the tracking context for the third object from a set of active tracking contexts maintained by the re-identification algorithm, thereby freeing up memory resources for maintaining the tracking context for the third object.

[0034] According to a second aspect of the present invention there is provided a computer program product comprising computer program code portions configured to, when executed by a processing device, perform a method for tracking an object in a video sequence of a scene according to the method of the first aspect or any of its embodiments.

[0035] According to a third aspect of the present invention, there is provided an object tracking system, comprising: at least one camera for capturing a video sequence of the scene; a processing device configured to track an object in a video sequence according to the method of the first aspect or any of its embodiments; Equipped with.

[0036] The second and third aspects feature the same or equivalent advantages as the first aspect, and any feature described in relation to the first aspect may have a corresponding feature in the system, and vice versa.

[0037] This and other aspects of the invention will now be described in more detail with reference to the accompanying drawings, in which embodiments of the invention are shown. [Brief explanation of the drawings]

[0038] [Figure 1] FIG. 2 shows a schematic view of a scene; [Figure 2] FIG. 1 illustrates an implementation of an object tracking system. [Figure 3] 1 is a flowchart of a method for tracking an object in a video sequence of a scene. [Figure 4] 4 is a flowchart of an extension of the method of FIG. 3. DETAILED DESCRIPTION OF THE INVENTION

[0039] 1 shows a schematic diagram of an exemplary view of a scene 1 monitored by a camera. This view may correspond to a video frame of a video sequence captured by the camera. At the time shown in FIG. 1, the scene 1 includes several objects in the form of vehicles moving around within the scene 1, each of which is being tracked by a re-identification algorithm.

[0040] Scene 1 includes several sources 2a-c and several sinks 4a-c. Note that the respective numbers of sources and sinks in FIG. 1 are merely non-limiting examples, and that a scene may more generally include any number of sources and sinks, but may include at least one source and at least one sink. Each source 2a-c represents an area of ​​scene 1 where an object may enter scene 1. Conversely, each sink 4a-c represents an area of ​​scene 1 where an object may exit scene 1. Thus, each source 2a-c represents a possible entry point of an object into scene 1, and each sink 4a-c represents a possible exit point of an object from scene 1.

[0041] In the illustrated example, each of sources 2a-c and sinks 4a-c is located along a respective lane of a road segment entering or exiting the monitored view of scene 1. The sources and sinks may be located adjacent to one another, such as along a respective road lane, as shown, e.g., source 2a and sink 4a, source 2b and sink 4b, and source 2c and sink 4c in FIG.

[0042] At the time represented by FIG. 1, one of the tracked objects, e.g., car 10a, has just entered scene 1 at source 2c and is moving toward sink 4a. Using re-identification, car 10a can be tracked across successive video frames as it moves through scene 1. While moving, car 10a may become temporarily occluded, fully or partially, by another tracked object (e.g., another car) or by a fixed obstacle or structure (e.g., a tree canopy, a traffic light, or a building). Here, the use of re-identification allows car 10a to be re-identified as the same object before and after the occlusion event, so that car 10a can be associated with the currently identified object and its tracking information (e.g., a tracklet that records the object's previous coordinates within the frame).

[0043] More specifically, the re-identification algorithm may be implemented by extracting a set of object features of an object detected in a video frame from several consecutive video frames of a video sequence. The object features extracted from a given video frame may be compared to object features extracted from one or more preceding video frames to generate a matching score. The re-identification algorithm may perform a threshold test including comparing the matching score to a re-identification threshold associated with each tracked object. In response to the matching score passing the threshold test, the object features extracted from the given frame and one or more preceding frames may be determined to belong to the same (physical) object, and therefore, the object in the given video frame may be re-identified as such. The location of the re-identified object in the given frame may therefore be recorded in a tracklet associated with the object. If the matching score does not pass the threshold test, the object features extracted from the object in the given video frame may be determined to be a different object from the object identified in the preceding one or more video frames, and therefore, its location in the given frame may not be associated with the previously tracked object.

[0044] For each object being tracked, a respective tracking context may be maintained. The tracking context may include an object identifier and a tracklet that defines the tracking information along which the object has progressed. As new objects are detected and identified within the video sequence, new tracking contexts may be created. At some point, when tracking of an object should end, further attempts to re-identify the detected object with the currently tracked object may be stopped, and updates of the tracklets of the tracking context associated with the currently tracked object may be stopped.

[0045] The re-identification algorithm may maintain a set of active or current tracking contexts, each associated with a respective object currently being tracked by the re-identification algorithm. Each time a new object is detected and identified by the re-identification algorithm in the video sequence, a new tracking context may be created and added to the set of active tracking contexts. Upon terminating tracking of a given object associated with a tracking context in the set of active tracking contexts, the associated tracking context may be removed from the set of active tracking contexts. This may prevent re-identification of a new object that subsequently enters the scene at the source as the given object (i.e., because the tracking context associated with the given object has been removed from the set of active tracking contexts). Tracking may be terminated in response to determining that the given object has exited the scene at the sink, thereby substantially eliminating the possibility that the object will re-enter the scene at the source. Tracking may also be terminated in response to determining that more than a predetermined time has expired since detecting that the object has exited the scene at the sink.

[0046] A video sequence of a scene, such as Scene 1, is subject to dynamic conditions, such as changing lighting conditions, fluctuating weather, and visibility conditions. Accordingly, each tracked object is associated with a re-identification threshold, which may be set to allow some tolerance for the tracked object's changing appearance. The corresponding re-identification threshold may be included in or stored in the tracking context of the tracked object. For example, lighting conditions in a scene may change from sunny to cloudy over the course of several video frames, resulting in a change in the apparent color of a tracked object, such as car 10a. As another example, an object may move in and out of an area lit by streetlights. To ensure that an object, such as car 10a, can be reliably tracked during such changes, the re-identification threshold may be set accordingly to a value that allows the re-identification algorithm to still re-identify car 10a as the same object despite any change in the apparent color of car 10a.

[0047] However, the tolerance provided by the re-identification algorithm and re-identification threshold can create problems when a tracked object exits a scene at a sink. For example, in FIG. 1, consider car 10b exiting scene 1 at sink 4a. After car 10b exits scene 1 at sink 4a, if a different car of a similar make, model, and / or color (i.e., similar within the tolerance defined by the re-identification threshold associated with car 10b) enters scene 1 at a source, such as adjacent source 2a, there is a risk that the re-identification algorithm will re-identify this new, different car as the same object as car 10b and thus continue to track the new car as the currently tracked car 10b (e.g., record the new car's further detected position in the tracklet of the tracking context of the currently tracked car 10b). On the other hand, in some instances, such as adjacent source 2a and sink 4a, car 10b may make a U-turn after exiting scene 1 at sink 4a and thus re-enter scene 1 at source 2a. In this case, when car 10b enters scene 1 at source 2a, it is preferably re-identified as indeed the same object and continually tracked.

[0048] These seemingly competing goals may be addressed, as described in this disclosure, by adjusting a re-identification threshold associated with a first tracked object, in response to detecting that the first tracked object has exited the scene at the sink, such that the probability that the re-identification algorithm will re-identify as the first object a second object that enters the scene at the source after the first tracked object has exited the scene at the sink is reduced. In other words, the tolerance of the re-identification algorithm when attempting to re-identify the second object as the first tracked object may be reduced such that greater visual similarity between the second object and the first object is required for the re-identification algorithm to re-identify the second object as the first tracked object.

[0049] An exemplary implementation of the method will now be described in more detail with reference to FIG. 1 and further with reference to FIGS.

[0050] 1 illustrates an example of a vehicle tracking application, the locations and types of sources and sinks will vary depending on the type of scene and object being tracked, as can be appreciated. For example, if additionally or alternatively tracking a pedestrian, a source 6a and sink 8a for the pedestrian may be located along a pavement or a walking path, such as on the pavement entering or exiting the scene or a portion of that path. Source 6b and sink 8b illustrate an example of overlapping sources and sinks. It should be noted, therefore, that the following disclosure is equally applicable to other tracking applications and other configurations and combinations of sources and sinks.

[0051] Figure 2 shows an example implementation of an object tracking system 20. System 20 includes a video surveillance camera 22 for capturing a video sequence V of a scene, such as Scene 1. While Figure 1 shows a single view scene and Figure 2 shows a single camera 22, system 20 may include a system of two or more cameras, each monitoring a respective sub-view of the scene.

[0052] System 20 further comprises a processing device 24 and an associated memory 26. Memory 26 may be coupled to or included in processing device 24. Processing device 24 is configured to receive a video sequence V in the form of a sequence of video frames from camera 22. As shown in Figure 2, video sequence V may be stored in memory 26, and processing device 24 may retrieve and process the video frames from memory 26 to track an object in the video sequence across the sequence of video frames.

[0053] Processing device 24 may maintain in memory 26 a set of active tracking contexts including a respective tracking context for each object currently being tracked. While FIG. 2 schematically illustrates two tracking contexts 28a-b, the number of tracking contexts 28a-b depends on the number of objects currently being tracked. A tracking context may include a set of data fields, such as an object ID, a set of object features representing the object, a re-identification (ReID) threshold, and one or more of tracklets, as shown. A tracking context may optionally further include a timer field, which is described in more detail below. The data fields of a tracking context may be updated by processing device 24 during tracking, as described further below.

[0054] The object tracking method may be implemented in both hardware and software. In a software implementation, the processing device 24 may be realized in the form of one or more processors, such as one or more central processing units and / or graphical processing units, that, in conjunction with computer program code instructions stored in a (non-transitory) computer-readable medium, such as a non-volatile memory, cause the processing device 24 to perform the steps of the object tracking method. Examples of non-volatile memory include read-only memory, flash memory, ferroelectric RAM, magnetic computer storage, optical disks, etc. In a hardware implementation, the processing device 24 may instead be realized by dedicated circuitry configured to implement the steps of the object tracking method. The circuitry may be in the form of one or more integrated circuits, such as one or more application-specific integrated circuits (ASICs) or one or more field-programmable gate arrays (FPGAs). It should be understood that it is also possible to have a combination of hardware and software implementations, meaning that some method steps may be implemented in dedicated circuitry and other method steps may be implemented in software.

[0055] FIG. 3 shows a flowchart of an example implementation of the object tracking method.

[0056] In step S1, processing device 24 determines the respective locations of at least one sink (e.g., one or more of sinks 4a-c) within scene 1 through which an object may exit scene 1 and the respective locations of at least one source (e.g., one or more of sources 2a-c) through which an object may enter scene 1. The locations of source 2a-c and sink 4a-c may be determined by an operator manually indicating, for example, via a graphical user input interface, where source 2a-c and sink 4a-c are located within scene 1. However, processing device 24 may also perform an automatic determination of the respective locations of one or more sources and sinks, for example, using an image recognition algorithm. As a non-limiting example, the image recognition algorithm may be configured or trained to identify road lanes within scene 1 and entry and exit points of road lanes in the periphery of scene 1.

[0057] In step S2, processing device 24 uses a re-identification algorithm to track one or more objects detected in scene 1. For example, when an object such as car 10a enters scene 1 at a source such as source 2c, processing device 24 attempts to determine whether the detected object (e.g., car 10a) corresponds to an object that is already being tracked. Accordingly, processing device 24 extracts a set of object features from a video frame in which the object (e.g., car 10a) is detected at source 2c and compares the set of object features with respective sets of object features associated with one or more currently tracked objects (e.g., each object for which a respective tracking context 28a-b is maintained by processing device 24).

[0058] To generate a matching score, processing device 24 may determine a distance (e.g., Euclidean distance or another distance, possibly multidimensional, suitable for comparing sets of object features) between the set of object features of the newly detected object and the set of object features of the currently tracked object. The extracted object features may generally include visual features and latent (hidden) features. Non-limiting examples of object features include feature vectors extracted by a convolutional neural network (CNN), color histograms, and computer vision feature descriptors such as Histogram of Oriented Gradients (HOG) or Speeded-Up Robust Features (SURF). Processing device 24 may compare the matching score to a re-identification threshold associated with the currently tracked object (e.g., stored in the tracking context of each currently tracked object). The matching score may be defined to increase (e.g., monotonically, typically, perfectly monotonically) with increasing similarity (e.g., decreasing distance) between the sets of compared object features, and the newly detected object may be re-identified as the currently tracked object in response to a matching score that meets or exceeds a re-identification threshold. Alternatively, the matching score may be defined to decrease (e.g., monotonically, typically, perfectly monotonically) with increasing similarity (e.g., decreasing distance) between the sets of compared object features, and the newly detected object may be re-identified as the currently tracked object in response to a matching score that meets or falls below the re-identification threshold. In either case, if the threshold test is passed, processing device 24 may proceed to track the object (e.g., car 10a) in subsequent frames as the currently tracked object, e.g., record the object's successive positions within the tracklets of the respective tracking context (e.g., tracking context 28a).

[0059] Car 10b is an example of an object (hereinafter interchangeably referred to as a "first object") being tracked by processing device 24 using a re-identification algorithm across a series of preceding video frames at the time shown in FIG. 1 . Thus, processing device 24 may maintain in memory 26 a tracking context 28b for first object 10b (i.e., tracking context 28b is maintained within the set of active tracking contexts 28a-b) at the time shown in FIG. 1 . First object 10b may have entered the scene, for example, at source 2b or source 2c. First object 10b is exiting scene 1 at sink 4a, as shown in FIG. 1 . Thus, in step S3, processing device 24 detects that first object 10b has exited the scene at sink 4a in a video frame following the frame shown in FIG. 1 .

[0060] In step S4, processing device 24 adjusts a re-identification threshold associated with first object 10b (e.g., the ReID threshold of tracking context 28b) in response to detecting that first object 10b has exited the scene at the sink. The re-identification threshold may be set to a default re-identification threshold before being adjusted. If the matching score is defined to increase with increasing similarity, the re-identification threshold may be raised by applying an additional offset to the default value of the re-identification threshold or by scaling the default value of the re-identification threshold by a factor greater than 1. If the matching score is defined to decrease with increasing similarity, the re-identification threshold may be lowered by applying a subtractive (negative) offset to the default value of the re-identification threshold or by scaling the default value of the re-identification threshold by a factor less than 1.

[0061] The default value of the re-identification threshold and the amount by which the re-identification threshold is adjusted each represent design parameters whose values ​​may be established based on a priori knowledge such as the type of object being tracked, the location of the scene, the source and sink, the tolerance required to enable reliable re-identification of the tracked object, and what risk is acceptable for re-identifying a similar but different object at the sink as an object that has exited the scene. The matching score used by the re-identification algorithm may typically be determined as a normalized value. Thus, the matching score and re-identification threshold may be in the range [0, 1]. If the matching score is defined to increase when object features are more similar, the default value of the re-identification threshold may be, by way of non-limiting example, in the range of 0.6 to 0.7. If the matching score is defined to decrease when object features are more similar, the default value of the re-identification threshold may be in the range of 0.3 to 0.4. In either case, the re-identification threshold may be adjusted (increased or decreased) by, for example, 10 to 30%. Thus, in either case, the re-identification threshold is adjusted so that the probability that the re-identification algorithm will re-identify a second object that enters the scene at the source after the first object has exited the scene at the sink as the first object is reduced.

[0062] After adjusting the re-identification threshold associated with the first object 10b, the processing device 24 may detect, at S5, the entry of a second object at the source into the scene 1. The second object may be detected at any one of the sources 2a-c of the scene, for example, at the source 2a adjacent to the sink 4a.

[0063] In response to detecting the second object at the source, processing device 24 attempts to re-identify the second object as first object 10b at S6 using a re-identification algorithm and an increased re-identification threshold. As described above, processing device 24 may compare a set of second object features of the second object with a set of first object features of first object 10b to generate a matching score. Processing device 24 may perform a threshold test including comparing the matching score to the adjusted re-identification threshold. In response to the matching score passing the threshold test (e.g., in the case of an increased re-identification threshold, the matching score meets or exceeds the increased re-identification threshold, or in the case of a decreased re-identification threshold, the matching score meets or falls below the decreased re-identification threshold), the second object is re-identified as first object 10b, and the method proceeds along the “Yes” branch of FIG. 3 . In response to the matching score not passing the threshold test (e.g., in the case of an increased re-identification threshold, the matching score is less than the increased re-identification, or in the case of a decreased re-identification threshold, the matching score exceeds the decreased re-identification threshold), the second object is determined to be different from the first object 10b, and the method proceeds along the "No" branch of FIG. 3.

[0064] If the method proceeds down the "Yes" branch, after re-identifying the second object as the first object 10b, the processing device 24 restores the adjusted re-identification threshold to the value of the re-identification threshold before it was adjusted, e.g., the default value, in step S7. The re-identification threshold may be restored by subtracting an additional offset, adding a subtraction offset, or rescaling the increased re-identification threshold by the inverse of the aforementioned scaling factor.

[0065] Processing device 24 may then continue tracking the second object as first object 10b at S8 using the restored re-identification threshold.

[0066] If the method proceeds down the "No" branch, processing device 24 proceeds in step S9 by tracking the second object as a different object from first object 10b. Assuming processing device 24 fails to re-identify the second object as any currently tracked object, the second object may be tracked as a new object, creating and adding a new tracking context associated with the second object to the set of active tracking contexts, e.g., as described with reference to the entry of object 10a at source 2c in FIG. 1.

[0067] Thus, by adjusting the re-identification threshold, the above-described method may reduce the risk of continuing to erroneously track a second object that is likely to correspond to a different physical object as the tracked object 10b, and thus requires greater visual similarity between the second object and the first object 10b for successful re-identification. However, if the second object and the first object 10b are similar enough to produce a matching score that passes the threshold test using the adjusted re-identification threshold (in which case the second object is more likely to correspond to the same physical object as the first object 10b), the second object may still be re-identified as the first object 10b and continue to be tracked accordingly.

[0068] To avoid endless attempts to re-identify an object entering scene 1 at the source as the first object 10b, a further condition for re-identifying a second object as the first object 10b may be applied: the second object must enter scene 1 at the source within a predetermined time from the exit of the first object 10b at the sink. If this additional condition is met, the method may proceed according to the "Yes" branch. If this additional condition is not met, the method may proceed according to the "No" branch. The processing device 24 may, for example, maintain a timer for each tracking context 28a-b (the "Timer" field in FIG. 2). In response to detecting that the first object 10b has exited scene 1 at the sink 4a, the processing device 24 may begin incrementing the value of the timer. If the second object entering the scene has not been re-identified as the first object 10b when the timer reaches a predetermined time limit, tracking of the first object 10b may be terminated and the tracking context 28b of the first object 10b may be removed from the set of active tracking contexts. If the first object 10b later re-enters the scene 1, the first object 10b may be detected, identified, and tracked as a new object, for example, using a new tracking context. The value of the predetermined time may be based on scene knowledge. For example, the predetermined time may be set depending on the type of tracking application, the specific types and locations of sources and sinks in the scene, etc. The predetermined time may also be set taking into account several tracking contexts that can be maintained by the available computational resources of the object tracking system 20.

[0069] FIG. 4 is a flowchart of an optional extension of the method of FIG. 3, which may be performed in parallel with the method steps of FIG.

[0070] As mentioned above, a scene may contain source-sink pairs where the probability of an object leaving the scene at the sink and immediately thereafter re-entering the scene at the source is effectively zero, or at least too small to motivate a re-identification attempt. The method of Figure 4 makes it possible to address this scenario.

[0071] At S10, processing device 24 determines the location of the second sink where the object exits the scene. With reference to Figure 1, with respect to source 2a, the second sink may be represented, for example, by sink 4b.

[0072] At S11, processing device 24 tracks a "third object" moving within scene 1 using a re-identification algorithm and a corresponding re-identification threshold associated with the third object. With reference to FIG. 1, the third object may be represented by, for example, a car or a large truck 10c. As shown in FIG. 1, third object 10c is exiting scene 1 at second sink 4b. Thus, at step S12, processing device 24 detects that third object 10c has exited the scene at second sink 4c in a video frame following the frame shown in FIG. 1.

[0073] In response to detecting that the third object 10c has exited scene 1 at the second sink 4b, the processing device 24 terminates tracking of the third object 10c at S13 so as to prevent re-identification of a fourth object that may enter scene 1 at any one of the sources 2a-c as the third object. As mentioned above, terminating tracking may include removing the tracking context associated with the third object from the set of active tracking contexts maintained by the re-identification algorithm.

[0074] The location of sink 4b may be determined manually by an operator or automatically using an image recognition algorithm, as described above. The decision to determine sink 4b as the "second sink" where tracking is terminated may be based on scene knowledge. For example, knowledge that in example scene 1, a U-turn along the road segment following sink 4b is prohibited or even impossible may be taken into account.

[0075] Those skilled in the art will appreciate that the present invention is in no way limited to the embodiments described above: on the contrary, many modifications and variations are possible within the scope of the appended claims.

Claims

1. 1. A computer-implemented method for tracking an object in a video sequence of a scene, the method comprising: determining a sink location within the scene where an object exits the scene and a source location where an object enters the scene; tracking a first object moving within the scene using a re-identification algorithm, wherein the first object is associated with a re-identification threshold of the re-identification algorithm; Detecting that the first object has exited the scene at the sink; and in response to detecting that the first object has exited the scene at the sink, adjusting the re-identification threshold associated with the first object such that the probability that the re-identification algorithm will re-identify a second object that enters the scene at the source after the first object has exited the scene at the sink is reduced. Including, Following adjusting the re-identification threshold associated with the first object, the method further comprises: detecting the ingress of the second object at the source; and in response to detecting the ingress of the second object at the source, attempting to re-identify the second object as the first object using the re-identification algorithm and the adjusted re-identification threshold, wherein attempting to re-identify the second object as the first object includes: comparing a set of second object features of the second object with a set of first object features of the first object to generate a matching score; and performing a threshold test by comparing the matching score to the adjusted re-identification threshold. In response to the matching score passing the threshold test, re-identifying the second object as the first object, thereafter restoring the re-identification threshold, and continuing to track the second object as the first object using the restored re-identification threshold; In response to the matching score failing the threshold test, determining that the second object is distinct from the first object, and then tracking the second object as a distinct object from the first object using the re-identification algorithm and a re-identification threshold associated with the second object; Including, the matching score is defined to increase as the set of first object features and the set of second object features become more similar, and the re-identification threshold is adjusted by increasing the re-identification threshold; or The method, wherein the matching score is defined to decrease as the set of first object features and the set of second object features become more similar, and the re-identification threshold is adjusted by lowering the re-identification threshold.

2. 2. The method of claim 1 , wherein tracking the first object includes maintaining a tracking context for the first object, and wherein re-identifying the second object as the first object includes associating the second object with the tracking context for the first object.

3. The method of claim 1 , wherein the entry of the second object at the source is detected within a predetermined time period after detecting that the first object has exited the scene at the sink.

4. The method of claim 1, wherein tracking the first object includes maintaining a tracking context for the first object, and the method further includes removing the tracking context for the first object from a set of active tracking contexts maintained by the re-identification algorithm in response to determining that more than a predetermined time has elapsed since the first object was detected at the sink to have exited the scene.

5. 2. The method of claim 1, wherein the method includes determining a location of a set of sources from which an object enters the scene, the source being one source in the set of sources, and wherein, following adjusting the re-identification threshold, the probability that the re-identification algorithm will re-identify an object entering the scene at any one of the set of sources as the first object is reduced.

6. The method described in claim 5, wherein the ingress of the second object is detected in any one of the sets of sources.

7. The sink is a first sink, and the method further includes determining a position of a second sink at which the object exits the scene, the method comprising: using the re-identification algorithm to track a third object moving within the scene, the third object being associated with a corresponding re-identification threshold of the re-identification algorithm; detecting that the third object has exited the scene at the second sink; in response to detecting that the third object has exited the scene at the second sink, terminating tracking of the third object such that a fourth object entering the scene at the source is prevented from being re-identified as the third object; The method of claim 1 further comprising:

8. 8. The method of claim 7, wherein tracking the third object includes maintaining a tracking context for the third object, and terminating tracking of the third object includes removing the tracking context for the third object from a set of active tracking contexts maintained by the re-identification algorithm.

9. The method of claim 1 , wherein the source and the sink are adjacent to or overlap each other.

10. at least one camera for capturing a video sequence of the scene; a processing device configured to track an object in the video sequence according to the method of any one of claims 1 to 9; An object tracking system comprising:

11. 10. A non-transitory computer readable storage medium comprising computer program code portions configured, when executed by a processing device, to perform the method for tracking an object in a video sequence of a scene according to any one of claims 1 to 9.

Images