Method of tracking objects

Abstract

To provide a computer-implemented method of tracking objects in a video sequence of a scene.SOLUTION: A method comprises: determining a location of a sink in the scene where objects exit the scene and a location of a source where objects enter the scene; tracking a first object moving in the scene using a re-identification algorithm, wherein the first object is associated with a re-identification threshold of the re-identification algorithm; detecting that the first object has exited the scene at the sink; and, in response to detecting that the first object has exited the scene at the sink, adjusting the re-identification threshold associated with the first object such that a probability that the re-identification algorithm re-identifies a second object, which enters the scene at the source after the first object has exited the scene at the sink, as the first object is reduced.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention generally relates to a computer-implemented method for tracking objects within a video sequence of a scene.

Background Art

[0002] A common use of video surveillance systems is the tracking of moving objects within a monitored scene. Computer vision-based techniques for object tracking are re-identification (ReID) in which an object of interest (e.g., a person, vehicle, animal) is identified within one frame and then re-identified within successive frames. Tracking information of an object (e.g., coordinates of the object within a frame) can be recorded within a "tracklet" maintained for the object.

[0003] Re-identification has a wide range of applications including surveillance, traffic monitoring, and crowd analysis. More generally, re-identification can be useful in situations where an object being tracked can be temporarily occluded during tracking. If an object after (partial or complete) occlusion can be re-identified as being the same object as before the occlusion, the re-identified object (and its tracking information) can be associated with the object and tracking information identified before the occlusion after the occlusion. The corresponding technique can be used when tracking an object moving from a first camera view of a scene to a second camera view of the scene.

Summary of the Invention

[0004] In a tracking scenario involving the tracking of objects having similar appearances (e.g., vehicles of similar types, models, and / or colors, or people having similar clothing and / or appearances), the use of re-identification as conventionally implemented poses a risk, recognized by the inventors, of misidentifying a similar but different object (e.g., a different car of the same model and similar color) as the same object when one of the similar objects exits the scene at a sink and a similar but different object then appears at the source. The object of the present invention is to provide a method for tracking objects using re-identification that reduces this risk.

[0005] Accordingly, according to a first aspect of the present invention, there is provided a computer-implemented method for tracking an object within a video sequence of a scene. The method comprises determining a position of a sink within the scene where the object exits the scene and a position of a source where the object enters the scene, tracking a first object moving within the scene using a re-identification algorithm, wherein the first object is associated with a re-identification threshold of the re-identification algorithm, detecting that the first object has exited the scene at the sink, in response to detecting that the first object has exited the scene at the sink, adjusting the re-identification threshold associated with the first object such that the probability that the re-identification algorithm re-identifies a second object entering the scene at the source after the first object has exited the scene at the sink as the first object is reduced comprising.

[0006] By adjusting the re-identification threshold, a greater similarity between the second object and the first object is required for the re-identification algorithm to re-identify the second object as the first object. Thereby, the present method reduces the risk of mis-re-identifying an object that enters the scene at the source and is visually similar to a tracked object that previously exited the scene at the sink, but corresponds to a different physical object, as the tracked object.

[0007] In the case of temporary occlusion of an object moving across a scene, similar objects detected within the video sequence before and after the occlusion are likely to correspond to the same physical object. Thus, in this case, re-identification is appropriate and desirable. However, the likelihood that a physical object exits the scene at the sink and then immediately re-enters the scene is usually relatively low. However, some scenes include a source and a sink that have a relative position such that it is still possible for the same physical object to re-enter at the source of the physical object after exiting the scene at the sink after a while. Thus, detecting the exit of a tracked object at the sink, a simple approach of directly terminating the tracking of the object does not enable proper handling of such exit and re-entry scenarios. That is, identifying a re-entering object as a new object may potentially result in loss of relevant tracking information. In contrast, by adjusting the re-identification threshold associated with the first object rather than directly terminating the tracking of the exiting object, the method of the present invention enables correct re-identification of an object that previously exited the scene at the sink and re-enters at the source.

[0008] Thus, the method enables object tracking with improved reliability, particularly in tracking scenarios that are likely to include visually similar objects. The improved reliability is provided without sacrificing the advantages associated with re-identification, namely, the handling of temporary occlusion of the object being tracked and the movement of the object being tracked between different camera views of the scene.

[0009] The further usefulness of the method of the present invention can be particularly prominent when tracking objects in a scene that includes sources and sinks that are adjacent to or overlapping each other.

[0010] The sink and the source may be located along respective lanes of a road (e.g., in a vehicle tracking application), at respective exits and entrances of parking spaces, or at respective exit ramps and entrance ramps along a road.

[0011] The sink and the source may also be located at a part of a walking path or sidewalk leading out of the scene or respective adjacent parts thereof (e.g., in an application for tracking individuals such as pedestrians and / or cyclists), or at respective exits and entrances of a building, an indoor space (e.g., a room), or an outdoor space (e.g., a park).

[0012] As used herein, the "re-identification threshold" means, for each respective object being tracked, the respective threshold associated with the object being tracked for re-identifying, by a re-identification algorithm, an object detected within a frame of a video sequence as being the respective object being tracked. That is, for an object detected in a second frame to be re-identified as being the object identified in a preceding first frame, the object features extracted from the second frame need to match the object features extracted from the first frame to an extent defined by the re-identification threshold.

[0013] The term "object" (e.g., "first object", "second object", etc.) as used herein means a depiction within a video sequence of a physical object in a scene (where the physical object has a type that is being tracked, such as a vehicle or a person), and more particularly, within one or more frames of the video sequence. Correspondingly, the term "physical object" refers to an actual physical object that moves around within the scene being monitored. Thus, as can be understood from the foregoing description, two "objects" within a video sequence (e.g., "first object" and "second object") may depict the same physical object or different physical objects within the scene, depending on the scenario.

[0014] In some embodiments, the method includes, following adjusting a re-identification threshold associated with a first object, detecting an entry of a second object at a source, re-identifying, by a re-identification algorithm, the second object as the first object using the adjusted re-identification threshold, and subsequently continuing to track the second object as the first object. Thereby, a scenario where the second object is sufficiently similar to the first object that the second object is currently being tracked (in which case, in particular, when the first and second objects actually correspond to the same physical object) may be resolved by re-identifying the second object as the first object so that tracking of the first object can continue.

[0015]

[0016] ​In some embodiments, the method further includes, after re-identifying the second object as the first object, restoring a re-identification threshold, and using the restored re-identification threshold to continue tracking the second object as the first object. As used herein, "restoring" the re-identification threshold means restoring or returning the re-identification threshold to the value it had before being adjusted, e.g., a predetermined default re-identification threshold. This approach is based on the concept that after resolving an exit and re-entry scenario (i.e., re-identifying the second object as the first object using an adjusted re-identification threshold), the tracking of the first object can proceed using the original value of the re-identification threshold.

[0017] In some embodiments, re-identifying the second object as the first object includes comparing a set of second object features of the second object (i.e., features of the second object extracted from a video frame including the second object) with a set of first object features of the first object (i.e., features of the first object extracted from a video frame including the first object) to generate a matching score, and comparing the matching score with an adjusted re-identification threshold. By generating a matching score between the first object features and the second object features in this way, it becomes possible to evaluate the re-identification using a threshold test.

[0018] The matching score can be defined such that the matching score increases as the set of first object features and the set of second object features become more similar (e.g., the matching score increases as the distance between the set of first object features and the set of second object features decreases). In this case, the re-identification threshold can be adjusted by being raised. Thus, the second object can be re-identified as the first object in response to the matching score meeting or exceeding the adjusted (raised) re-identification threshold.

[0019] Alternatively, the matching score can be defined such that the more similar the first set of object features and the second set of object features are, the lower the matching score (e.g., the matching score decreases as the distance between the first set of object features and the second set of object features decreases). In this case, the re-identification threshold can be adjusted by being lowered. Thus, the second object can be re-identified as the first object in response to the matching score satisfying or falling below the adjusted (lowered) re-identification threshold.

[0020] In some embodiments, tracking the first object includes maintaining a tracking context for the first object, and re-identifying the second object as the first object includes associating the second object with the tracking context of the first object.

[0021] Thus, the "re-identification" of the second object as the first object can be easily achieved by associating the second object with the existing tracking context of the first object. Thus, in a sense, the second object can inherit previously determined tracking data (e.g., a tracklet) about the first object.

[0022] In some embodiments, the entry of the second object at the source is detected within a predetermined time after detecting that the first object has left the scene at the sink. That is, a further condition for re-identifying the second object as the first object can be that the second object enters the scene at the source within a predetermined time after the exit of the first object at the sink.

[0023] In some embodiments, following the determination that a time exceeding a predetermined time has elapsed after detecting that the first object has left the scene at the sink, the method Detecting the entry of a second object at the source, using a re-identification algorithm and a re-identification threshold associated with the second object to track the second object as a different object from the first object, and further comprising.

[0024] Therefore, if it is determined that a time exceeding a predetermined time has elapsed since detecting that the first object has left the scene at the sink, it may no longer be appropriate to attempt to re-identify a new object entering the scene at the source as the first object. In this case, the method may simply determine that the second object should be tracked as a new object. In embodiments where tracking the first object includes maintaining a tracking context for the first object, the method may further include removing the tracking context of the first object from the set of active tracking contexts maintained by the re-identification algorithm in response to determining that a time exceeding a predetermined time has elapsed since detecting that the first object has left the scene at the sink. Thus, by ending the tracking of the first object, both computational resources and memory resources can be conserved.

[0025] Furthermore, as described above, a scenario may occur where the second object is not sufficiently similar to the currently tracked first object that is to be re-identified as the first object. Thus, in some embodiments, the method continues by adjusting the re-identification threshold associated with the first object, detecting the entry of a second object at the source, using the re-identification algorithm and the adjusted re-identification threshold to determine that the second object is different from the first object, and Subsequently, using the re-identification algorithm and a re-identification threshold associated with the second object, track the second object as an object different from the first object further includes.

[0026] The second re-identification threshold can usually be set to the same value as the (unadjusted) re-identification threshold associated with the first object, for example, a predetermined default re-identification threshold.

[0027] In some embodiments, the method includes determining the position of the set of sources where the object enters the scene, the source being one of the sources in the set of sources, and following the adjustment of the re-identification threshold, the probability that the re-identification algorithm re-identifies an object entering the scene at any one of the set of sources as the first object is reduced.

[0028] That is, it is sufficient to adjust and maintain a single common re-identification threshold associated with the first object, which can be used by the re-identification algorithm regardless of which source among some sources of the scene the second object enters.

[0029] In some embodiments, the method, following adjusting the re-identification threshold associated with the first object, detects the entry of the second object at any one of the set of sources, uses the adjusted re-identification threshold by the re-identification algorithm to re-identify the second object as the first object, and subsequently continues to track the second object as the first object further includes.

[0030] A scenario that can be solved by re-identifying a second object that enters the scene in any one of the source sets and is sufficiently similar to the first object currently being tracked as being the first object so that the tracking of the first object can continue.

[0031] In some embodiments, the sink is the first sink, the method further includes determining the location of a second sink where the object exits the scene, and the method tracking a third object moving within the scene using a re-identification algorithm, the third object being associated with a corresponding re-identification threshold of the re-identification algorithm, and tracking the third object; detecting that the third object has exited the scene at the second sink; ending the tracking of the third object in response to detecting that the third object has exited the scene at the second sink, so as to prevent the re-identification that a fourth object entering the scene at the source is the third object further includes.

[0032] Accordingly, the method recognizes that some scenes may include a sink (the "second sink") such that after an object (the "third object") exits the scene at the second sink, the probability that the third object re-enters the scene at the source is substantially zero. Accordingly, computational resources can be conserved by ending the tracking of the third object.

[0033] In some embodiments, tracking the third object includes maintaining a tracking context for the third object, and ending the tracking of the third object includes removing the tracking context of the third object from the set of active tracking contexts maintained by the re-identification algorithm. Accordingly, memory resources for maintaining the tracking context of the third object can be freed.

[0034] According to a second aspect of the present invention, there is provided a computer program product including a computer program code portion configured to, when executed by a processing device, execute a method of tracking an object within a video sequence of a scene according to any method of the first aspect or an embodiment thereof.

[0035] According to a third aspect of the present invention, there is provided an object tracking system. The object tracking system at least one camera for capturing a video sequence of a scene, and a processing device configured to track an object within the video sequence according to any method of the first aspect or an embodiment thereof and.

[0036] The second and third aspects are characterized by the same or equivalent advantages as the first aspect. Any function described in relation to the first aspect may have a corresponding feature in the system, and vice versa.

[0037] Next, this aspect and other aspects of the present invention will be described in more detail with reference to the accompanying drawings showing embodiments of the present invention.

Brief Description of the Drawings

[0038]

Figure 1

Figure 2

Figure 3

Figure 4

Mode for Carrying Out the Invention

[0039] FIG. 1 schematically shows an exemplary view of Scene 1 monitored by a camera. This view may correspond to a video frame of a video sequence captured by the camera. At the time shown in FIG. 1, Scene 1 includes several objects in the form of vehicles moving around within Scene 1, and each object is being tracked by a re-identification algorithm.

[0040] Scene 1 includes several sources 2a - c and several sinks 4a - c. The respective numbers of sources and sinks in FIG. 1 are merely non-limiting examples, and it should be noted that the scene may more generally include any number of sources and sinks, but may include at least one source and at least one sink. Each source 2a - c represents an area of Scene 1 where an object may enter Scene 1. Conversely, each sink 4a - c represents an area of Scene 1 where an object may exit Scene 1. Thus, each source 2a - c represents a possible entry point for an object into Scene 1, and each sink 4a - c represents a possible exit point for an object from Scene 1.

[0041] In the illustrated example, each of the sources 2a - c and sinks 4a - c is arranged along each lane of the road segment that enters or exits the monitored view of Scene 1. The sources and sinks may be arranged adjacent to each other along respective road lanes as shown, for example, in FIG. 1, sources 2a and sink 4a, sources 2b and sink 4b, and sources 2c and sink 4c.

[0042] At the point in time represented by FIG. 1, one of the objects being tracked, e.g., vehicle 10a, has just entered scene 1 at source 2c and is moving towards sink 4a. Using re-identification, vehicle 10a can be tracked across successive video frames as it moves through scene 1. Vehicle 10a can be temporarily occluded, either completely or partially, while in motion, by another object being tracked (e.g., another vehicle), or by a stationary obstacle or structure (e.g., a tree canopy, a traffic signal, or a building). Here, the use of re-identification enables vehicle 10a to be re-identified as the same object before and after an occlusion event, such that vehicle 10a can be associated with the currently identified object and its tracking information (e.g., a tracklet that records the previous coordinates of the object within the frame).

[0043] More specifically, the re-identification algorithm can be implemented by extracting, from several consecutive video frames of a video sequence, a set of object features of the objects detected within the video frames. The object features extracted from a given video frame can be compared with the object features extracted from one or more preceding video frames in order to generate a matching score. The re-identification algorithm can perform a threshold test that includes comparing the matching score with a re-identification threshold associated with each tracked object. In response to the matching score passing the threshold test, the object features extracted from the given frame and one or more preceding frames may be determined to belong to the same (physical) object, and thus, an object within a given video frame may be re-identified as such. The position of the re-identified object within a given frame can thus be recorded within the tracklet associated with the object. If the matching score fails the threshold test, the object features extracted from an object within a given video frame may be determined to be a different object than the objects identified within one or more preceding video frames, and thus, its position within the given frame may not be associated with a previously tracked object.

[0044] For each object being tracked, a respective tracking context can be maintained. The tracking context may include an object identifier and a tracklet that defines the tracking information the object has traveled. When a new object is detected and identified within the video sequence, a new tracking context can be created. At some point, when the tracking of an object should be terminated, further attempts to re-identify the detected object using the currently tracked object may be stopped, and the update of the tracklet of the tracking context associated with the currently tracked object may be stopped.

[0045] The re-identification algorithm may maintain a set of active or current tracking contexts, each associated with a respective object currently being tracked by the re-identification algorithm. Each time a new object is detected and identified by the re-identification algorithm within a video sequence, a new tracking context may be created and added to the set of active tracking contexts. When the tracking of a given object associated with a tracking context in the set of active tracking contexts ends, the associated tracking context may be removed from the set of active tracking contexts. Thereby, re-identification of a new object subsequently entering the scene at the source as the given object may be prevented (i.e., because the tracking context associated with the given object has been removed from the set of active tracking contexts). Tracking may end in response to determining that a given object has left the scene at the sink, such that the probability of the object re-entering the scene at the source is substantially zero. Tracking may also end in response to determining that a time period exceeding a predetermined time period has elapsed since detecting that an object has left the scene at the sink.

[0046] Video sequences of scenes such as scene 1 are exposed to dynamic conditions such as changing lighting conditions, varying weather, and visibility conditions. Accordingly, each object to be tracked is associated with a re-identification threshold that can be set to provide some tolerance for the changing appearance of the object being tracked. The corresponding re-identification threshold can be included in or stored in the tracking context of the object being tracked. For example, the lighting conditions within a scene may change from sunny to cloudy over several video frames, resulting in a change in the apparent color of an object being tracked such as vehicle 10a. As another example, an object may move in and out of an area illuminated by streetlights. To ensure that an object such as vehicle 10a can be tracked reliably, during such changes, the re-identification threshold can be set accordingly to a value such that the re-identification algorithm can still re-identify vehicle 10a as the same object despite any change in the apparent color of vehicle 10a.

[0047] However, the tolerance provided by the re-identification algorithm and the re-identification threshold can cause problems when the object being tracked exits the scene at the sink. For example, in FIG. 1, consider a vehicle 10b attempting to exit scene 1 at sink 4a. After the vehicle 10b exits the scene at sink 4a, if a different vehicle of a similar type, model, and / or color (i.e., within the tolerance defined by the re-identification threshold associated with the vehicle 10b) enters scene 1 at a source such as the adjacent source 2a, there is a risk that the re-identification algorithm will re-identify this new different vehicle as the same object as vehicle 10b and thus continue to track the new vehicle as the currently tracked vehicle 10b (e.g., record the further detected position of the new vehicle within the tracklet of the tracking context of the currently tracked vehicle 10b). On the other hand, in some examples such as the adjacent source 2a and sink 4a, the vehicle 10b can make a U-turn after exiting scene 1 at sink 4a and thus re-enter scene 1 at source 2a. In this case, it is desirable for the vehicle 10b to be re-identified as the same object and continuously tracked when it enters scene 1 at source 2a.

[0048] These seemingly conflicting goals can be addressed, as described in the present disclosure, by adjusting the re-identification threshold associated with the first tracked object in response to detecting that the first tracked object has exited the scene at the sink, such that the probability that the re-identification algorithm re-identifies a second object entering the scene at the source after the first tracked object has exited the scene at the sink as the first object is reduced. In other words, the tolerance of the re-identification algorithm when attempting to re-identify the second object as the first tracked object can be reduced such that a greater visual similarity between the second object and the first object is required for the re-identification algorithm to re-identify the second object as the first tracked object.

[0049] Hereinafter, with reference to FIG. 1 and further with reference to FIGS. 2 to 4, exemplary implementations of the present method will be described in more detail.

[0050] FIG. 1 shows an example of a vehicle tracking application, but the positions and types of the source and sink change according to the scene being tracked and the type of object, as may be understood. For example, when additionally or alternatively tracking pedestrians, the source 6a and sink 8a for pedestrians may be located along a paved or walking path, e.g., a paved part of the path where it enters or exits the scene. The sources 6b and sinks 8b show an example of overlapping sources and sinks. It should be noted that the following disclosure is equally applicable to other tracking applications as well as other configurations and combinations of sources and sinks.

[0051] FIG. 2 shows an exemplary implementation of an object tracking system 20. The system 20 includes a video surveillance camera 22 for capturing a video sequence V of a scene such as scene 1. FIG. 1 shows a single view scene and FIG. 2 shows a single camera 22, but the system 20 may include a system of two or more cameras each monitoring a respective sub - view of the scene.

[0052] The system 20 further includes a processing device 24 and an associated memory 26. The memory 26 may be coupled to or included in the processing device 24. The processing device 24 is configured to receive the video sequence V in the form of a sequence of video frames from the camera 22. As shown in FIG. 2, the video sequence V may be stored in the memory 26, and the processing device 24 may retrieve and process video frames from the memory 26 to track objects within the video sequence over the sequence of video frames.

[0053] The processing device 24 may maintain, in the memory 26, a set of active tracking contexts including the respective tracking context of each object currently being tracked. FIG. 2 schematically shows two tracking contexts 28a - b, but the number of tracking contexts 28a - b depends on the number of objects currently being tracked. The tracking context may include a set of data fields, such as an object ID, a set of object features representing the object, a re - identification (ReID) threshold, and one or more of tracklets, as shown. The tracking context may optionally further include a timer field, which will be described in more detail below. The data fields of the tracking context may be updated by the processing device 24 during tracking, as will be further described below.

[0054] The object tracking method may be implemented in both hardware and software. In a software implementation, the processing device 24 may be realized in the form of one or more processors, such as one or more central processing units and / or graphical processing units, in association with computer program code instructions stored in a (non - transient) computer - readable medium such as non - volatile memory. Examples of non - volatile memory include read - only memory, flash memory, ferroelectric RAM, magnetic computer storage devices, optical disks, etc. In a hardware implementation, the processing device 24 may instead be realized by a dedicated circuit configured to implement the steps of the object tracking method. The circuit may be in the form of one or more integrated circuits, such as one or more application - specific integrated circuits (ASICs) or one or more field - programmable gate arrays (FPGAs). It is also possible to have a combination of a hardware implementation and a software implementation, which means that some method steps may be implemented in dedicated circuits and other method steps may be implemented in software.

[0055] FIG. 3 shows a flowchart of an exemplary implementation of an object tracking method.

[0056] In step S1, the processing device 24 determines the respective positions of at least one sink (e.g., one or more of sinks 4a - c) within scene 1 where the object may exit scene 1, and the respective positions of at least one source (e.g., one or more of sources 2a - c) where the object may enter scene 1. The positions of sources 2a - c and sinks 4a - c can be determined, for example, by the operator manually indicating via a graphical user input interface where sources 2a - c and sinks 4a - c are located within scene 1. However, the processing device 24 may also perform an automatic determination of the respective positions of one or more sources and sinks, for example, using an image recognition algorithm. As a non - limiting example, the image recognition algorithm can be configured or trained to identify road lanes within scene 1 and the entry and exit points of the road lanes at the periphery of scene 1.

[0057] In step S2, the processing device 24 uses a re - identification algorithm to track one or more objects detected within scene 1. For example, when an object such as vehicle 10a enters scene 1 at a source such as source 2c, the processing device 24 attempts to determine whether the detected object (e.g., vehicle 10a) corresponds to an object that is already being tracked. Thus, the processing device 24 extracts a set of object features from the video frame in which the object (e.g., vehicle 10a) is detected at source 2c and compares the set of object features with each set of object features associated with one or more currently tracked objects (e.g., each object for which a respective tracking context 28a - b is maintained by the processing device 24).

[0058] To generate a matching score, processing device 24 may determine the distance (e.g., Euclidean distance, or in some cases another distance suitable for comparing sets of object features that is multi-dimensional) between a set of object features of a newly detected object and a set of object features of the currently tracked object. The extracted object features may generally include visual features and latent (hidden) features. Non-limiting examples of object features include feature vectors extracted by a convolutional neural network (CNN), color histograms, and computer vision feature descriptors such as Histogram of Oriented Gradients (HOG) or Speeded-Up Robust Features (SURF). Processing device 24 may compare the matching score to a re-identification threshold associated with the currently tracked object (e.g., stored in the tracking context of each currently tracked object). The matching score may be defined to increase (e.g., monotonically, typically completely monotonically) with an increase in the similarity (e.g., decrease in distance) between the compared sets of object features, and the newly detected object may be re-identified as the currently tracked object in response to a matching score that meets or exceeds the re-identification threshold. Alternatively, the matching score may be defined to decrease (e.g., monotonically, typically completely monotonically) with an increase in the similarity (e.g., decrease in distance) between the compared sets of object features, and the newly detected object may be re-identified as the currently tracked object in response to a matching score that meets or falls below the re-identification threshold. In either case, upon passing the threshold test, processing device 24 may proceed to track the object (e.g., vehicle 10a) in subsequent frames as the currently tracked object, e.g., by recording the successive positions of the object within the tracklet of each tracking context (e.g., tracking context 28a).

[0059] Vehicle 10b is an example of an object (hereinafter, interchangeably referred to as the "first object") being tracked by the processing device 24 using a re-identification algorithm over a series of preceding video frames at the time shown in FIG. 1. Thus, the processing device 24 may maintain, in the memory 26, the tracking context 28b of the first object 10b at the time shown in FIG. 1 (i.e., the tracking context 28b is maintained within the set of active tracking contexts 28a - b). The first object 10b may enter the scene, for example, at source 2b or source 2c. The first object 10b is attempting to leave scene 1 at sink 4a as shown in FIG. 1. Thus, in step S3, the processing device 24 detects that the first object 10b has left the scene at sink 4a within a video frame subsequent to the frame shown in FIG. 1.

[0060] In step S4, in response to detecting that the first object 10b has left the scene at the sink, the processing device 24 adjusts the re-identification threshold associated with the first object 10b (e.g., the ReID threshold of the tracking context 28b). The re-identification threshold may be set to a default re-identification threshold before being adjusted. If the matching score is defined to increase with an increase in similarity, the re-identification threshold may be raised by applying an additional offset to the default value of the re-identification threshold or by scaling the default value of the re-identification threshold by a factor greater than 1. If the matching score is defined to decrease with an increase in similarity, the re-identification threshold may be lowered by applying a subtractive (negative) offset to the default value of the re-identification threshold or by scaling the default value of the re-identification threshold by a factor less than 1.

[0061] The default value of the re-identification threshold and the amount by which the re-identification threshold is adjusted each represent design parameters whose values can be established based on prior knowledge such as the type of object being tracked, the scene, the positions of the source and sink, the tolerance required to enable reliable re-identification of the object being tracked, and what risks are acceptable, if any, to re-identify a similar but different object as the object that exited the scene at the sink. The matching score used by the re-identification algorithm can typically be determined as a normalized value. Thus, the matching score and the re-identification threshold can be within the range [0, 1]. If defined such that the matching score increases when object features are more similar, the default value of the re-identification threshold can be, by way of non-limiting example, within the range of 0.6 to 0.7. If defined such that the matching score decreases when object features are more similar, the default value of the re-identification threshold can be within the range of 0.3 to 0.4. In either case, the re-identification threshold can be adjusted (increased or decreased) by, for example, 10 to 30%. Thus, in either case, the re-identification threshold is adjusted such that the probability that the re-identification algorithm re-identifies a second object entering the scene at the source after the first object exits the scene at the sink as the first object is reduced.

[0062] After adjusting the re-identification threshold associated with the first object 10b, the processing device 24 may, in S5, detect the entry of a second object into scene 1 at the source. The second object may be detected at any one of the scene sources 2a-c, for example at source 2a adjacent to the sink 4a.

[0063] In response to detecting a second object at the source, at S6, processing device 24 attempts to re-identify the second object as the first object 10b using a re-identification algorithm and the raised re-identification threshold. As described above, processing device 24 may compare a set of second object features of the second object with a set of first object features of the first object 10b to generate a matching score. Processing device 24 may perform a threshold test that includes comparing the matching score with an adjusted re-identification threshold. In response to the matching score passing the threshold test (e.g., in the case of a raised re-identification threshold, the matching score meets or exceeds the raised re-identification threshold, or in the case of a lowered re-identification threshold, the matching score meets or falls below the lowered re-identification threshold), the second object is re-identified as the first object 10b, and the method proceeds along the "yes" branch of FIG. 3. In response to the matching score failing the threshold test (e.g., in the case of a raised re-identification threshold, the matching score is less than the raised re-identification threshold, or in the case of a lowered re-identification threshold, the matching score exceeds the lowered re-identification threshold), the second object is determined to be different from the first object 10b, and the method proceeds along the "no" branch of FIG. 3.

[0064] If the method proceeds along the "yes" branch, after re-identifying the second object as the first object 10b, at step S7, processing device 24 restores the adjusted re-identification threshold to its value before adjustment, e.g., a default value. The re-identification threshold may be restored by subtracting an additional offset, adding a subtracted offset, or re-scaling a raised re-identification threshold by the reciprocal of the scaling factor described above.

[0065] Subsequently, in S8, the processing device 24 may continue to track the second object as the first object 10b using the restored re-identification threshold.

[0066] If the method proceeds to the "no" branch, the processing device 24 proceeds in S9 by tracking the second object as an object different from the first object 10b. Assuming that the processing device 24 fails to re-identify the second object as any currently tracked object, the second object may be tracked as a new object. For example, as described with reference to the entry of the object 10a in the source 2c of FIG. 1, a new tracking context associated with the second object is created and added to the set of active tracking contexts.

[0067] Thus, the method described above may reduce the risk of erroneously continuing to track a second object that is likely to correspond to a physical object different from the tracked object 10b by adjusting the re-identification threshold. Therefore, for successful re-identification, a greater visual similarity between the second object and the first object 10b is required. However, if the second object and the first object 10b are similar enough to generate a matching score that passes the threshold test using the adjusted re-identification threshold (in which case the second object is more likely to correspond to the same physical object as the first object 10b), the second object may still be re-identified as the first object 10b and accordingly continuously tracked.

[0068] To avoid attempting to re-identify an object entering scene 1 in the source as the first object 10b indefinitely, an additional condition for re-identifying a second object as the first object 10b can be applied, that is, the second object enters scene 1 in the source within a predetermined time from the exit of the first object 10b at the sink. If this additional condition is met, the method can proceed according to the "yes" branch. If this additional condition is not met, the method can proceed according to the "no" branch. The processing device 24 may, for example, maintain timers (the "timer" fields in FIG. 2) for each tracking context 28a - b. In response to detecting that the first object 10b has exited scene 1 at the sink 4a, the processing device 24 may start increasing the value of the timer. When the timer reaches a predetermined limit time, if a second object entering the scene is not re-identified as the first object 10b, the tracking of the first object 10b may be terminated, and the tracking context 28b of the first object 10b may be removed from the set of active tracking contexts. If the first object 10b later re-enters scene 1, the first object 10b can be detected, identified, and tracked as a new object, for example, using a new tracking context. The value of the predetermined time may be based on scene knowledge. For example, the predetermined time may be set according to the type of the tracking application, the specific types and positions of the source and sink within the scene, etc. The predetermined time may also be set considering several tracking contexts that can be maintained by the available computing resources of the object tracking system 20.

[0069] FIG. 4 is a flowchart of any extension of the method of FIG. 3, which can be executed in parallel with the method steps of FIG. 3.

[0070] As described above, the scenario may include a pair of source and sink where the object that exits the scene at the sink has a substantially zero possibility of immediately re-entering the scene at the source, or is at least too small to motivate an attempt at re-identification. The method of FIG. 4 enables handling of this scenario.

[0071] In S10, the processing device 24 determines the position of the second sink where the object exits the scene. Referring to FIG. 1, for the source 2a, the second sink may be represented, for example, by the sink 4b.

[0072] In S11, the processing device 24 uses a re-identification algorithm and a corresponding re-identification threshold associated with the third object to track the "third object" moving within scene 1. Referring to FIG. 1, the third object may be represented, for example, by a vehicle or a large truck 10c. The third object 10c is attempting to exit scene 1 at the second sink 4b as shown in FIG. 1. Thus, in step S12, the processing device 24 detects that the third object 10c has exited the scene at the second sink 4c within the video frame following the frame shown in FIG. 1.

[0073] In response to detecting that the third object 10c has exited scene 1 at the second sink 4b, in S13, the processing device 24 terminates the tracking of the third object 10c so that re-identification of a fourth object that may enter scene 1 at any one of the sources 2a - c as being the third object is prevented. As described above, terminating the tracking may include removing the tracking context associated with the third object from the set of active tracking contexts maintained by the re-identification algorithm.

[0074] The position of the sink 4b can be determined manually by an operator or automatically using an image recognition algorithm as described above. The determination to designate the sink 4b as the "second sink" at which tracking ends may be based on scene knowledge. For example, in the exemplary scene 1, knowledge that a U-turn along the road segment following the sink 4b is prohibited or even impossible may be considered.

[0075] Those skilled in the art will understand that the present invention is in no way limited to the above-described embodiments. On the contrary, many modifications and variations are possible within the scope of the appended claims.

Claims

1. 1. A computer-implemented method for tracking an object in a video sequence of a scene, the method comprising: determining a sink location within the scene where an object exits the scene and a source location where an object enters the scene; tracking a first object moving within the scene using a re-identification algorithm, the first object being associated with a re-identification threshold of the re-identification algorithm; Detecting that the first object has exited the scene at the sink; and in response to detecting that the first object has exited the scene at the sink, adjusting the re-identification threshold associated with the first object such that the probability that the re-identification algorithm will re-identify a second object that enters the scene at the source after the first object has exited the scene at the sink is reduced. Including, Following adjusting the re-identification threshold associated with the first object, the method further comprises: detecting the ingress of the second object at the source; and in response to detecting the ingress of the second object at the source, attempting to re-identify the second object as the first object using the re-identification algorithm and the adjusted re-identification threshold, wherein attempting to re-identify the second object as the first object includes: comparing a set of second object features of the second object with a set of first object features of the first object to generate a matching score; and performing a threshold test by comparing the matching score to the adjusted re-identification threshold. In response to the matching score passing the threshold test, re-identifying the second object as the first object, thereafter restoring the re-identification threshold, and continuing to track the second object as the first object using the restored re-identification threshold; In response to the matching score failing the threshold test, determining that the second object is distinct from the first object, and then tracking the second object as a distinct object from the first object using the re-identification algorithm and a re-identification threshold associated with the second object; Including, the matching score is defined to increase as the set of first object features and the set of second object features become more similar, and the re-identification threshold is adjusted by increasing the re-identification threshold; or The method, wherein the matching score is defined to decrease as the set of first object features and the set of second object features become more similar, and the re-identification threshold is adjusted by lowering the re-identification threshold.

2. 2. The method of claim 1 , wherein tracking the first object includes maintaining a tracking context for the first object, and wherein re-identifying the second object as the first object includes associating the second object with the tracking context for the first object.

3. The method of claim 1 , wherein the entry of the second object at the source is detected within a predetermined time period after detecting that the first object has exited the scene at the sink.

4. The method of claim 1, wherein tracking the first object includes maintaining a tracking context for the first object, and the method further includes removing the tracking context for the first object from a set of active tracking contexts maintained by the re-identification algorithm in response to determining that more than a predetermined time has elapsed since the first object was detected at the sink to have exited the scene.

5. 2. The method of claim 1, wherein the method includes determining a location of a set of sources from which an object enters the scene, the source being one source in the set of sources, and wherein, following adjusting the re-identification threshold, the probability that the re-identification algorithm will re-identify an object entering the scene at any one of the set of sources as the first object is reduced.

6. The method described in claim 5, wherein the ingress of the second object is detected in any one of the sets of sources.

7. The sink is a first sink, and the method further includes determining a position of a second sink at which the object exits the scene, the method comprising: using the re-identification algorithm to track a third object moving within the scene, the third object being associated with a corresponding re-identification threshold of the re-identification algorithm; detecting that the third object has exited the scene at the second sink; in response to detecting that the third object has exited the scene at the second sink, terminating tracking of the third object such that a fourth object entering the scene at the source is prevented from being re-identified as the third object; The method of claim 1 further comprising:

8. 8. The method of claim 7, wherein tracking the third object includes maintaining a tracking context for the third object, and terminating tracking of the third object includes removing the tracking context for the third object from a set of active tracking contexts maintained by the re-identification algorithm.

9. The method of claim 1 , wherein the source and the sink are adjacent to or overlap each other.

10. at least one camera for capturing a video sequence of the scene; a processing device configured to track an object in the video sequence according to the method of any one of claims 1 to 9; An object tracking system comprising:

11. 10. A non-transitory computer readable storage medium comprising computer program code portions configured, when executed by a processing device, to perform the method for tracking an object in a video sequence of a scene according to any one of claims 1 to 9.

Images