Method of tracking object

Abstract

The invention provides a method for tracking an object. According to an aspect, there is provided a computer-implemented method of tracking an object in a video sequence of a scene, the method comprising: determining a location in the scene at which the object leaves a sink of the scene and a location at which the object enters a source of the scene; tracking a first object moving in the scene using a re-recognition algorithm, where the first object is associated with a re-recognition threshold of the re-recognition algorithm; detecting that the first object has left the scene at the sink; and in response to detecting that the first object has exited the scene at the sink, adjusting a re-recognition threshold associated with the first object to reduce the probability that the re-recognition algorithm re-recognizes a second object as the first object, the second object entering the scene at the source after the first object has exited the scene at the sink.

Description

Technical Field

[0001] The present invention generally relates to a computer - implemented method for tracking objects in a video sequence of a scene. Background Art

[0002] A common application of video surveillance systems is to track objects moving in a monitored scene. One computer - vision - based object - tracking technique is re - identification (“ReID”), in which an object of interest (e.g., a person, a vehicle, an animal) is identified in one frame and then re - identified in consecutive frames. The trajectory of the object (e.g., the coordinates of the object in the frame) can be recorded in a “tracklet” maintained for the object.

[0003] Re - identification has a wide range of applications, including surveillance, traffic monitoring, and crowd analysis. More generally, re - identification can be useful in cases where the object being tracked may be temporarily occluded during tracking. If the object after occlusion (partially or completely) can be re - identified as the same object as before occlusion, then the re - identified object after occlusion (and its trajectory) can be associated with the object and trajectory identified before occlusion. When tracking an object moving from a first camera view of a scene to a second camera view of the scene, corresponding methods can be used. Summary of the Invention

[0004] The inventors have realized that in tracking scenarios involving tracking objects with similar appearances (e.g., vehicles with similar makes, models, and / or colors or people with similar clothes and / or appearances), when one of the similar objects has left the scene at a sink and a similar but different object subsequently appears at a source, using re - identification as conventionally implemented introduces a risk of inadvertently identifying similar but different objects (e.g., different cars of the same model and similar color) as the same object. The aim of the present invention is to provide a method for tracking objects using re - identification that mitigates this risk.

[0005] Thus, according to a first aspect of the present invention, there is provided a computer - implemented method for tracking objects in a video sequence of a scene, the method comprising:

[0006] Determining the location of a sink where an object leaves the scene and the location of a source where the object enters the scene;

[0007] Tracking a first object moving in the scene using a re - identification algorithm, wherein the first object is associated with a re - identification threshold of the re - identification algorithm;

[0008] Detecting that the first object has left the scene at the sink; and

[0009] In response to detecting that a first object has left the scene at a sink, adjust the re-identification threshold associated with the first object to reduce the probability that a re-identification algorithm re-identifies a second object as the first object, where the second object enters the scene at a source after the first object has left the scene at the sink.

[0010] By adjusting the re-identification threshold, a greater similarity between the second object and the first object is required for the re-identification algorithm to re-identify the second object as the first object. Thus, the method is able to reduce the risk of inadvertently re-identifying as the tracked object an object that enters the scene at the source and is visually similar to the tracked object that has previously left the scene at the sink, but corresponds to a different physical object from the tracked object.

[0011] In the case of temporarily occluding an object moving through the scene, similar objects detected in the video sequence before and after the occlusion may correspond to the same physical object. Thus, in this case, re-identification is appropriate and desirable. However, the likelihood that a physical object re-enters the scene soon after leaving the scene at the sink is typically low. However, some scenes include a source and a sink with relative positions such that it is still possible for the same physical object to re-enter at the source after a period of time has elapsed since the physical object left the scene at the sink. Thus, a simple approach of directly terminating the tracking of an object when the tracked object is detected leaving at the sink does not correctly handle such leave and re-enter situations. That is, identifying the re-entering object as a new object may result in the loss of potentially relevant tracking information. In contrast, by adjusting the re-identification threshold associated with the first object, rather than directly terminating the tracking of the leaving object, the method of the present invention allows for the correct re-identification of an object that re-enters at the source and that has previously left the scene at the sink.

[0012] Thus, the method is able to improve the reliability of object tracking, particularly in cases where the tracking may involve visually similar objects. Additionally, improved reliability is provided without sacrificing the benefits associated with re-identification, namely, handling temporary occlusions of the tracked object and movement of the tracked object between different camera views of the scene.

[0013] The further utility of the method of the present invention may be particularly evident when tracking an object in a scene that includes a source and a sink that are adjacent to or overlap with each other.

[0014] The sink and the source may be arranged (e.g., in a vehicle tracking application) along respective lanes of a road, at the exit and entrance of a parking space, respectively, or along a road at an exit ramp and an entrance ramp, respectively.

[0015] Huiheyuan may also be located (for example, in an application for tracking individuals such as pedestrians and / or cyclists) at a part of a walking path or sidewalk leading from the scene or a corresponding adjacent part, or respectively at the exit and entrance of a building, an indoor space (such as a room) or an outdoor space (such as a park).

[0016] The "re-identification threshold" here refers to the corresponding threshold associated with each respective tracked object, which is used to re-identify an object detected in a video sequence frame as the corresponding tracked object through a re-identification algorithm. That is, in order to re-identify an object detected in the second frame as the object identified in the previous first frame, the object features extracted from the second frame need to match the object features extracted from the first frame to the extent defined by the re-identification threshold.

[0017] The term "object" (such as "first object", "second object", etc.) here refers to the depiction of a physical object in the scene (more specifically, in one or more frames of a video sequence), where the physical object is of the type being tracked, such as a vehicle or an individual. Correspondingly, the term "physical object" refers to the actual physical object moving in the monitored scene. Thus, from the previous discussion, it can be understood that two "objects" (such as "first object" and "second object") in a video sequence may depict the same physical object or different physical objects in the scene depending on the scene.

[0018] In some embodiments, the method further includes, after adjusting the re-identification threshold associated with the first object:

[0019] detecting the entry of the second object at the source;

[0020] re-identifying the second object as the first object by using the re-identification algorithm with the adjusted re-identification threshold; and

[0021] subsequently continuing to track the second object as the first object.

[0022] Therefore, by re-identifying the second object as the first object, situations where the second object is similar enough to the currently tracked first object (especially when the first object and the second object actually correspond to the same physical object) can be resolved, so that the first object can continue to be tracked.

[0023] In some embodiments, the method further includes, after re-identifying the second object as the first object, restoring the re-identification threshold and using the restored re-identification threshold to continue tracking the second object as the first object. "Restoring" the re-identification threshold here means restoring or reverting the re-identification threshold to the value before adjustment, e.g., a predetermined default re-identification threshold. The method is based on the concept that after resolving the leaving and re-entering scenarios (i.e., re-identifying the second object as the first object using the adjusted re-identification threshold), the original value of the re-identification threshold can be used to continue tracking the first object.

[0024] In some embodiments, re-identifying the second object as the first object includes comparing a second object feature set of the second object (i.e., second object features extracted from a video frame including the second object) with a first object feature set of the first object (i.e., first object features extracted from a video frame including the first object) to generate a matching score, and comparing the matching score with the adjusted re-identification threshold. Generating a matching score between the first object features and the second object features in this way enables the use of a threshold test to evaluate the re-identification.

[0025] The matching score can be defined such that the more similar the first object feature set and the second object feature set are, the higher the matching score (e.g., the matching score increases as the distance between the first object feature set and the second object feature set decreases). In this case, the re-identification threshold can be adjusted by increasing. Thus, in response to the matching score reaching or exceeding the adjusted (increased) re-identification threshold, the second object can be re-identified as the first object.

[0026] Alternatively, the matching score can be defined such that the more similar the first object feature set and the second object feature set are, the lower the matching score (e.g., the matching score decreases as the distance between the first object feature set and the second object feature set decreases). In this case, the re-identification threshold can be adjusted by decreasing. Thus, in response to the matching score reaching or falling below the adjusted (decreased) re-identification threshold, the second object can be re-identified as the first object.

[0027] In some embodiments, tracking the first object includes maintaining a tracking context of the first object, and wherein re-identifying the second object as the first object includes associating the second object with the tracking context of the first object.

[0028] Thus, by associating the second object with the existing tracking context of the first object, simply "re-identifying" the second object as the first object can be achieved. Thus, in a sense, the second object can inherit the previously determined tracking data (e.g., trajectory segments) of the first object.

[0029] In some embodiments, within a predetermined time from when it is detected that the first object has left the scene at the sink, the entry of a second object at the source is detected. That is, a further condition for re-identifying the second object as the first object may be that the second object enters the scene at the source within a predetermined time from when the first object leaves at the sink.

[0030] In some embodiments, the method further comprises, after determining that more than a predetermined time has elapsed since it was detected that the first object has left the scene at the sink:

[0031] detecting the entry of the second object at the source; and

[0032] using a re-identification algorithm and a re-identification threshold associated with the second object, tracking the second object as an object different from the first object.

[0033] Thus, if it is determined that more than a predetermined time has elapsed since it was detected that the first object has left the scene at the sink, it may no longer make sense to attempt to re-identify the new object entering the scene at the source as the first object. In such a case, the method can simply determine to track the second object as a new object. In embodiments where tracking the first object includes maintaining a tracking context of the first object, the method can further comprise, in response to determining that more than a predetermined time has elapsed since it was detected that the first object has left the scene at the sink, removing the tracking context of the first object from the set of active tracking contexts maintained by the re-identification algorithm. Thus, by terminating the tracking of the first object, both computational resources and memory resources can be conserved.

[0034] Furthermore, as described above, there may be cases where the second object is not similar enough to the currently tracked first object to be re-identified as the first object. Thus, in some embodiments, the method can further comprise, after adjusting the re-identification threshold associated with the first object:

[0035] detecting the entry of the second object at the source;

[0036] using the re-identification algorithm and the adjusted re-identification threshold, determining that the second object is different from the first object; and

[0037] subsequently using the re-identification algorithm and a re-identification threshold associated with the second object, tracking the second object as an object different from the first object.

[0038] The second re-identification threshold can generally be set to the same value as the (unadjusted) re-identification threshold associated with the first object, such as a predetermined default re-identification threshold.

[0039] In some embodiments, the method includes determining the location of a set of sources at which an object enters the scene, where the source is one of the set of sources, and where, after adjusting a re-identification threshold, the probability that a re-identification algorithm re-identifies an object entering the scene at any of the set of sources as a first object is reduced.

[0040] That is, it is sufficient to adjust and maintain a single common re-identification threshold associated with the first object, and the re-identification algorithm can use this threshold regardless of which source among the multiple sources of the scene the second object enters the scene at.

[0041] In some embodiments, the method further includes, after adjusting the re-identification threshold associated with the first object:

[0042] Detecting the entry of a second object at any of the set of sources;

[0043] Re-identifying the second object as the first object by using the re-identification algorithm with the adjusted re-identification threshold; and

[0044] Subsequently continuing to track the second object as the first object.

[0045] Thus, a situation where a second object that is sufficiently similar to the currently tracked first object enters the scene at any of the set of sources can be addressed by re-identifying the second object as the first object, so that the first object can continue to be tracked.

[0046] In some embodiments, the sink is a first sink, and the method further includes determining the location of a second sink at which the object leaves the scene, and the method further includes:

[0047] Tracking a third object moving in the scene by using a re-identification algorithm, where the third object is associated with a corresponding re-identification threshold of the re-identification algorithm;

[0048] Detecting that the third object has left the scene at the second sink; and

[0049] In response to detecting that the third object has left the scene at the second sink, terminating the tracking of the third object, thereby preventing a fourth object entering the scene at the source from being re-identified as the third object.

[0050] Thus, the method recognizes the fact that some scenes may include a sink (“second sink”) such that after an object (“third object”) leaves the scene at the second sink, the possibility of the third object re-entering the scene at the source is substantially zero. Therefore, computational resources can be conserved by terminating the tracking of the third object.

[0051] In some embodiments, tracking a third object includes maintaining a tracking context for the third object, and wherein terminating the tracking of the third object includes removing the tracking context of the third object from the set of active tracking contexts maintained by the re-identification algorithm. Thereby, the memory resources used to maintain the tracking context of the third object can be freed.

[0052] According to a second aspect of the present invention, there is provided a computer program product comprising a computer program code portion configured to perform, when run by a processing device, a method of tracking an object in a video sequence of a scene according to the method of the first aspect or any of its embodiments.

[0053] According to a third aspect of the present invention, there is provided an object tracking system comprising:

[0054] at least one camera for capturing a video sequence of a scene; and

[0055] a processing device configured to track an object in the video sequence according to the method of the first aspect or any of its embodiments.

[0056] The features of the second and third aspects have the same or equivalent benefits as the first aspect. Any function described with respect to the first aspect can have a corresponding feature in the system, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] This aspect and other aspects of the present invention will now be described in more detail with reference to the drawings showing embodiments of the present invention.

[0058] Figure 1 A view of a scene is schematically shown.

[0059] Figure 2 An implementation of an object tracking system is shown.

[0060] Figure 3 is a flowchart of a method of tracking an object in a video sequence of a scene.

[0061] Figure 4 is Figure 3 an extended flowchart of the method of. DETAILED DESCRIPTION

[0062] Figure 1 An example view of a scene 1 monitored by a camera is schematically shown. This view may correspond to a video frame of a video sequence captured by the camera. At Figure 1 the time point shown in, scene 1 includes a plurality of objects in the form of vehicles moving in scene 1, each object being tracked by a re-identification algorithm.

[0063] Scenario 1 includes multiple sources 2a-c and multiple sinks 4a-c. It should be noted that Figure 1 the corresponding number of sources and sinks in Figure 1 are only non-limiting examples, and the scenario can more generally include any number of sources and sinks, but includes at least one source and at least one sink. Each source 2a-c represents an area of Scenario 1 where or in which an object can enter Scenario 1. Conversely, each sink 4a-c represents an area of Scenario 1 where or in which an object can leave Scenario 1. Thus, each source 2a-c represents a possible entry point for an object to enter Scenario 1, while each sink 4a-c represents a possible exit point for an object to leave Scenario 1.

[0064] In the illustrated example, each of the sources 2a-c and sinks 4a-c is arranged along a respective lane of a section of the monitored view leading to or from Scenario 1. The sources and sinks can be arranged adjacent to each other along their respective road lanes as shown, for example in Figure 1 Figure 1 , source 2a and sink 4a, source 2b and sink 4b, and source 2c and sink 4c.

[0065] At Figure 1 the moment shown, one of the tracked objects, such as car 10a, has just entered Scenario 1 at source 2c and is moving towards sink 4a. As car 10a moves through Scenario 1, using re-identification, car 10a can be tracked in consecutive video frames. During the movement, car 10a may be temporarily completely or partially occluded by another tracked object (such as another car) or by a stationary obstacle or structure (such as a tree canopy, traffic light, or building). The use of re-identification here allows car 10a to be re-identified as the same object before and after the occlusion event, such that car 10a can be associated with the currently identified object and its trajectory (e.g., a fragment of the trajectory recording the previous coordinates of the object in the recorded frames).

[0066] More specifically, the re-identification algorithm can be implemented by extracting a set of object features of an object detected in a video frame from multiple consecutive video frames of a video sequence. The object features extracted from a given video frame can be compared with the object features extracted from one or more previous video frames to generate a matching score. The re-identification algorithm can perform a threshold test that includes comparing the matching score with a re-identification threshold associated with the corresponding tracked object. In response to the matching score passing the threshold test, it can be determined that the object features extracted from the given frame and one or more previous frames belong to the same (physical) object, and thus the object in the given video frame can be re-identified as such. Accordingly, the position of the re-identified object in the given frame can be recorded in the trajectory segment associated with the object. In the case where the matching score fails the threshold test, the object features extracted from the object in the given video frame can be determined to be a different object from the object identified in the previous one or more video frames, and thus its position in the given frame may not be associated with the previously tracked object.

[0067] A corresponding tracking context can be maintained for each tracked object. The tracking context can include an object identifier and a trajectory segment that defines the trajectory traveled by the object. When a new object is detected and identified in the video sequence, a new tracking context can be created. When it is desired to terminate the tracking of an object at a certain point, further attempts to re-identify the detected object with the currently tracked object can be stopped, and the trajectory segment of the tracking context associated with the currently tracked object can be stopped from being updated.

[0068] The re-identification algorithm can maintain a set of active or current tracking contexts, each tracking context associated with the corresponding object currently being tracked by the re-identification algorithm. Each time a new object is detected and identified in the video sequence by the re-identification algorithm, a new tracking context can be created and added to the set of active tracking contexts. Once the tracking of a given object associated with a tracking context in the set of active tracking contexts is terminated, the associated tracking context can be removed from the set of active tracking contexts. Thus, it can be prevented that a new object subsequently entering the scene at the source is re-identified as the given object (i.e., because the tracking context associated with the given object has been removed from the set of active tracking contexts). In response to determining that the given object has left the scene at the sink such that the likelihood of the object re-entering the scene at the source is substantially zero, the tracking can be terminated. The tracking can also be terminated in response to determining that more than a predetermined time has passed since it was detected that the object has left the scene at the sink.

[0069] A video sequence of a scene such as Scene 1 is affected by dynamic conditions (e.g., changing lighting conditions, variable weather and visibility conditions, etc.). Accordingly, each tracked object is associated with a re-identification threshold, which can be set to provide a certain tolerance for changes in the appearance of the tracked object. The corresponding re-identification threshold can be included or stored in the tracking context of the tracked object. For example, during the course of several video frames, the lighting conditions in the scene may change from sunny to cloudy, resulting in a significant change in the color of the tracked object (such as vehicle 10a). As another example, an object can move into and out of an area illuminated by a street lamp. To be able to reliably track an object, such as vehicle 10a, during such changes, the re-identification threshold can be set accordingly such that although there are some changes in the appearance color of vehicle 10a, the re-identification algorithm can still re-identify vehicle 10a as the same object.

[0070] However, once the tracked object leaves the scene at a sink, the tolerance provided by the re-identification algorithm and the re-identification threshold may cause problems. For example, consider Figure 1 vehicle 10b in [scene] is about to leave Scene 1 at sink 4a. If, after vehicle 10b leaves the scene at sink 4a, a different vehicle with a similar make, model, and / or color (i.e., similar within the tolerance defined by the re-identification threshold associated with vehicle 10b) enters Scene 1 at a source such as adjacent source 2a, there is a risk that the re-identification algorithm will re-identify this new different vehicle as the same object as vehicle 10b, and thus continue to track the new vehicle as the currently tracked vehicle 10b (e.g., record the further detection locations of the new vehicle in the trajectory segment of the tracking context of the currently tracked vehicle 10b). On the other hand, in some cases, such as for adjacent source 2a and sink 4a, vehicle 10b may turn around after leaving Scene 1 at sink 4a and thus enter Scene 1 again at source 2a. In this case, it is desirable that vehicle 10b is indeed re-identified and continuously tracked as the same object when it enters Scene 1 at source 2a.

[0071] As described in the present disclosure, these seemingly competing objectives can be addressed by: in response to detecting that a first tracked object has left the scene at a sink, adjusting the re-identification threshold associated with the first tracked object such that the probability that the re-identification algorithm re-identifies a second object that enters the scene at a source after the first tracked object has left the scene at the sink as the first object is reduced. In other words, the tolerance of the re-identification algorithm can be reduced when attempting to re-identify the second object as the first tracked object, so that the re-identification algorithm requires a greater visual similarity between the second object and the first object to re-identify the second object as the first tracked object.

[0072] Reference will be made below toFigure 1 and further referring to Figures 2 to 4 to discuss an example implementation of the method in more detail.

[0073] While Figure 1 an example of a vehicle tracking application is shown, it will be understood that the location and type of the source and sink will vary depending on the type of scenario and object being tracked. For example, additionally or alternatively, in the case of tracking a pedestrian, the source 6a and sink 8a of the pedestrian can be set along a sidewalk or walking path, such as at a part of the sidewalk or its walking path leading to or leaving the scenario. The source 6b and sink 8b illustrate an example of an overlapping source and sink. Accordingly, it should be noted that the following disclosure is equally applicable to other tracking applications and other configurations and combinations of sources and sinks.

[0074] Figure 2 an example implementation of an object tracking system 20 is shown. The system 20 includes a video surveillance camera 22 for capturing a video sequence V of a scenario (such as scenario 1). While Figure 1 a single-view scenario is shown, and Figure 2 a single camera 22 is shown, the system 20 can also include a system of two or more cameras, each camera monitoring a corresponding sub-view of the scenario.

[0075] The system 20 further includes a processing device 24 and an associated memory 26. The memory 26 can be coupled to the processing device 24 or included in the processing device 24. The processing device 24 is configured to receive the video sequence V in the form of a sequence of video frames from the camera 22. As Figure 2 shown, the video sequence V can be stored in the memory 26, where the processing device 24 can retrieve and process video frames from the memory 26 to track an object in the video sequence in the sequence of video frames.

[0076] The processing device 24 can maintain in the memory 26 an active set of tracking contexts including a corresponding tracking context for each currently tracked object. Figure 2 Two tracking contexts 28a - b are schematically indicated, but the number of tracking contexts 28a - b will depend on the number of currently tracked objects. A tracking context can include a set of data fields as shown, such as an object ID, a set of object features representing the object, a re-identification (ReID) threshold, and one or more of a trajectory segment. The tracking context can optionally further include a timer field, which will be described in more detail below. As will be described further below, the data fields of the tracking context can be updated by the processing device 24 during the tracking process.

[0077] Object tracking methods can be implemented in hardware and software. In a software implementation, the processing device 24 can be implemented in the form of one or more processors such as one or more central processing units and / or graphics processing units, which are associated with computer program code instructions stored on a (non-transitory) computer-readable medium (e.g., non-volatile memory), causing the processing device 24 to perform the steps of the object tracking method. Examples of non-volatile memory include read-only memory, flash memory, ferroelectric RAM, magnetic computer storage devices, and optical discs, etc. In a hardware implementation, the processing device 24 can instead be implemented by a dedicated circuit configured to implement the steps of the object tracking method. The circuit can be in the form of one or more integrated circuits, such as one or more application-specific integrated circuits (ASICs) or one or more field-programmable gate arrays (FPGAs). It should be understood that a combination of hardware and software implementations can also be had, meaning that some method steps can be implemented in dedicated circuits while other steps can be implemented in software.

[0078] Figure 3 A flowchart showing an example implementation of an object tracking method is shown.

[0079] At step S1, the processing device 24 determines the respective positions of at least one sink (e.g., one or more of sinks 4a-c) where an object in scene 1 can leave scene 1, and the respective positions of at least one source (e.g., one or more of sources 2a-c) where an object can enter scene 1. The positions of sources 2a-c and sinks 4a-c can be manually determined by an operator, which positions are indicated, for example, via a graphical user input interface as to where sources 2a-c and sinks 4a-c are in scene 1. However, the processing device 24 can also, for example, use an image recognition algorithm to achieve an automatic determination of the respective positions of one or more sources and sinks. As a non-limiting example, the image recognition algorithm can be configured or trained to recognize road lanes in scene 1 and the entry and exit points of the road lanes surrounding scene 1.

[0080] At step S2, the processing device 24 uses a re-identification algorithm to track one or more objects detected in scene 1. For example, when an object such as car 10a enters scene 1 at a source such as source 2c, the processing device 24 attempts to determine whether the detected object (e.g., car 10a) corresponds to an object that has been tracked. Accordingly, the processing device 24 extracts a set of object features from the video frame in which the object (e.g., car 10a) is detected at source 2c and compares this set of object features with the respective sets of object features associated with one or more currently tracked objects (e.g., the processing device 24 maintains a respective tracking context 28a-b for each object).

[0081] The processing device 24 can determine the distance (such as Euclidean distance or other distances suitable for comparing object feature sets, which may be multi-dimensional) between the object feature set of the newly detected object and the object feature set of the currently tracked object to generate a matching score. The extracted object features can generally include visual features and potential (hidden) features. Non-limiting examples of object features include feature vectors extracted by a convolutional neural network (CNN), color histograms, and computer vision feature descriptors such as histogram of oriented gradients (HOG) or speeded up robust features (SURF). The processing device 24 can compare the matching score with a re-identification threshold associated with the currently tracked object (e.g., stored in the tracking context of the corresponding currently tracked object). The matching score can be defined to increase (e.g., monotonically, usually strictly monotonically) as the similarity between the compared object feature sets increases (e.g., the distance decreases), where, in response to the matching score reaching or exceeding the re-identification threshold, the newly detected object can be re-identified as the currently tracked object. Alternatively, the matching score can be defined to decrease (e.g., monotonically, usually strictly monotonically) as the similarity between the compared object feature sets increases (e.g., the distance decreases), where, in response to the matching score reaching or falling below the re-identification threshold, the newly detected object can be re-identified as the currently tracked object. In either case, after passing the threshold test, the processing device 24 can continue to track the object (e.g., vehicle 10a) in subsequent frames as the currently tracked object, such as recording the successive positions of the object in the trajectory segment of the corresponding tracking context (e.g., tracking context 28a).

[0082] Vehicle 10b is an example of an object (hereinafter interchangeably referred to as the "first object") that has been tracked in the sequence of previous video frames by the processing device 24 at the moment shown in Figure 1 Accordingly, the processing device 24 can maintain the tracking context 28b of the first object 10b in the memory 26 at the moment shown in Figure 1 That is, the tracking context 28b is maintained in the set of active tracking contexts 28a-b. For example, the first object 10b has entered the scene at, for example, source 2b or source 2c. As shown in Figure 1 the first object 10b is about to leave the scene 1 at sink 4a. Accordingly, at step S3, the processing device 24 detects that in the video frame after the frame shown in Figure 1 the first object 10b has left the scene at sink 4a.

[0083] At step S4, the processing device 24 adjusts the re-identification threshold associated with the first object 10b (e.g., the ReID threshold of the tracking context 28b) in response to detecting that the first object 10b has left the scene at the sink. Before the adjustment, the re-identification threshold can be set to a default re-identification threshold. If the matching score is defined to increase as the similarity increases, the re-identification threshold can be increased by applying an additive offset to the default value of the re-identification threshold, or by scaling the default value of the re-identification threshold by a factor greater than 1. If the matching score is defined to decrease as the similarity increases, the re-identification threshold can be decreased by applying a subtractive (negative) offset to the default value of the re-identification threshold, or by scaling the default value of the re-identification threshold by a factor less than 1.

[0084] The default value of the re-identification threshold and the amount of adjustment of the re-identification threshold each represent design parameters, the values of which can be determined based on prior knowledge such as the type of the object being tracked, the scene, the positions of the source and the sink, the tolerance deviation required to achieve reliable re-identification of the object being tracked, and the acceptable risk of re-identifying similar but different objects as objects that have left the scene at the sink. The matching score used by the re-identification algorithm can generally be determined as a normalized value. Therefore, the matching score and the re-identification threshold can be in the range of [0, 1]. If the matching score is defined to increase when the object features are more similar, then as a non-limiting example, the default value of the re-identification threshold can be in the range of 0.6 - 0.7. If the matching score is defined to decrease when the object features are more similar, then the default value of the re-identification threshold can be in the range of 0.3 - 0.4. In either case, the re-identification threshold can be adjusted (increased or decreased) by 10% - 30%. In either case, the re-identification threshold is thus adjusted such that the probability that the re-identification algorithm re-identifies a second object entering the scene at the source after the first object has left the scene at the sink as the first object is reduced.

[0085] After adjusting the re-identification threshold associated with the first object 10b, at step S5, the processing device 24 can detect that a second object enters the scene 1 at the source. The second object can be detected at any one of the sources 2a - c of the scene (e.g., at the source 2a adjacent to the sink 4a).

[0086] In response to detecting a second object at the source, the processing device 24 attempts to re-identify the second object as the first object 10b using a re-identification algorithm and an increased re-identification threshold at step S6. As described above, the processing device 24 may compare the second object feature set of the second object with the first object feature set of the first object 10b to generate a matching score. The processing device 24 may perform a threshold test including comparing the matching score with the adjusted re-identification threshold. In response to a matching score passing the threshold test (e.g., in the case of an increased re-identification threshold, the matching score meets or exceeds the increased re-identification threshold, or in the case of a decreased re-identification threshold, the matching score meets or is lower than the decreased re-identification threshold), the second object is re-identified as the first object 10b, where the method proceeds along Figure 3 the "Yes" branch in. In response to the matching score failing the threshold test (e.g., in the case of an increased re-identification threshold, the matching score is less than the increased re-identification threshold, or in the case of a decreased re-identification threshold, the matching score exceeds the decreased re-identification threshold), it is determined that the second object is different from the first object 10b, where the method proceeds along Figure 3 the "No" branch in.

[0087] If the method continues to the "Yes" branch, after re-identifying the second object as the first object 10b, the processing device 24 restores the adjusted re-identification threshold to the value of the re-identification threshold before adjustment, e.g., the default value, at step S7. The re-identification threshold can be restored by subtracting an additive offset, adding a subtractive offset, or re-scaling the increased re-identification threshold by the reciprocal of the above scaling factor.

[0088] Subsequently, at step S8, the processing device 24 may continue to track the second object as the first object 10b using the restored re-identification threshold.

[0089] If the method advances to the "No" branch, the processing device 24 continues at step S9 by tracking the second object as an object different from the first object 10b. Assuming the processing device 24 fails to re-identify the second object as any currently tracked object, the second object may be tracked as a new object, e.g., creating a new tracking context associated with the second object and adding it to the set of active tracking contexts, as stated for the entry of object 10a at the source 2c in Figure 1 the reference.

[0090] Thus, the method as described above can reduce the risk of inadvertently continuing to track a second object as the tracked object 10b, where the second object may correspond to a physical object different from the tracked object 10b, and thus requires a greater visual similarity between the second object and the first object 10b to be successfully re-identified. However, if the second object and the first object 10b are similar enough to generate a matching score that passes the threshold test using the adjusted re-identification threshold (in which case the second object is more likely to correspond to the same physical object as the first object 10b), the second object may still be re-identified as the first object 10b and accordingly continue to be tracked.

[0091] To avoid the ambiguous re-identification of an object entering the scene 1 at the source as the first object 10b, a further condition for re-identifying the second object as the second object 10b can be applied, i.e., the second object enters the scene 1 at the source within a predetermined time from when the first object 10b leaves at the sink. If this additional condition is met, the method can continue according to the "yes" branch. If this additional condition is not met, the method can continue according to the "no" branch. For example, the processing device 24 can maintain a timer ( Figure 2 the "timer" field in) for each tracking context 28a-b. In response to detecting that the first object 10b has left the scene 1 at the sink 4a, the processing device 24 can start incrementing the value of the timer. If no second object entering the scene is re-identified as the first object 10b when the timer reaches a predetermined time limit, the tracking of the first object 10b can be terminated, where the tracking context 28b of the first object 10b can be deleted from the set of active tracking contexts. In the case where the first object 10b re-enters the scene 1 at a later time, it can thus be detected, identified, and tracked as a new object, e.g., using a new tracking context. The value of the predetermined time can be based on scene knowledge. For example, the predetermined time can be set according to the type of tracking application, the specific type and location of the source and sink in the scene, etc. The predetermined time can also be set according to the number of tracking contexts that can be maintained by the available computing resources of the object tracking system 20.

[0092] Figure 4 Yes Figure 3 is a flowchart of an optional extension of the method in, which can be executed in parallel with the Figure 3 method steps in.

[0093] As described above, the scene can include a pair of source and sink, for which the probability that an object leaving the scene at the sink re-enters the scene at the source shortly after is essentially zero, or at least too small to trigger an attempt at re-identification. Figure 4 The method of can address this situation.

[0094] At step S10, the processing device 24 determines the position of the second sink where the object leaves the scene. Refer to Figure 1 And relative to the source 2a, the second sink can be represented by, for example, sink 4b.

[0095] At step S11, the processing device 24 uses a re-identification algorithm and a corresponding re-identification threshold associated with the third object to track the "third object" moving in scene 1. Refer to Figure 1 , the third object can be represented by, for example, a car or a truck 10c. As Figure 1 shown, the third object 10c is about to leave scene 1 at the second sink 4b. Therefore, at step S12, the processing device 24 detects that in Figure 1 the video frame after the frame shown, the third object 10c has left the scene at the second sink 4c.

[0096] In response to detecting that the third object 10c has left scene 1 at the second sink 4b, the processing device 24 terminates the tracking of the third object 10c at step S13, thereby preventing a fourth object entering scene 1 at any one of the sources 2a-c from being re-identified as the third object. As previously mentioned, terminating the tracking can include deleting the tracking context associated with the third object from the set of active tracking contexts maintained by the re-identification algorithm.

[0097] As described above, the position of the sink 4b can be determined manually by an operator or automatically using an image recognition algorithm. The decision to determine the sink 4b as the "second sink" where the tracking terminates can be based on scene knowledge. For example, in the knowledge of example scene 1, it can be considered that U-turns are prohibited or even impossible along the road section after the sink 4b.

[0098] Those skilled in the art will recognize that the present invention is in no way limited to the above-described embodiments. On the contrary, within the scope of the appended claims, many modifications and variations are possible.

Claims

1. A computer-implemented method for tracking an object in a video sequence of a scene, the method comprising: determining locations of sinks in the scene at which objects exit the scene and locations of sources at which objects enter the scene; tracking a first object moving in the scene using a re-identification algorithm, wherein the first object is associated with a re-identification threshold of the re-identification algorithm; detecting that the first object has left the scene at the sink; and In response to detecting that the first object has left the scene at the sink, adjusting the re-identification threshold associated with the first object to reduce the probability that the re-identification algorithm will re-identify a second object as the first object, the second object entering the scene at the source after the first object has left the scene at the sink.

2. The method of claim 1 , further comprising, after adjusting the re-identification threshold associated with the first object: detecting entry of the second object at the source; re-identifying the second object as the first object by using the re-identification algorithm with the adjusted re-identification threshold; as well as The second object is then tracked as the first object. 3 . The method according to claim 2 , further comprising, after re-identifying the second object as the first object, restoring the re-identification threshold, and continuing to track the second object as the first object using the restored re-identification threshold.

4. The method according to claim 2, wherein: Re-identifying the second object as the first object comprises: comparing a second object feature set of the second object to a first object feature set of the first object to generate a match score, and The match score is compared to the adjusted re-identification threshold.

5. The method according to claim 2, wherein: Tracking the first object includes maintaining a tracking context for the first object, and Wherein, re-identifying the second object as the first object comprises associating the second object with the tracking context of the first object.

6. The method according to claim 2, wherein: The entry of the second object at the source is detected within a predetermined time from detecting that the first object has left the scene at the sink.

7. The method of claim 1 , further comprising, after determining that more than a predetermined time has elapsed since detecting that the first object has left the scene at the sink: detecting entry of the second object at the source; and The second object is tracked as a distinct object from the first object using the re-identification algorithm and a re-identification threshold associated with the second object.

8. The method of claim 1, further comprising, after adjusting the re-identification threshold associated with the first object: detecting entry of the second object at the source; using the re-identification algorithm and the adjusted re-identification threshold, determining that the second object is different from the first object; and The second object is then tracked as a distinct object from the first object using the re-identification algorithm and a re-identification threshold associated with the second object.

9. The method according to claim 1, wherein: The method includes determining a location of a set of sources at which an object enters the scene, wherein the source is one of the set of sources, and wherein, after adjusting the re-identification threshold, the re-identification algorithm reduces the probability of re-identifying an object entering the scene at any source in the set of sources as the first object.

10. The method of claim 9, further comprising, after adjusting the re-identification threshold associated with the first object: detecting entry of the second object at any one of the set of sources; re-identifying the second object as the first object by using the re-identification algorithm with the adjusted re-identification threshold; as well as The second object is then tracked as the first object.

11. The method according to claim 1, wherein: The sink is a first sink, and the method further comprises determining a position of a second sink at which an object leaves the scene, and the method further comprises: tracking a third object moving in the scene using the re-identification algorithm, wherein the third object is associated with a corresponding re-identification threshold of the re-identification algorithm; detecting that the third object has left the scene at the second sink; and In response to detecting that the third object has left the scene at the second sink, tracking of the third object is terminated, thereby preventing a fourth object entering the scene at the source from being re-identified as the third object.

12. The method according to claim 11, wherein: Tracking the third object includes maintaining a tracking context for the third object, and wherein terminating tracking of the third object includes deleting the tracking context for the third object from a set of activated tracking contexts maintained by the re-identification algorithm.

13. The method according to claim 1, wherein: The source and the sink are adjacent to or overlap each other.

14. An object tracking system comprising: at least one camera for capturing a video sequence of a scene; as well as A processing device configured to track an object in the video sequence according to the method of claim 1.

15. A non-transitory computer-readable storage medium comprising computer program code portions configured to perform the method of tracking an object in a video sequence of a scene according to claim 1 when executed by a processing device.

Images