METHOD FOR TRACKING AN ELEMENT, AN ASSOCIATED ELECTRONIC DEVICE AND A COMPUTER PROGRAM PRODUCT

Description

[0001] The present invention relates to a method for tracking an element. The present invention also relates to an associated electronic device for tracking an element, as well as an associated computer program product.

[0002] In the field of video surveillance, tracking devices are used to follow individuals over time. Such devices allow for the re-identification of the individual in successive images.

[0003] In particular, it is known from tracking systems where the profile of a new individual to be identified is compared to the profile of a reference individual. A score is then assigned to the comparison. The new individual to be identified is considered the reference individual when the score obtained for the comparison is above a threshold, and is not identified otherwise.

[0004] However, such devices cannot track an individual when their appearance changes, for example, if the individual removes an article of clothing, which can lead to false negatives. US 9 524 448 82 concerns multi-camera object tracking, in which the apparent signature of an object varies from one camera to another due to changes in viewpoint and posture.

[0005] Therefore, there is a need for a device to track elements in order to reduce the false negative rate during monitoring. The invention is defined in the independent claims.

[0006] To this end, the invention relates to a method for tracking a reference element in successive images of an environment, the reference element being associated with a list of reference visual signatures comprising at least one reference visual signature, the visual signature of an element being a vector describing the appearance of the element, the method being implemented by an electronic tracking device and comprising at each instant the steps of: reception of at least one image of the environment, where applicable, detection of at least one first element on at least one received image, determination of a first visual signature for at least one detection of each first element, calculation, for each first element, of at least one distance between the first visual signature(s) and the reference visual signature(s), comparison, for each first element, of a distance, a function of the distance(s) calculated for the first element, to a first threshold and to a second threshold, the first threshold being less than or equal to the second threshold, for each first element, identification of the first element as being the reference element, when the compared distance is less than or equal to the first threshold, the first element being unidentified when the compared distance is strictly greater than the second threshold, and where applicable,When the first element was not identified during the identification step, and the first element is the same element as the reference element but with a different appearance, the first visual signature is added to the list of reference visual signatures after receiving a user's add command.

[0007] According to other advantageous aspects of the invention, the tracking method comprises one or more of the following features, taken individually or in all technically possible combinations: The method further includes the steps of: detecting an initial element on at least one previously received image relative to the image on which the first element or each first element was detected; determining a visual signature for at least one detection of the initial element; acquiring a tracking command for the initial element such that the initial element is the reference element and each visual signature of the initial element is a reference visual signature from the list of reference signatures; the first threshold is different from the second threshold; when the compared distance is strictly greater than the first threshold and less than or equal to the second threshold, the method includes a step of sending a validation request for the identification of the first element;The process includes a step of adding the first visual signature to the list of reference visual signatures after receiving user validation in response to the validation request; the images come from a camera network; the reception step includes receiving at least two images of the environment, imaging the environment from two different viewpoints, the two images coming from two cameras in the camera network positioned at distinct locations; each visual signature is determined by a model comprising a neural network.

[0008] The invention further relates to an electronic device for tracking a reference element in successive images of an environment, the reference element being associated with a list of reference visual signatures comprising at least one reference visual signature, the visual signature of an element being a vector describing the appearance of the element, the tracking device comprising: an image reception channel, a human-machine interface, and a computer configured to implement a tracking process as described above.

[0009] The invention also relates to a computer program product comprising a readable information carrier, on which is stored a computer program comprising program instructions, the computer program being loadable onto a data processing unit and adapted to drive the implementation of a process as described above when the computer program is implemented on the data processing unit.

[0010] Other features and advantages of the invention will become apparent from the following description of embodiments of the invention, given by way of example only, and with reference to the drawings which are: [ Fig 1] figure 1 , a schematic representation of a tracking device receiving images from a network of cameras imaging an environment, and [ Fig 2] figure 2 , an organizational chart of an example of the implementation of a monitoring process.

[0011] An electronic device 10 for tracking a reference element E in an environment is illustrated by the figure 1 The environment is, for example, urban.

[0012] In the example illustrated by the figure 1 The environment is imaged by a network of 12 cameras. The 12 cameras in the network are positioned to image the environment from different viewpoints. In particular, the viewpoints differ between each camera depending on the weather conditions, the surrounding environment, and the position of the cameras.

[0013] Alternatively, the environment is imaged by a single camera 12.

[0014] The element E ref is, for example, an individual or an animal. Alternatively, the element E ref is an object, such as a vehicle.

[0015] The device 10 includes an image reception channel 20, a human-machine interface 22 and a computer 24.

[0016] Image reception channel 20 is intended to be connected to an image acquisition system, such as the camera array 12 illustrated in figure 1 The connection is made via wired or wireless connection (e.g., Wi-Fi or Bluetooth).

[0017] The human-machine interface 22 includes, for example, a display and / or a keyboard. The human-machine interface 22 allows requests to be sent to a user 25 and commands to be received from the user 25.

[0018] In a first example, the calculator 24 includes a processor comprising a data processing unit, memories and an information support reader.

[0019] In this example, the calculator 24 interacts with a computer program product that contains program instructions. The computer program product is stored on a data storage medium.

[0020] The information storage medium is a medium readable by the computer 24, usually by the computer's data processing unit 24. The information storage medium is a medium suitable for storing electronic instructions and capable of being connected to a bus of a computer system. Examples of information storage media include a floppy disk, an optical disc, a CD-ROM, a magneto-optical disc, a ROM, a RAM, an EPROM, an EEPROM, a magnetic card, an optical card, a USB flash drive, or an SSD. The computer program product, comprising program instructions, is stored on the information storage medium.

[0021] The computer program can be loaded onto the data processing unit of computer 24 and is adapted to trigger the implementation of a tracking method when the computer program is implemented on the processing unit of computer 24. The tracking method will be described in detail later in the description.

[0022] In a second example, the calculator 24 is implemented in the form of one or more programmable logic components, such as FPGAs (from the English Field Programmable Gate Array ), or in the form of dedicated integrated circuits, such as ASICs (from the English Application Specific Integrated Circuit ).

[0023] In a third example, the calculator 24 is implemented in the form of a central computing unit (CPU) and a graphics computing unit (GPU), the latter having the function of a coprocessor.

[0024] The operation of the monitoring device 10 allows for the implementation of a monitoring process which is described with reference to the organizational chart of the figure 2 .

[0025] The monitoring process includes a first phase 100 of determining an element to be monitored, called reference element E ref, and a second phase 200 of monitoring the reference element E ref.

[0026] The first phase 100 of the tracking process includes a step 110 of receiving at least one image of an environment, for example, from the camera network 12 illustrated in figure 1 .

[0027] The first phase 100 includes, where applicable, a step 120 of detecting an initial element on at least one of the received images. Detection is performed, for example, by a detection algorithm comprising a category detection model. For example, the detection algorithm uses a Faster RCNN / RFCN type deep learning model.

[0028] The first phase 100 then includes a step 130 of determining a visual signature for at least one of the detections of the initial element. For example, if the initial element was detected in several images acquired by cameras positioned at different locations, for each camera, the detection(s) of the initial element will result in a visual signature specific to that camera. The images of an element acquired by the same camera form a time-series image aggregate called a "tracklet".

[0029] The visual signature of an element is a vector describing its appearance. The term "appearance" refers to the element's external characteristics (for example, its color, dimensions, or shape). The visual signature takes into account factors such as the viewpoint from which the element is viewed, the technology of the camera used to capture the image, and even the weather conditions; therefore, a unique visual signature is obtained for each detection of the same element.

[0030] The visual signature is, for example, determined by a model obtained by training a neural network with a database of images. For example, the visual signature is obtained using the method described in the article entitled "Person re-identification across different datasets with multi-task learning" by Mathieu Ospici and Antoine Cecchi, published on arXiv on July 25, 2018.

[0031] If necessary, the first phase 100 includes a step 130 of acquiring a user command to track the initial element, such that the initial element is the reference element E ref. Each visual signature of the initial element is then called a reference visual signature. The reference element E ref is then associated with a list of reference signatures. The reference element E ref and the list of reference visual signatures are then stored, at least temporarily, in a memory of the computer 24.

[0032] The command is obtained via the human-machine interface 22 of the tracking device 10.

[0033] If no tracking command is received from the user, the receive steps 110, detect 120 and determine 130 are repeated until an item to track is selected by the user.

[0034] Alternatively, the first phase 100 of the tracking process includes only the receipt of a tracking order for an E ref element associated with a list of reference visual signatures, the reference element E ref and the associated list having been determined prior to the tracking process.

[0035] Once the reference element E ref has been determined, the second phase 200 of the monitoring process is implemented.

[0036] The second phase 200 includes a step 210 of receiving at least one image of the environment acquired after the image(s) used to determine the reference element. Generally, the images received by the tracking device 10 are images acquired successively by the same acquisition system, such as the camera array 12 illustrated in figure 1 .

[0037] Next, the second phase 200 includes, where applicable, a step 220 for detecting at least one first element E1 on at least one received image. The detection step 220 is implemented in the same way as the detection step 120.

[0038] The second phase 200 then includes a step 230 of determining a first visual signature for at least one detection of each first element E1. The determination step 230 is implemented in the same way as the determination step 130.

[0039] Next, the second phase 200 includes a step 240 of calculation, for each first element E1, of at least one distance between the first visual signature(s) and the reference visual signature(s).

[0040] For example, for each first element E1, a distance is obtained between a signature resulting from the average of the first visual signatures and each reference visual signature.

[0041] Alternatively, for each first element E1, a distance is obtained between each first visual signature and a signature resulting from the average of the reference visual signatures.

[0042] Alternatively, for each first element E1, a distance is obtained between a signature resulting from the average of the first visual signatures and a signature resulting from the average of the reference visual signatures.

[0043] Alternatively, for each first element E1, a distance is obtained between each first visual signature and each reference visual signature.

[0044] Each calculated distance is, for example, a Euclidean distance. For example, the calculated distance is obtained using the method described in the article entitled "Person re-identification across different datasets with multi-task learning" by Mathieu Ospici and Antoine Cecchi, published in ArXiv on July 25, 2018.

[0045] The second phase 200 includes a step 250 of comparison, for each first element E1, of a distance function of the calculated distance(s), to a first threshold S1 and to a second threshold S2. The first threshold S1 is less than or equal to the second threshold S2.

[0046] Advantageously, the first threshold S1 is different from the second threshold S2.

[0047] For example, when only one distance is obtained for the same first element, the distance compared is the calculated distance.

[0048] For example, when several distances are obtained for the same first element E1, the distance compared is the smallest distance obtained for the first element E1.

[0049] Alternatively, when several distances are obtained for the same first element E1, the distance compared is a distance resulting from the average of the distances obtained for the first element E1.

[0050] The second phase 200 then includes, for each first element E1, a step 260 of identifying the first element E1 as the reference element, when the compared distance is less than or equal to the first threshold S1. The first element E1 is not identified when the compared distance is strictly greater than the second threshold S2.

[0051] If necessary, when the first element E1 has not been identified during the identification step 260 (distance strictly greater than S2) and the first element E1 is the same element as the reference element E ref with a different appearance from the reference element E ref, the second phase 200 includes a step 270 of adding the first visual signature to the list of reference visual signatures after receiving a command from the user.

[0052] The change in appearance of the first element E1 relative to the reference element E ref is, for example, induced by a change in the color or shape of the first element E1. For instance, when the reference element E ref is an individual, the addition or removal of clothing by the individual is likely to significantly impact the individual's appearance. Step 270 allows a user to update the list of visual signatures if they notice such a change.

[0053] The command is obtained via the human-machine interface 22 of the tracking device 10.

[0054] At the end of the addition step 270, the reception steps 210, detection 220, determination 230, calculation 240, comparison 250, identification 260 and, where applicable, subsequent steps of the second phase 200 of the monitoring process are repeated at the next instant with the updated list of visual signatures.

[0055] In the absence of the addition step 270, the reception steps 210, detection 220, determination 230, calculation 240, comparison 250, identification 260 and, where applicable, the subsequent steps of the second phase 200 of the monitoring process are also repeated at the next instant with the initial list of visual signatures.

[0056] Advantageously, when the compared distance is strictly greater than the first threshold S1 and less than or equal to the second threshold S2, the second phase 200 includes a step 280 of sending a request to validate the identification of the first element E1. The interval between the first threshold S1 and the second threshold S2 thus corresponds to a user consultation interval.

[0057] When, in response to the validation request, the tracking device 10 receives validation of the identification, the second phase 200 includes a step 290 of adding the first visual signature to the list of reference visual signatures. The command is obtained via the human-machine interface 22 of the tracking device 10.

[0058] At the end of the addition step 290, the reception steps 210, detection 220, determination 230, calculation 240, comparison 250, identification 260 and, where applicable, subsequent steps of the second phase 200 of the monitoring process are repeated at the next instant with the updated list of visual signatures.

[0059] Thus, the present tracking process allows for improved tracking of an element through actions that update the list of reference visual signatures following user commands.

[0060] In particular, step 270, which adds a visual signature of an unidentified item to the visual signature list, allows tracking of an item to continue even if its visual appearance has changed. This is the case, for example, when an individual adds or removes an item of clothing. This step therefore improves the quality of tracking and reduces the false negative rate.

[0061] Furthermore, validation and addition steps 280 and 290 allow for the re-identification of the reference element E ref even when external conditions or the viewpoint of the element in the images have changed, or the images are noisy. External conditions include, for example, brightness and weather. Indeed, if a new appearance occurs from a similar viewpoint (same camera technology, similar external conditions), the re-identification process based on signature comparison will work with a high probability (distance less than the first threshold). Conversely, if a new appearance occurs from a very different viewpoint (different camera technology, very different external conditions), the re-identification process will have a lower probability of working (distance less than the first threshold but greater than the second threshold), hence the benefit of having a user validate the identification.

[0062] Finally, the ability to save multiple reference visual signatures improves the robustness of identifications.

[0063] Such a process can, for example, be used to identify the path of the detected element and thus create training datasets.

[0064] Such a process can also be used, for example, to follow indifferently two coupled entities, such as a man and his horse, two people acting together or a parent and child.

Claims

1. A method for tracking a reference element (Eréf) on successive images of an environment, the reference element (Eréf) being associated with a list of reference visual signatures comprising at least one reference visual signature, the visual signature of an element being a vector describing the appearance of the element, the method being implemented by an electronic tracking device (10) and comprising at each instant the steps of: - reception of at least one image of the environment; - where applicable, detection of at least a first element (E1) on at least one received image; - determination of a first visual signature for at least one detection of each first element (E1); - calculation, for each first element (E1), of at least one distance between the first visual signature(s) and the reference visual signature(s); - comparison, for each first element (E1), of a distance, as a function of the distance or distances calculated for the first element (E1), with a first threshold (S1) and a second threshold (S2), the first threshold (S1) being strictly less than the second threshold (S2); - for each first element (E1), identification of the first element (E1) as the reference element, when the compared distance is less than or equal to the first threshold (S1), the first element (E1) being unidentified when the compared distance is strictly greater than the second threshold (S2); - where applicable, when the first element (E1) has not been identified during the identification step and the first element (E1) is the same element as the reference element (Eréf) with a different appearance from the reference element, adding the first visual signature to the list of reference visual signatures after receipt of a user's add command; - when the compared distance is strictly greater than the first threshold (S1) and less than or equal to the second threshold (S2), sending a request for validation of the identification of the first element (E1); and - adding the first visual signature to the list of reference visual signatures after receiving a validation from the user in response to the validation request.

2. The tracking method according to claim 1, wherein the method further comprises the steps of: - detection of an initial element on at least one image received previously with respect to the image on which the or each first element (E1) was detected, - determination of a visual signature for at least one detection of the initial element, - acquisition of a tracking command of the initial element so that the initial element is the reference element (Eréf) and each visual signature of the initial element is a reference visual signature of the list of reference signatures.

3. The tracking method according to claim 1 or 2, wherein the images are from a camera network.

4. The tracking method according to claim 3, wherein the reception step comprises reception of at least two images of the environment imaging the environment from two different viewpoints, the two images being from two cameras of the camera network positioned at separate locations.

5. The tracking method according to any one of claims 1 to 4, wherein each visual signature is determined by a model comprising a neural network.

6. An electronic tracking device (10) for tracking a reference element (Eréf) over successive images of an environment, the reference element (Eréf) being associated with a list of reference visual signatures comprising at least one reference visual signature, the visual signature of an element being a vector describing the appearance of the element, the tracking device (10) comprising: - an image reception channel (20), - a human-machine interface (22), and - a computer (24) configured to implement a tracking method according to any one of claims 1 to 5.

7. Computer program product comprising a readable information medium, on which is stored a computer program comprising program instructions, the computer program being loadable onto a data processing unit and adapted to cause the implementation of a method according to any one of claims 1 to 5 when the computer program is executed on the data processing unit.