Method and device for anonymizing images from at least one surveillance camera

Abstract

A method for anonymizing raw images (IMb) from at least one surveillance camera, the raw images (IMb) comprising at least one identifiable person, the method comprising, for at least one raw image (IMb), steps consisting of processing the raw image (IMb) by an anonymization algorithm (E1) so as to obtain an intermediate image (IM1) comprising an outline for each identifiable person, processing the raw image (IMb) by a skeletonization algorithm (E2) so as to obtain a skeleton table (TS) comprising a skeleton (S) of the person and merging (E3) the intermediate images (IM1, TS) so as to form an enriched anonymized image (IMa) comprising a skeleton and an outline for each identifiable person. Abstract figure: Figure 3

Description

Title of the invention: Method and device for anonymizing images from at least one surveillance camera Technical field

[0001] The present invention relates to the field of video surveillance and video protection of a scene in which people are moving around, in particular, in a railway vehicle, a railway station, or on a railway right-of-way. The present invention relates more particularly to the anonymization of images from at least one surveillance camera.

[0002] In order to guarantee the safety of users and / or to obtain statistical information enabling the improvement of services, it is known to position surveillance cameras in a railway vehicle or in a railway station.

[0003] The use of images from surveillance cameras is restrictive given that they may represent an invasion of privacy for persons recognizable in the images. Without the express consent of these persons, the images must be processed by sworn agents solely for security / safety purposes, which is a disadvantage given the quantity of images to be processed by all the surveillance cameras.

[0004] One solution to this problem would be to anonymize the people in the images so that they can be used with fewer constraints, preferably without constraints. In practice, there are algorithms for anonymizing people in images. However, these algorithms degrade the images and make them difficult to use. It is therefore complicated or even impossible to analyze the movements of people from anonymized images. In addition, the calculation time of these algorithms is very long, and is proportional to the number of people to be anonymized. They are therefore not suitable for processing images showing many people, as is the case for images of railway stations.

[0005] The invention thus aims to eliminate at least some of these drawbacks by proposing a method for anonymizing images from surveillance cameras allowing movement analysis. PRESENTATION OF THE INVENTION

[0006] The invention relates to a method for anonymizing raw images from at least one surveillance camera, the raw images comprising at least one identifiable person, the method comprising, for at least one raw image, steps consisting of: • Process the raw image by an anonymization algorithm configured to anonymize each identifiable person on the raw image so as to obtain an intermediate image comprising an outline for each identifiable person, • Process the raw image by a skeletonization algorithm configured to determine points of interest of each identifiable person on the raw image so as to obtain a skeleton table comprising a skeleton for each identifiable person, • Merge the intermediate image and the skeleton table to form an enriched anonymized image including a skeleton and an outline for each identifiable person.

[0007] Thanks to the invention, enriched anonymized images are obtained which can be processed without constraints while making it possible to detect people, count them and their accessories, and exploit the movements of these people without them being identifiable.

[0008] In the presence of several identifiable people, an outline and a skeleton are determined for each, which are then associated.

[0009] According to one aspect, the contour is closed and defines an anonymized interior area. This allows movements to be tracked while respecting strict criteria on personal data.

[0010] According to one aspect, the method comprises a step of reducing the weight of the raw image, for example by reducing its dimensions, prior to the processing steps. This makes it possible to reduce the size of the raw image to facilitate its processing on many types of computer having limited computing resources.

[0011] According to one aspect, the anonymization algorithm comprises a step of coloring the area inside the outline so as to obtain a solid outline. Such a step is simple to implement. Preferably, each person is colored by a different color.

[0012] According to another aspect, the anonymization algorithm comprises a step of subsampling the area within the contour, adding random white noise to it, interpolating the image to return it to the correct dimensions, and smoothing the whole. This makes it possible to give the contour a natural appearance.

[0013] The invention also relates to a method for analyzing movements from raw images from at least one surveillance camera, the raw images comprising at least one identifiable person, the method comprising, for at least one raw image, steps consisting, for each identifiable person, in: • Implement the anonymization process as presented previously to form a plurality of enriched anonymized images and • Implement an algorithm for determining movements by analyzing the skeleton of enriched anonymized images.

[0014] Advantageously, a video stream of enriched anonymized images can be formed in which the movements of the skeletons are visible. Such movements are simple and practical to interpret by a movement determination algorithm. This makes it possible to determine any risk or danger situation quickly and practically.

[0015] According to one aspect, the movement determination algorithm comprises steps consisting of: • Determine the person’s movements by analyzing the skeleton, • Comparing the determined movements to a movement database associating a plurality of movements with movement alerts so as to determine, where appropriate, at least one movement alert.

[0016] According to one aspect, the movement determination algorithm comprises a step of determining the movements of the person by analyzing the skeleton and the deformation of the contour. This allows for greater robustness in determining movements, and better confidence in the determined movements.

[0017] Determining motion alerts makes it easier for operators to work with access to a large number of video streams of anonymized images.

[0018] The invention also relates to a computer program type product, comprising at least one sequence of instructions stored and readable by a processor and which, once read by this processor, causes the steps of the method as presented previously to be carried out.

[0019] The invention further relates to a computer-readable medium comprising the computer program product as presented above.

[0020] The invention also relates to a device for anonymizing raw images from at least one surveillance camera, the raw images comprising at least one identifiable person, the anonymizing device comprising a computer configured to implement, for at least one raw image, steps consisting of: • Process the raw image by an anonymization algorithm configured to anonymize each identifiable person on the raw image so as to obtain an intermediate image comprising an outline for each identifiable person, • Process the raw image by a skeletonization algorithm configured to determine points of interest of each identifiable person on the raw image so as to obtain a skeleton table comprising a skeleton for each identifiable person, • Merge the intermediate image and the skeleton table to form an enriched anonymized image including a skeleton and an outline for each identifiable person.

[0021] The invention also relates to a movement analysis system comprising an anonymization device as presented previously and a movement analysis device comprising a computer configured to implement an algorithm for determining movements from the enriched anonymized images from the anonymization device.

[0022] Preferably, the anonymization device and the movement analysis device form a single device. The analysis system is advantageously integrated.

[0023] The invention also relates to an anonymization system comprising an anonymization device as presented previously associated with at least one surveillance camera. PRESENTATION OF THE FIGURES

[0024] The invention will be better understood on reading the description which follows, given by way of example, and referring to the following figures, given by way of non-limiting examples, in which identical references are given to similar objects.

[0025] [Fig.l] is a schematic representation of a station in which a surveillance camera is installed.

[0026] [Fig.2] is a schematic representation of an exemplary anonymization system according to one embodiment of the invention.

[0027] [Fig. 3] is a schematic representation of an example of implementation of an anonymization method according to the invention.

[0028] [Fig.4] is a schematic representation of an example anonymization algorithm

[0029] [Fig.5] is a schematic representation of a processing of several enriched anonymized images.

[0030] [Fig.6] is a schematic representation of the method of temporal analysis of movements recorded by a surveillance camera on at least one image.

[0031] [Fig.7] is a schematic representation of a raw image.

[0032] [Fig.8] is a schematic representation of an intermediate image.

[0033] [Fig.9] is a schematic representation of the skeletons extracted to determine the skeleton table.

[0034] [Fig. 10] is a schematic representation of an enriched anonymized image.

[0035] It should be noted that the figures set out the invention in detail to highlight implement the invention, said figures being able of course to serve to better define the invention where appropriate. DETAILED DESCRIPTION OF THE INVENTION

[0036] With reference to [Fig. 1], there is shown schematically a surveillance camera 1 filming a scene SC comprising several identifiable people 2 positioned in a railway station, in particular, near a railway vehicle 3. The surveillance camera 1 can be positioned both in the railway station and in a railway vehicle 3 to monitor the actions carried out by the people in order to detect any accident or danger at an early stage.

[0037] The invention is presented in a railway context but it applies to any place receiving the public, in particular, an airport, a bus station, a port, a public building, etc.

[0038] In a known manner, with reference to [Fig. 2], the surveillance camera 1 acquires a plurality of raw images IMb in order to form a video stream. In [Fig. 2], there is shown an anonymization device 4, connected to the surveillance camera 1, which is configured to receive as input at least one raw image IMb from the surveillance camera 1 and to provide as output an enriched anonymized image IMa. Such an anonymization device 4 is advantageous given that it allows the enriched anonymized image IMa to be used with few constraints, in particular, to allow its consultation by non-sworn operators 8, to allow its temporary or permanent storage 9 or to allow subsequent processing 10, for example, on remote and / or third-party servers to implement artificial intelligence algorithms.Thanks to the method according to the invention, a video stream of raw IMb images at input becomes easily usable subsequently, because it complies with the regulations in force on the protection of privacy.

[0039] In this case, the anonymization of a video stream is presented, but it goes without saying that the invention also applies to one or more individual images. The raw images can be an individual image or a succession of images or a video which is a succession of temporally ordered images.

[0040] Similarly, the anonymization of images captured by a surveillance camera is presented, but it goes without saying that the invention applies to images captured by any image acquisition device.

[0041] As illustrated in [Fig.2], the anonymization device 4 comprises a computer 40 configured to implement an anonymization method which will now be presented. In this example, the computer 40 comprises a computer processor which may take various forms, in particular, an FPGA or a graphics card.

[0042] In this example, as illustrated in [Fig.7], each raw image IMb comprises two identifiable persons 2 who interact together.

[0043] [Fig. 3] schematically represents an implementation of an anonymization method according to the invention by means of the anonymization device 4. The anonymization method comprises steps consisting of: • Process the raw image IMb by an anonymization algorithm configured to anonymize the identifiable persons 2 on the raw image IMb so as to obtain an intermediate image IM1 comprising a contour C for each identifiable person 2 as illustrated in [Fig.8], • Process E2 the raw image IMb by a skeletonization algorithm configured to determine a skeleton S for each identifiable person 2 so as to obtain a skeleton table TS comprising a skeleton S for each identifiable person 2, each skeleton S comprising points of interest determined for each identifiable person 2 connected by segments as illustrated in [Fig.9], • Merge E3 the intermediate image IM1 and the skeleton table TS so as to form an enriched anonymized image IMa which includes the contour C and the skeleton S of each person 2 as illustrated in [Fig. 10].

[0044] In practice, the relevant points of each skeleton are added to the intermediate image.

[0045] In this example, the contour C is a closed contour. It goes without saying that the invention could also be applied to an open contour.

[0046] Preferably, access to the raw IMb images recorded in real time by the surveillance camera 1 is done by using an RTSP protocol (Real Time Streaming Protocol). Preferably, several consecutive raw IMb images can be processed together, regardless of their recording format.

[0047] Preferably, steps E1 and E2 are implemented in parallel in order to quickly obtain the enriched anonymized image IMa, preferably in real time.

[0048] The anonymization step E1 processes a raw image IMb and modifies it to give an intermediate image IM1 which cannot be used by the skeletonization algorithm. Indeed, the anonymization will degrade the relevant information relating to the skeleton S, which will prohibit any interpretation of the movement based on the skeleton. The skeletonization step E2 processes a raw image IMb to determine the skeletons S of the identifiable persons 2 present on the raw image IMb, and extracts them to a skeleton table TS. According to one aspect, the skeletonization step E2 also determines, in a preliminary manner, a bounding box for each identifiable person in order to detect the skeleton. The recording of the coordinates of this bounding box advantageously makes it possible to have greater robustness and greater confidence in the detection of the skeletons S, in particular in the detection of partial skeletons S.This skeletonization step E2 has no impact on the raw image IMb, and can therefore be carried out upstream or in parallel with the anonymization step EL. Putting the two steps El and E2 in parallel advantageously allows their advantages to be combined on an enriched anonymized image IMa obtained by fusion E3 of the intermediate image IM1 and the skeleton table TS. In addition, the bounding box can also be added to the skeleton table TS in order to be used for motion / behavior detection.

[0049] Optionally, with reference to [Fig.3], the anonymization method comprises an optional step consisting of reducing the weight E0 of the raw image IMb, for example by reducing its dimensions, so as to allow the processing of the raw image lmb by any computer system regardless of its computing power.

[0050] The anonymization algorithm and the skeletonization algorithm will now be presented in detail.

[0051] With reference to [Fig.4], according to a first example of implementation, the anonymization algorithm comprises steps consisting of determining Eli the areas containing identifiable persons 2 on the raw image lmb.

[0052] Preferably, contours C of the identifiable persons 2 are determined by semantic or instance segmentation. This can be configured according to the future use of the enriched anonymized images Ima and the data to be extracted. This determination is more precise and allows less information to be lost than by determining a bounding box for each identifiable person present on the raw image lmb. Preferably, the detection of the contours C of the identifiable persons 2 is done by using a deep neural network, for example of the “Mask RCNN” type, known to those skilled in the art.

[0053] According to one aspect, the anonymization algorithm comprises a step consisting of sub-sampling E12 the interior of the previously determined areas. The persons 2 are therefore no longer recognizable, but the color, intensity and brightness are preserved. To anonymize the persons 2, the anonymization algorithm comprises, alternatively, a step consisting of coloring E12' the interior of the previously determined areas with a solid color mask.

[0054] According to one aspect, the anonymization algorithm comprises a step consisting of adding random noise E13 in order to prevent any reconstruction of the raw image lmb, that is to say, any possibility of identifying people 2.

[0055] According to one aspect, the anonymization algorithm comprises a step consisting of performing E14 an interpolation in order to form an intermediate image IM1 having the same resolution as the raw image lmb. This interpolation E14 advantageously makes it possible to restore the intermediate image IM1 to the correct dimensions while avoiding reconstructing the information removed by the anonymization.

[0056] According to one aspect, the anonymization algorithm comprises a step consisting of smoothing E15 the contours of the sub-sampled zones making it possible to give a natural appearance to the intermediate image IM1. It goes without saying that such a step is not carried out when the determined zones are colored uniformly (step E12').

[0057] As illustrated in [Fig.4] and [Fig.8], an intermediate image IM1 is advantageously obtained which includes the precise contours C of the persons 2. The persons and their movements are not identifiable.

[0058] In this example, the intermediate image IM1 also includes the background F of the scene SC, here, the railway vehicle. It goes without saying that the background F could be omitted or present only on the skeleton table TS. The background F is useful but remains optional.

[0059] According to one aspect, the skeletonization algorithm implements an image processing method so as to define a skeleton S for each identifiable person 2 of the raw image lmb. In a known manner, points of interest are previously determined for each identifiable person 2, for example, at the joints (elbow, knee, etc.) and the extremities of the body (hand, foot, head). These points of interest are connected by segments in order to form a skeleton S which visually illustrates the position of the body of each identifiable person 2 without providing information on the outline of the person. Such a skeletonization method is known to those skilled in the art.

[0060] As illustrated in [Fig.9], the skeleton table TS only comprises the skeletons S of the identifiable persons 2 present in the raw image lmb. The skeleton table S comprises, for example, all the coordinates of the joints of the skeletons S, and can therefore, advantageously, be easily manipulated and processed by analysis algorithms.

[0061] With reference to [Fig. 10], during the fusion step E3, the intermediate image IM1, which comprises the contours C of the identified persons and the background F; and the skeleton table TS, which comprises the skeletons S, are merged so as to form an enriched anonymized image Ima which comprises the contours C of the identified persons associated with their skeletons S as well as the background F.

[0062] It is thus possible to obtain an enriched anonymized image Ima, the exploitation of which is easy as presented previously. Advantageously, as illustrated in [Fig.5], it is possible to form an enriched anonymized video stream VIDa from a plurality of enriched anonymized images Ima obtained over time.

[0063] With reference to [Fig.5], the enriched anonymized video stream VIDa is provided to a motion analysis device 5. In this example, the motion analysis device 5 comprises a computer 50 configured to implement a motion analysis method so as to provide at least one RA alert report in the event of detection of an abnormal movement. In this example, the computer 50 is a computer processor which may take various forms, in particular, an FPGA.

[0064] According to a particular aspect, the movement analysis device 5 and the anonymization device 4 are the same equipment.

[0065] In this example, with reference to [Fig.6], the motion analysis method comprises steps consisting of: • Determine E51 the movements of the people present image by image, in particular, by analyzing the evolution of the postures of the skeletons S of said people. • Compare E52 the determined movements to a movement database DBm associating a plurality of movements with movement alerts, so as to detect at least one movement alert Alm.

[0066] Preferably, the comparison step E52 is carried out by a classification algorithm, for example, a deep neural network.

[0067] According to one aspect of the invention, the step of determining the movements E51 can be carried out by an analysis of the deformation of the contours C from one image to another. The analysis of the movements, that is to say the succession of postures, is thus improved and more robust given that it is carried out, on the one hand, from the skeletons S and, on the other hand, from the deformation of the contours C.

[0068] The RA alert report includes the Alm movement alert and makes it possible to provide precise information to an operator, for example, a person falling in order to be able to alert the emergency services quickly. The presence of the C contours makes it possible to better contextualize the understanding of the SC scene and to facilitate interpretation. In addition, the coupling of the use of the C contours and the S skeletons provides great robustness of detection and analysis.

[0069] The anonymization and motion analysis methods can advantageously be implemented in real time, i.e. at the same time as the raw LMB images are acquired. They can also be implemented after the acquisition, and without the anonymization device 4 or the movement analysis device 5 being connected to an Internet network.

[0070] Thanks to the RA alert report, it is thus possible to carry out a statistical analysis on the data obtained from the enriched anonymized images Ima, in order to enable an improvement of the services carried out in / on the station and in / on the railway vehicles.

Claims

Claims

1. Method for analyzing movements from raw images (IMb) from at least one surveillance camera (1), the raw images (IMb) comprising at least one identifiable person (2), the method comprising, for at least one raw image (IMb), steps consisting, for each identifiable person (2), in: • Processing the raw image (lmb) by an anonymization algorithm (El) configured to anonymize each identifiable person (2) on the raw image (lmb) so as to obtain an intermediate image (IM1) comprising a person outline (C) for each identifiable person (2), • Processing the raw image (IMb) by a skeletonization algorithm (E2) configured to determine points of interest of each identifiable person (2) on the raw image (IMb) so as to obtain a skeleton table (TS) comprising a skeleton (S) for each identifiable person (2),• Merge (E3) the intermediate image (IM1) and the skeleton table (TS) so as to form an enriched anonymized image (IMa) comprising a skeleton (S) and an outline (C) for each identifiable person (2) and thus obtain a plurality of enriched anonymized images (IMa) and • Implement an algorithm for determining movements by analyzing the skeleton (S) and the deformation of the outline (C) of the enriched anonymized images (IMa).,

2. A method according to claim 1, wherein the contour (C) is closed and defines an anonymized inner area.

3. Method according to one of claims 1 to 2, comprising a step consisting of reducing the weight (EO) of the raw image (IMb) prior to the processing steps (El, E2).

4. A method according to claim 2 to 3, wherein the anonymization algorithm comprises a step of sub-sampling (E12) the area within the contour (C).

5. Method according to claim 2 to 3, wherein the anonymization algorithm comprises a step of coloring (E12') the area inside the outline (C).

6. Method for analyzing movements according to one of claims 1 to 5, in which the algorithm for determining movements comprises steps consisting of: • Determining (E51) the movements of the person by analyzing the skeleton (S) and the deformation of the contour (C) • Comparing (E52) the determined movements with a movement database (DBm) associating a plurality of movements with movement alerts (ALm) so as to determine, where appropriate, at least one movement alert (ALm).

7. Computer program type product, comprising at least one sequence of instructions stored and readable by a processor and which, once read by this processor, causes the steps of the method according to one of the preceding claims to be carried out.

8. A computer-readable medium comprising the computer program product of claim 7.

Images