Method and device for anonymizing images from at least one surveillance camera
The method enhances image anonymization by creating enriched images with contours and skeletons, addressing computational inefficiencies and privacy concerns, enabling efficient movement analysis and risk detection in surveillance.
Patent Information
- Application Number
- FR2023005639
- Authority / Receiving Office
- FR · FR
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-06-05
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-06-05
AI Technical Summary
Existing image anonymization algorithms for surveillance cameras degrade images, making them difficult to analyze and are computationally inefficient, especially for scenes with many people, such as railway stations.
A method involving anonymization and skeletonization processes to create enriched anonymized images, allowing analysis of movements without identifying individuals, using contour and skeleton data, with optional steps to reduce image weight and add noise for natural appearance.
Enriched anonymized images facilitate movement detection and analysis, enabling efficient processing and real-time risk assessment while respecting privacy, suitable for various computing resources.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Title of the invention: Method and device for anonymizing images from at least one surveillance camera Technical field
[0001] The present invention relates to the field of video surveillance and video protection of a scene in which people are moving around, in particular, in a railway vehicle, a railway station, or on a railway right-of-way. The present invention relates more particularly to the anonymization of images from at least one surveillance camera.
[0002] In order to guarantee the safety of users and / or to obtain statistical information enabling the improvement of services, it is known to position surveillance cameras in a railway vehicle or in a railway station.
[0003] The use of images from surveillance cameras is restrictive since they may represent an invasion of privacy for persons recognizable in the images. Without the express consent of these persons, the images must be processed by sworn agents solely for security / safety purposes, which is a disadvantage given the quantity of images to be processed by all the surveillance cameras.
[0004] One solution to this problem would be to anonymize the people in the images so that they can be used with fewer constraints, preferably without constraints. In practice, there are algorithms for anonymizing people in images. However, these algorithms degrade the images and make them difficult to use. It is therefore complicated or even impossible to analyze the movements of people from anonymized images. In addition, the calculation time of these algorithms is very long, and is proportional to the number of people to be anonymized. They are therefore not suitable for processing images showing many people, as is the case for images of railway stations.
[0005] The invention thus aims to eliminate at least some of these drawbacks by proposing a method for anonymizing images from surveillance cameras allowing movement analysis. PRESENTATION OF THE INVENTION
[0006] The invention relates to a method for anonymizing raw images from at least one surveillance camera, the raw images comprising at least one identifiable person, the method comprising, for at least one raw image, steps consisting of: Process the raw image through an anonymization algorithm configured to anonymize each identifiable person in the raw image so as to obtain an intermediate image including an outline for each identifiable person, • Process the raw image using a skeletonization algorithm configured to determine points of interest of each identifiable person on the raw image so as to obtain a skeleton table comprising a skeleton for each identifiable person, • Merge the intermediate image and the skeleton table to form an enriched anonymized image including a skeleton and an outline for each identifiable person.
[0007] Thanks to the invention, enriched anonymized images are obtained which can be processed without constraints while making it possible to detect people, count them and their accessories, and exploit the movements of these people without them being identifiable.
[0008] In the presence of several identifiable people, an outline and a skeleton are determined for each person, which are then associated.
[0009] According to one aspect, the contour is closed and defines an anonymized interior area. This allows movements to be tracked while respecting strict criteria on personal data.
[0010] According to one aspect, the method comprises a step of reducing the weight of the raw image, for example by reducing its dimensions, prior to the processing steps. This makes it possible to reduce the size of the raw image to facilitate its processing on many types of computer having limited computing resources.
[0011] According to one aspect, the anonymization algorithm comprises a step of coloring the area inside the outline so as to obtain a solid outline. Such a step is simple to implement. Preferably, each person is colored by a different color.
[0012] According to another aspect, the anonymization algorithm comprises a step of subsampling the area within the contour, adding random white noise to it, interpolating the image to return it to the correct dimensions, and smoothing the whole. This makes it possible to give the contour a natural appearance.
[0013] The invention also relates to a method for analyzing movements from raw images from at least one surveillance camera, the raw images comprising at least one identifiable person, the method comprising, for at least one raw image, steps consisting, for each identifiable person, in: • Implement the anonymization process as presented previously to form a plurality of enriched anonymized images and
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020] • Implement an algorithm for determining movements by analyzing the skeleton of enriched anonymized images. Advantageously, a video stream of enriched anonymized images can be generated, in which the movements of the skeletons are visible. Such movements are easy and convenient to interpret using a movement determination algorithm. This allows any risk or danger situation to be determined quickly and conveniently. According to one aspect, the movement determination algorithm comprises steps of: • Determine the person’s movements by analyzing the skeleton, • Comparing the determined movements to a movement database associating a plurality of movements with movement alerts so as to determine, where appropriate, at least one movement alert. In one aspect, the motion determination algorithm includes determining the person's motions by analyzing the skeleton and contour deformation. This provides greater robustness in motion determination and greater confidence in the determined motions. Determining motion alerts makes it easier for operators to work with access to a large number of anonymized video streams. The invention also relates to a computer program type product, comprising at least one sequence of instructions stored and readable by a processor and which, once read by this processor, causes the steps of the method as presented previously to be carried out. The invention further relates to a computer-readable medium comprising the computer program product as presented above. The invention also relates to a device for anonymizing raw images from at least one surveillance camera, the raw images comprising at least one identifiable person, the anonymizing device comprising a calculator configured to implement, for at least one raw image, steps consisting of: • Process the raw image by an anonymization algorithm configured to anonymize each identifiable person on the raw image so as to obtain an intermediate image including an outline for each identifiable person, • Process the raw image using a skeletonization algorithm configured to determine points of interest of each identifiable person on the raw image so as to obtain a skeleton table comprising a skeleton for each identifiable person, • Merge the intermediate image and the skeleton table to form an enriched anonymized image including a skeleton and an outline for each identifiable person.
[0021] The invention also relates to a movement analysis system comprising an anonymization device as presented previously and a movement analysis device comprising a computer configured to implement an algorithm for determining movements from the enriched anonymized images from the anonymization device.
[0022] Preferably, the anonymization device and the movement analysis device form a single device. The analysis system is advantageously integrated.
[0023] The invention also relates to an anonymization system comprising an anonymization device as presented previously associated with at least one surveillance camera. PRESENTATION OF FIGURES
[0024] The invention will be better understood on reading the description which follows, given by way of example, and referring to the following figures, given by way of non-limiting examples, in which identical references are given to similar objects.
[0025] [Fig.l] is a schematic representation of a station in which a surveillance camera is installed.
[0026] [Fig.2] is a schematic representation of an exemplary anonymization system according to one embodiment of the invention.
[0027] [Fig. 3] is a schematic representation of an example of implementation of an anonymization method according to the invention.
[0028] [Fig.4] is a schematic representation of an example of an anonymization algorithm
[0029] [Fig.5] is a schematic representation of a processing of several enriched anonymized images.
[0030] [Fig.6] is a schematic representation of the method of temporal analysis of movements recorded by a surveillance camera on at least one image.
[0031] [Fig.7] is a schematic representation of a raw image.
[0032] [Fig.8] is a schematic representation of an intermediate image.
[0033] [Fig.9] is a schematic representation of the skeletons extracted to determine the skeleton table.
[0034] [Fig. 10] is a schematic representation of an enriched anonymized image.
[0035] It should be noted that the figures set out the invention in detail to illustrate implements the invention, said figures being able of course to serve to better define the invention where appropriate. DETAILED DESCRIPTION OF THE INVENTION
[0036] With reference to [Fig.l], there is shown schematically a surveillance camera 1 filming a scene SC comprising several identifiable people 2 positioned in a railway station, in particular, near a railway vehicle 3. The surveillance camera 1 can be positioned both in the railway station and in a railway vehicle 3 to monitor the actions carried out by the people in order to detect any accident or danger at an early stage.
[0037] The invention is presented in a railway context but it applies to any place receiving the public, in particular, an airport, a bus station, a port, a public building, etc.
[0038] In a known manner, with reference to [Fig. 2], the surveillance camera 1 acquires a plurality of raw images IMb in order to form a video stream. In [Fig. 2], there is shown an anonymization device 4, connected to the surveillance camera 1, which is configured to receive as input at least one raw image IMb from the surveillance camera 1 and to provide as output an enriched anonymized image IMa. Such an anonymization device 4 is advantageous given that it allows the enriched anonymized image IMa to be used with few constraints, in particular, to allow its consultation by non-sworn operators 8, to allow its temporary or permanent storage 9 or to allow subsequent processing 10, for example, on remote and / or third-party servers to implement artificial intelligence algorithms.Thanks to the method according to the invention, a video stream of raw IMb images at input becomes easily usable subsequently, because it complies with the regulations in force on the protection of privacy.
[0039] In this case, the anonymization of a video stream is presented, but it goes without saying that the invention also applies to one or more individual images. The raw images can be an individual image or a succession of images or a video which is a succession of temporally ordered images.
[0040] Similarly, the anonymization of images captured by a surveillance camera is presented, but it goes without saying that the invention applies to images captured by any image acquisition device.
[0041] As illustrated in [Fig.2], the anonymization device 4 comprises a computer 40 configured to implement an anonymization method which will now be presented. In this example, the computer 40 comprises a computer processor which may take various forms, in particular, an FPGA or a graphics card.
[0042] In this example, as illustrated in [Fig.7], each raw image IMb comprises two identifiable people 2 who interact together.
[0043] [Fig. 3] schematically represents an implementation of an anonymization method according to the invention by means of the anonymization device 4. The anonymization method comprises steps consisting of: • Process the raw image IMb by an anonymization algorithm configured to anonymize the identifiable persons 2 on the raw image IMb so as to obtain an intermediate image IM1 comprising a contour C for each identifiable person 2 as illustrated in [Fig.8], • Process E2 the raw image IMb by a skeletonization algorithm configured to determine a skeleton S for each identifiable person 2 so as to obtain a skeleton table TS comprising a skeleton S for each identifiable person 2, each skeleton S comprising points of interest determined for each identifiable person 2 connected by segments as illustrated in [Fig.9], • Merge E3 the intermediate image IM1 and the skeleton table TS so as to form an enriched anonymized image IMa which includes the contour C and the skeleton S of each person 2 as illustrated in [Fig. 10].
[0044] In practice, the relevant points of each skeleton are added to the intermediate image.
[0045] In this example, the contour C is a closed contour. It goes without saying that the invention could also be applied to an open contour.
[0046] Preferably, access to the raw IMb images recorded in real time by the surveillance camera 1 is done by using an RTSP protocol (Real Time Streaming Protocol). Preferably, several consecutive raw IMb images can be processed together, regardless of their recording format.
[0047] Preferably, steps E1 and E2 are implemented in parallel in order to quickly obtain the enriched anonymized image IMa, preferably in real time.
[0048] The anonymization step E1 processes a raw image IMb and modifies it to give an intermediate image IM1 which cannot be used by the skeletonization algorithm. Indeed, the anonymization will degrade the relevant information relating to the skeleton S, which will prohibit any interpretation of the movement based on the skeleton. The skeletonization step E2 processes a raw image IMb to determine the skeletons S of the identifiable persons 2 present on the raw image IMb, and extract them to a skeleton table TS. According to one aspect, the skeletonization step E2 also determines, in a preliminary manner, a bounding box for each identifiable person in order to detect the skeleton. Recording the coordinates of this box The bounding box advantageously allows for greater robustness and greater confidence in the detection of skeletons S, particularly in the detection of partial skeletons S. This skeletonization step E2 has no impact on the raw image IMb, and can therefore be carried out upstream or in parallel with the anonymization step El. Putting the two steps El and E2 in parallel advantageously allows their advantages to be combined on an enriched anonymized image IMa obtained by fusion E3 of the intermediate image IM1 and the skeleton table TS. In addition, the bounding box can also be added to the skeleton table TS in order to be used for motion / behavior detection.
[0049] Optionally, with reference to [Fig. 3], the anonymization method comprises an optional step consisting of reducing the weight E0 of the raw image IMb, for example by reducing its dimensions, so as to allow the processing of the raw image lmb by any computer system regardless of its computing power.
[0050] The anonymization algorithm and the skeletonization algorithm will now be presented in detail.
[0051] With reference to [Fig.4], according to a first example of implementation, the anonymization algorithm comprises steps consisting of determining Eli the areas containing identifiable persons 2 on the raw image lmb.
[0052] Preferably, contours C of the identifiable persons 2 are determined by semantic or instance segmentation. This can be parameterized according to the future use of the enriched anonymized images Ima and the data to be extracted. This determination is more precise and allows less information to be lost than by determining a bounding box for each identifiable person present on the raw image lmb. Preferably, the detection of the contours C of the identifiable persons 2 is done by using a deep neural network, for example of the “Mask RCNN” type, known to those skilled in the art.
[0053] According to one aspect, the anonymization algorithm comprises a step consisting of sub-sampling E12 the interior of the previously determined areas. The persons 2 are therefore no longer recognizable, but the color, intensity and brightness are preserved. To anonymize the persons 2, the anonymization algorithm comprises, alternatively, a step consisting of coloring E12' the interior of the previously determined areas with a solid color mask.
[0054] According to one aspect, the anonymization algorithm comprises a step consisting of adding random noise E13 in order to prevent any reconstruction of the raw image lmb, that is to say, any possibility of identifying people 2.
[0055] According to one aspect, the anonymization algorithm comprises a step consisting of performing E14 an interpolation in order to form an intermediate image IM1 having the same resolution as the raw lmb image. This E14 interpolation advantageously allows the intermediate IM1 image to be resized to the correct dimensions while avoiding reconstructing the information removed by anonymization.
[0056] According to one aspect, the anonymization algorithm comprises a step consisting of smoothing E15 the contours of the sub-sampled zones making it possible to give a natural appearance to the intermediate image IM1. It goes without saying that such a step is not carried out when the determined zones are colored uniformly (step E12').
[0057] As illustrated in [Fig.4] and [Fig.8], an intermediate image IM1 is advantageously obtained which includes the precise contours C of the persons 2. The persons and their movements are not identifiable.
[0058] In this example, the intermediate image IM1 also includes the background F of the scene SC, here, the railway vehicle. It goes without saying that the background F could be omitted or present only on the skeleton table TS. The background F is useful but remains optional.
[0059] According to one aspect, the skeletonization algorithm implements an image processing method so as to define a skeleton S for each identifiable person 2 of the raw image lmb. In a known manner, points of interest are previously determined for each identifiable person 2, for example, at the joints (elbow, knee, etc.) and the extremities of the body (hand, foot, head). These points of interest are connected by segments in order to form a skeleton S which visually illustrates the position of the body of each identifiable person 2 without providing information on the contour of the person. Such a skeletonization method is known to those skilled in the art.
[0060] As illustrated in [Fig.9], the skeleton table TS only comprises the skeletons S of the identifiable persons 2 present in the raw image lmb. The skeleton table S comprises, for example, all the coordinates of the joints of the skeletons S, and can therefore, advantageously, be easily manipulated and processed by analysis algorithms.
[0061] With reference to [Fig. 10], during the fusion step E3, the intermediate image IM1, which comprises the contours C of the identified persons and the background F; and the skeleton table TS, which comprises the skeletons S, are fused so as to form an enriched anonymized image Ima which comprises the contours C of the identified persons associated with their skeletons S as well as the background F.
[0062] It is thus possible to obtain an enriched anonymized image Ima, the exploitation of which is easy as presented previously. Advantageously, as illustrated in [Fig.5], it is possible to form an enriched anonymized video stream VIDa from a plurality of enriched anonymized images Ima obtained over time.
[0063] With reference to [Fig.5], the enriched anonymized video stream VIDa is provided to a motion analysis device 5. In this example, the motion analysis device 5 comprises a computer 50 configured to implement a motion analysis method so as to provide at least one RA alert report in the event of detection of an abnormal movement. In this example, the computer 50 is a computer processor which may take various forms, in particular, an FPGA.
[0064] According to a particular aspect, the movement analysis device 5 and the anonymization device 4 are the same equipment.
[0065] In this example, with reference to [Fig.6], the motion analysis method comprises steps consisting of: • Determine E51 the movements of the people present image by image, in in particular, by analyzing the evolution of the postures of the skeletons S of said people. • Compare E52 the determined movements to a movement database DBm associating a plurality of movements with movement alerts, so as to detect at least one movement alert Alm.
[0066] Preferably, the comparison step E52 is carried out by a classification algorithm, for example, a deep neural network.
[0067] According to one aspect of the invention, the step of determining the movements E51 can be carried out by an analysis of the deformation of the contours C from one image to another. The analysis of the movements, that is to say the succession of postures, is thus improved and more robust given that it is carried out, on the one hand, from the skeletons S and, on the other hand, from the deformation of the contours C.
[0068] The RA alert report includes the Alm movement alert and makes it possible to provide precise information to an operator, for example, a person falling in order to be able to alert the emergency services quickly. The presence of the C contours makes it possible to better contextualize the understanding of the SC scene and to facilitate interpretation. In addition, the coupling of the use of the C contours and the S skeletons provides great robustness of detection and analysis.
[0069] The anonymization and motion analysis methods can advantageously be implemented in real time, i.e. at the same time as the raw images lmb are acquired. They can also be implemented after acquisition, and without the anonymization device 4 or the motion analysis device 5 being connected to an Internet network.
[0070] Thanks to the RA alert report, it is thus possible to carry out a statistical analysis on the data obtained from the enriched anonymized images Ima, in order to enable an improvement of the services carried out in / on the station and in / on the railway vehicles.
Claims
Claims
1. Method for anonymizing raw images (lmb) from at least one surveillance camera (1), the raw images (lmb) comprising at least one identifiable person (2), the method comprising, for at least one raw image (lmb), steps consisting of: • Processing the raw image (lmb) by an anonymization algorithm (El) configured to anonymize each identifiable person (2) on the raw image (lmb) so as to obtain an intermediate image (IM1) comprising a contour (C) for each identifiable person (2), • Processing the raw image (IMb) by a skeletonization algorithm (E2) configured to determine points of interest of each identifiable person (2) on the raw image (IMb) so as to obtain a skeleton table (TS) comprising a skeleton (S) for each identifiable person (2),• Merge (E3) the intermediate image (IM1) and the skeleton table (TS) so as to form an enriched anonymized image (IMa) comprising a skeleton (S) and an outline (C) for each identifiable person (2).,
2. A method according to claim 1, wherein the contour (C) is closed and defines an anonymized inner area.
3. Method according to one of claims 1 to 2, comprising a step consisting of reducing the weight (E0) of the raw image (IMb) prior to the processing steps (El, E2).
4. Method according to claim 2 to 3, wherein the anonymization algorithm comprises a step of sub-sampling (El2) the area inside the contour (C).
5. Method according to claim 2 to 3, wherein the anonymization algorithm comprises a step of colorizing (E12') the area inside the contour (C).
6. Method for analyzing movements from raw images (IMb) from at least one surveillance camera (1), the raw images (IMb) comprising at least one identifiable person (2), the method comprising, for at least one raw image (IMb), steps consisting, for each identifiable person (2), in: • Implement the anonymization method according to one of claims 1 to 5 to form a plurality of enriched anonymized images (IMa) and • Implement an algorithm for determining movements by analyzing the skeleton (S) of the enriched anonymized images (IMa).
7. Method for analyzing movements according to claim 6, in which the algorithm for determining movements comprises steps consisting of: • Determining (E51) the movements of the person by analyzing the skeleton (S), • Comparing (E52) the determined movements with a movement database (DBm) associating a plurality of movements with movement alerts (ALm) so as to determine, where appropriate, at least one movement alert (ALm).
8. Computer program type product, comprising at least one sequence of instructions stored and readable by a processor and which, once read by this processor, causes the steps of the method according to one of the preceding claims to be carried out.
9. A computer-readable medium comprising the computer program product of claim 8.
10. Anonymization device (4) for raw images (IMb) from at least one surveillance camera (1), the raw images (IMb) comprising at least one identifiable person (2), the anonymization device (4) comprising a calculator (40) configured to implement, for at least one raw image (IMb), steps consisting of: • Processing the raw image (IMb) by an anonymization algorithm (El) configured to anonymize each identifiable person (2) on the raw image (IMb) so as to obtain an intermediate image (IM1) comprising a contour (C) for each identifiable person (2), • Processing the raw image (IMb) by a skeletonization algorithm (E2) configured to determine points of interest of each identifiable person (2) on the raw image (IMb) so as to obtain a skeleton table (TS) comprising a skeleton (S) for each identifiable person (2), • Merge (E3) the intermediate image (IM1) and the skeleton table (TS) to form an enriched anonymized image (IMa) comprising a skeleton (S) and an outline (C) for each identifiable person (2).
11. Motion analysis system comprising an anonymization device (4) according to claim 10 and a motion analysis device (5) comprising a computer (50) configured to implement a motion determination algorithm (5) from the enriched anonymized images (IMa) from the anonymization device (4).
