Method intended to protect personal data of individuals present in a public space

Abstract

The invention relates to a method intended to count people using image-capturing means, so as to allow images of people to be captured, the image-capturing means supplying initial data in a first step, and then the image-capturing means are equipped with a processing unit for processing the initial data, the method being characterized by a second step in which the processing means average n initial data, n being a natural integer, forming a set of digital data called averaged data, a third step in which the processing means communicate the averaged data to operating means in order to have these averaged data displayed by visualization means; in a fourth step, the processing means disable the means for communicating the data to the external operating means.

Description

METHOD INTENDED FOR THE PROTECTION OF PERSONAL DATA OF INDIVIDUALS PRESENT IN A PUBLIC SPACE.

[0001] The present invention relates to methods implemented to protect the personal, biometric data of individuals present in a public space, for example where people counting operations are carried out using image capture devices hereafter referred to as cameras.

[0002] Indeed, there are many applications for counting, for example to make public transport routes more fluid (optimization of bus passages, "smart city"), for improving road traffic flow (triggering a red light), for optimizing public lighting (presence or absence of people in the lit area) or even to identify free spaces in a car park (counting "vehicle" objects).

[0003] For people counting, the analysis of movements ("flows") also makes it possible to alert, among other things, that a space will be over capacity or to estimate the number of people passing in front of a specific point of sale and thus to ensure the relevance of the location chosen for this point of sale or in the case of a train station or metro station to determine the entrances and / or exits which are preferentially used by people but also to determine the preferential flows between these entrances / exits.

[0004] The counting can be carried out by means of exploitation from digital image data, hereafter called initial image data, delivered by cameras placed at locations chosen in the public space where the counting operations are carried out.

[0005] However, the installation of such image capture devices or cameras in these chosen locations raises the following problems: the installation of cameras in a public space is subject to legal constraints, and disseminating images of people that make their subsequent identification possible is even more sensitive, as mentioned in Article 35(1) of REGULATION (EU) 2016 / 679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 27 April 2016; determining optimal positions for these chosen camera locations in this public space, the placement of camera lenses, focusing the camera lenses, etc., requires at least temporary viewing of images of the public space by an operator on a screen, which seems to contradict, from a technical standpoint, the previous point concerning access to the personal data of individuals present in this public space.

[0006] It would also be relevant to physically separate, in a way that cannot be circumvented, a first phase of setting up these counting methods and a second phase where these counting methods are used to deliver their counting results to potentially remote operating means and where the potential dissemination of personal data of individuals present in the environment where the counting operations are carried out, for example, from the initial image data, is then prohibited.

[0007] In order to meet the constraints related to respecting individual freedoms, a process that does not involve any step of storing images in non-volatile memory and does not involve any step using means of transmitting personal individual data on the electronic networks to which the means of operation necessary for carrying out the process are connected either by radio link or by wired link (WiFi link, Ethernet link, USB link, RS232 serial link, among other non-restrictive examples) would then be compatible, from a legal point of view, with the use of image capture devices in public spaces.

[0008] The present invention makes it possible to solve the problems mentioned above.

[0009] To this end, a method according to the invention for the optimal implementation of an image processing device intended for applications of counting people and / or objects in an environment to be monitored, the method guaranteeing the anonymity of the people present in the environment to be monitored during its implementation and then during the counting phase itself, comprises the following steps:

[0010] - We use image capture devices arranged to allow the capture of images of people and / or objects in an environment to be monitored; these image capture devices provide, over time, digital data representative of the images, called initial data, from a device called a video sensor.

[0011] - The image capture devices are equipped with processing units to process the initial data; the processing units can be either on the same printed circuit board as the image capture devices, or located remotely from them.

[0012] - External operating methods are used to collect digital data from the processing of initial data by the processing unit in order to optimize the placement and orientation of the imaging equipment for counting applications,

[0013] - High-speed wireless and / or wired communication methods are used for communication between the processing unit and external operating systems,

[0014] - Low-bandwidth wireless and / or wired communication methods are used, which are incompatible with the transmission of a video stream, even compressed, for communication between the processing unit and remote operating means, arranged to collect and then process the counting results.

[0015] and is essentially characterized by:

[0016] - In a first step, the power supplies for the high-speed wireless and / or wired communication links are deactivated and the power supplies for the image capture system are activated simultaneously via a single power supply controlled by the processing unit,

[0017] - in a second step according to the process, the processing means average over time n initial data, n being a natural number greater than 1, to form a new set of numerical data over time called averaged data, which has the effect of erasing from the numerical data everything that is representative of movement in the environment to be monitored, in particular created by the people who circulate in it, and of keeping only the numerical data representative of static objects ("called background") in an anonymized manner,

[0018] - in a third step according to the process, the processing unit deactivates the power supply to the image capture means and simultaneously activates the power supply for the high-speed communication means via a single hardware link of the processing unit which controls their power supply control input pin(s), the processing means communicate the averaged data to operating means in order to allow the representation of this averaged data by visualization means,

[0019] - in a fourth step according to the process, the processing means activate the image capture means and simultaneously deactivate the high-speed communication means via the single hardware link of the processing unit which controls these power supplies, the processing unit then being able to communicate only the counting results to the remote operating means via the low-speed communication means.

[0020] In this way, image capture devices can be installed in a public space for counting operations while respecting the personal data of individuals present in said public space.

[0021] In preferred embodiments, one and / or the other of the following arrangements are used: the value n used for averaging the initial data is greater than or equal to 1 and preferably greater than 50 to erase the data related to movement and keep only the background of the scene, the processing unit is equipped with a communication module enabling communication via radio waves with external operating means, the processing unit is equipped with a communication module enabling communication via wired means with external operating means, the processing unit controls and authenticates the operation of the shooting system, when the processing unit is equipped with a communication module, when the communication module is removed during step 3 according to the method, then the counting step (4) is automatically selected.

[0022] The invention includes, apart from these main provisions, certain other provisions which are preferably used at the same time and which are discussed in more detail below.

[0023] In what follows, we will describe some preferred embodiments of the invention with reference to the figures attached hereto in a manner which is of course not limiting.

[0024] In the drawings,

[0025] The figure is a schematic view of the implementation of a counting device after using the method according to the invention.

[0026] The figure is an electronic diagram used during the third and fourth steps of the process according to the invention.

[0027] An example of a system set up for taking pictures of people (31) (32) (33) and / or pre-defined objects, for example "car", "motorcycle", "bicycle", etc., is described, comprising a shooting system (2)("camera") arranged to take pictures of these people and / or these pre-defined objects present in a public space (1).

[0028] The subspace (111) is defined as the part of space (1) that is visualized by the camera system (2).

[0029] When the system is used for object searches, the nature of these objects, for example: car, motorcycle, etc., is predefined or "predefined" by the user of the counting system described.

[0030] As is known per se, such a camera system (2) comprises at least one processor (21), for example, a microprocessor, such as the NXP iMX8M Mini, with 1, 2, or 4 cores, whose operating frequencies are greater than or equal to 800 MHz. The power supply (5) is provided by a device known per se, such as a MeanWell LRS-75-12 power supply. The program code (250) necessary for the operation of the processor (21) is executed here from an LPDDR type random access memory (RAM) connected to the processor, for example, a Micron MT53D512M32D2DS memory (25) with 2 GB of RAM, where by convention 1 GB = 10 9 bytes and 1MB=10 6bytes, but the processor can also use RAM with smaller capacities (1 GB, or even 512 MB or 256 MB). At least one non-volatile memory (23) is connected to the processor (21). It contains the processor's program code to enable autonomous startup and also allows for the storage of additional information such as camera operating parameters (exposure time, gain, resolution, among other non-exhaustive examples). This memory (23) is, for example, a MICRON MT25QU256ABA1EW7-0SIT memory, which has 32 MB of non-volatile memory.In one embodiment, when the processor (21) uses an advanced real-time operating system such as Linux, it is advantageous to use a second type of non-volatile memory with a larger capacity: the processor (21) is then connected to an eMMC (embedded multimedia card) with a memory capacity of 1 GB or more. For example, a Toshiba THGBMJG6C1LBAIL memory (24) with a capacity of 8 GB. This electronic assembly, comprising the processor (21) and the various RAM (25) and non-volatile (23) and / or (24) memory modules, constitutes a processing unit of the system.

[0031] The camera (2) also includes an imaging sensor (22). The sensor (22) delivers images in the form of digital data, hereafter referred to as initial image data, and is normally equipped with an optic (220).The sensor (22) is connected in a manner known per se to the processor (21), via a connection (28), this connection being, for example, a direct parallel connection between the pins of the processor (21) and those of the sensor (22) or a serial connection between the pins of the processor (21) and those of the sensor (22) such as I2C, SPI, and even a differential pair connection ("differential link") with a specific communication protocol such as mipi csi (Mobile Industry Processor Interface camera serial interface), lvds (low voltage differential signaling), sub-lvds, slvs-ec (Scalable Low Voltage Signaling with Embedded Clock) but also the 2-wire LAN communication protocol using not a coaxial cable but a pair of wires and a specific protocol "Windowed OFDM (Orthogonal Frequency Division Multiplexing)", among the non-restrictive examples of possible implementations.

[0032] The image sensor (22) is for example an OnSemi AS0149 component delivering images with a resolution of up to 1280x960 image elements or pixels in the form of digital data, and normally equipped with optics (220), for example with an M12 mount, adapted to the size of said sensor.

[0033] The image sensor (22) can also be a global shutter sensitive component such as a monochrome OnSemi AR0144 sensor or sensitive to the so-called thermal infrared spectrum such as a Lynred ATI320 sensor.

[0034] The AS0149, AR0144 and ATI320 image sensors deliver their digital data to the processor via a differential link using the mipi protocol, sometimes referred to as the mipi "link".

[0035] The optic (220) is chosen so as to be able to view the subspace (111) with a field of view suitable for shooting people and / or objects previously defined (3) in the environment (1): for example, an M12 Sunex DSL377A-650-F2.8 optic, compatible with the component (22) OnSemi AS0149 and whose horizontal field of view is 113°.

[0036] It is also possible to use a sensor (42) and an optic (420), for example respectively of the same nature as the sensor (22) previously described and the optic (220) previously described, placed this time on a separate electronic board "remote sensor board" (4), preferably of smaller dimensions than the processing unit which can advantageously be located a few meters away from the processing unit for a more discreet implementation for example.

[0037] In this latter implementation, we will subsequently refer to it as a remote sensor.

[0038] When the sensor is located away from the processing unit, specific electronic components are used to ensure the transport of information between the sensor and the processing unit.

[0039] The initial data is for example sent to the processor by means of a serial connection or "link" of type FPDLink or equivalent or a differential link of type mipi csi, lvds, sub-lvds, slvc-ec, or a GSML link or an APIX link among other non-restrictive examples of embodiment and the power supply of the card (4) containing the sensor (42) is supplied to the remote sensor card via a single coaxial cable (29), such as a cord: Johnson SMA #415-0029-3.0 whose central part carries both these initial image data and the power supply, this principle being called phantom power supply.

[0040] In one particular embodiment, the processing unit (2) may be equipped with a deserialization component such as a Texas Instruments DS90UB954 and the remote sensor board (4) may be equipped with an associated serialization component DS90UB953, thus creating an operational FPDLink serial link between the electronic board (4) and the computing entity.

[0041] Indeed, the initial digital image data, delivered according to the "mipi" standard by the sensor, are transformed by the DS90UB953 serialization component, transported via the coaxial cable, then deserialized by the DS90UB954 deserialization component and delivered to the processor (21) again in the form of initial image data according to the Mipi standard (270).

[0042] In one particular embodiment example, the remote sensor card (4) is equipped with operating means such as a microcontroller (43), for example an STM32G071GBUX having its own non-volatile memory and its own volatile memory.

[0043] The processor (21) can communicate with the microcontroller (43) using the communication link with the help of the device (29).

[0044] In this particular embodiment, the processor (21) can then regularly, for example once every 10 seconds, send commands to the microcontroller (43) in order to test the presence and correct operation of the latter by analyzing the responses provided by the microcontroller (43) to these requests.

[0045] For example, the processor (21) and the microcontroller (43) can use a process called "Rolling Code" (https: / fr.wikipedia.org / wiki / Code_tournant) when sending these test commands for presence and / or operation.

[0046] Therefore, if the microcontroller (43) malfunctions or if the remote sensor board (4) provides incorrect or incompatible responses with the sequence allowed by the "Rolling Code" process, for example due to substitution by a third party for any reason (accidental or malicious), then the processor (21) can detect this malfunction and thus invalidate its own subsequent results.

[0047] During this first step according to the process, the sensor (22) or the remote sensor (42) are powered by power supply means.

[0048] In a preferred embodiment, to implement these means, an output of the power supply (5) is used, followed by electronic circuits specialized in generating power supplies such as Diodes Incorporated AP22811, whose operation can be controlled from an input / output pin of the processor (21).

[0049] In this first stage of the process according to the process and prior to a counting process proper, the processing unit is also equipped with a communication module.

[0050]

[0051] The processing unit can therefore be equipped with a radio communication module, for example, without limitation: Wifi, Bluetooth, LoRa, ISM (433 MHz, 868 MHz, 915 MHz or 2.4 GHz).

[0052]

[0053] In one particular embodiment the communication module uses the Wifi standard, for example: a TP-Link AC600 WiFi USB Key module (26).

[0054] In this first step of the process according to the invention, the power supply is switched off by the processing unit.

[0055] To achieve an exclusion between the operation of the power supply of the communication module (26) and that of the sensor (22) or the remote sensor (42), it is planned to use an electronic control from a single output pin (2110) of the processor (21), whose logical behavior is reversed between that for the power supply of the communication module, for example the Wifi module (26), and that for the power supply of this second source of power supply of the sensor (22) or the remote sensor (42).

[0056] For example, it is possible to use AP22811 Diodes Incorporated integrated circuits, designated AP22811A (141) and AP22811B (142), for these two different power supplies. These integrated circuits have strictly opposite activation mechanisms. The same activation signal on an AP22811A integrated circuit will be a deactivation signal on an AP22811B integrated circuit, and vice versa. Thus, the operation of the activation pin of these components is physically reversed between the AP22811A (1410) and AP22811B (1411) models. When the activation pins of these two components are physically connected on the electronic board, it is guaranteed that the 2 power supplies generated at the output of the components by respectively, the AP22811A model, on its output pin (1411), and the AP22811B model, on its output pin (1421) can never be simultaneously active.

[0057] This can also be achieved by connecting the power control signal of the communication module (26) to the pin (2110) of the processor (21) and using an inverter logic circuit whose input is this same processor pin and whose output is the inverse of the processor pin to control the second power supply of the video sensor.

[0058] For example, a Texas Instruments SN74LVC1G04 logic circuit can be used to perform this logic inversion.

[0059] In this way, the two power supplies for the high-speed communication module (26) and for the sensor can never be in operation simultaneously.

[0060]

[0061] In a second step of the implementation of the process according to the invention, during the setting up of the image capture system (2) as previously presented, the processor (21) averages the initial image data over time by accumulating the data of n successive images, n being a natural number greater than or equal to 1, for example 50, then divides each cumulative value of the image elements, also called pixels, by n, which has the effect of making the details of objects and / or people moving in the subspace (111) of the space (1) viewed by the camera (2) disappear from the result following these two processing steps.

[0062] When the sensor delivers the initial data at a rate of 20 images per second, 50 successive images thus represent 2.5s of cumulative images.

[0063] On the other hand, the details of the static objects present in this subspace (111) will also remain present in the averaged image representing this average of the n successive images.

[0064] Any movement of the person, even a blink of the eye, will therefore result in the disappearance of details of the person's face in the result following these two processing steps; the 2.5s representing on average the period of a blink.

[0065] Values ​​of n greater than 50 can of course be used.

[0066] Thus, this second step of the process according to the invention allows for easier selection of locations for an operator responsible for setting up the system in order to place the camera in space (1) to view the subspace (111), capable of delivering the counting data, by an operator responsible for setting up the counting system (2).

[0067] Indeed, during this second step according to the process, the processor creates a set of digital image data called averaged data, all static objects can be viewed without alteration of quality and therefore the choice of viewing angles and the focusing of the camera optics (2) can be carried out more easily by an operator in charge of setting up the counting system by the diffusion by the processor (21) to means of operation external to the system (2) of this averaged data.

[0068] On the other hand, all moving objects during these n images, especially people moving around in the area to be monitored, will become indistinguishable on the averaged image because they will never appear in the same position on the different images taken during the time they are in front of the suitable field of view (3) and the averaging operation will very significantly reduce their weight in the final averaged image compared to the background objects of the scene which are static and will continue to appear in the same place: therefore, the identification of people and / or moving objects will no longer be possible by any means of exploitation whatsoever.

[0069] This step in the process allows the operator in charge of setting up the counting system to avoid setting up areas inaccessible to people moving around in the area to be monitored during the installation of the camera system.

[0070] When the processor (21) has created the set of averaged data, the second step of the process according to the invention is completed and we proceed to a third step.

[0071] In a third step according to the process, the processor (21) switches the state of its pin (2110) in order to disable the power supply to the sensor (22) or remote sensor (22) and simultaneously to enable the power supply to the communication module (26).

[0072] The operator in charge of setting up the device (2), using the results obtained during the second step according to the process, i.e. the averaged data, can validate a chosen location by viewing on a computer (7) having a communication module compatible with the module (26), the subspace (111), by checking the focus setting of the optic (220) of the sensor (22) or of the optic (420) of the remote sensor (42) and possibly of the focal length of the optic (220) or of the optic (420).

[0073] It should be noted that during this third stage, the processing unit can also communicate with external operating means using the high-speed link, parameter data such as the position of the image capture systems, their optics, etc., with the exception of any digital data representing images subsequent to the end of the second stage according to the process, the power supply to the image capture system being deactivated at this time.

[0074] At this stage of operation according to the process, if the results of the averaged data are not validated by the operator in charge of setting up the device, for example due to a blurry rendering of the averaged data, then the latter sends a command to the processor (21) using the communication device (26) from the computer (7) in order to return the process according to the invention to the first stage.

[0075] In this case, the processor switches pin (2110) again to reactivate the shooting system and simultaneously deactivate the communication module (26).

[0076] When the operator in charge of setting up the camera system has satisfactorily positioned the device (2), the third step in the process is completed and the rest of the process is carried out in a fourth step.

[0077] The operator can send a process step change command by addressing a command to the processor (21) using the device (26) from the computer (7).

[0078] During the fourth step of the process according to the invention, the processor (21) again changes the state of its input output pin (2110) in order to again disable the power supply to the communication module (26), which is a high-speed link compatible with image broadcasting, and simultaneously the processor activates by construction the power supply to the sensor (22) or the remote sensor (42).

[0079] In a particular embodiment of the process in order to proceed to this fourth step, a pin of the processor (21) can be used to automatically detect the removal by the same operator of the communication module (26) in order to automatically complete this third step and proceed to the next step.

[0080] During this fourth stage, the processor (21) can only communicate with remote processing means and using, in particular for counting, a very low bandwidth link.

[0081] It should also be noted that the communications necessary for counting are carried out using only this very low bandwidth link (< 115.2 Kbits / s) which therefore no longer allows, from the fourth step according to the process, the sending of digital data representing images ("video") in real time or delayed even compressed by a process such as "jpeg" among other non-restrictive examples.

[0082] Regardless of the method of implementation of the physical disconnection between the processor (21), the sensor (22) or the remote sensor (42) and the communication module (26), this disconnection therefore makes it impossible to broadcast the initial data or the data averaged by the high-speed communication module and during this fourth step of the process according to the invention, personal data, for example biometric data such as faces even altered by the averaging of step 2 of the process according to the invention, of individuals present in the space (1) can therefore never be communicated to means of operation external to the camera (2) and only the counting results, communicated by this very low-speed link are communicated to means of operation external.

[0083] Indeed, the communications necessary for counting are carried out using only a very low bandwidth link (<= 115.2 kbits / s), which would not allow the broadcasting of a real-time video stream.

[0084] In a particular example of implementation, it is possible to limit the set of data transmitted for counting to a set of a few dozen bytes transmitted per day, for example 64 bytes, which still allows the values ​​reached for 16 counters in the middle (1) to be represented each day on 32 bits of dynamic range.

[0085] It is therefore no longer possible to speak of video streams: a single image with a resolution of 320x200 pixels, assuming a jpeg-type compression of a factor of 10, would still take 100 days to be transmitted to remote operating means using such a rate.

[0086] This therefore leads to this surprising result for the person skilled in the art where although the camera (2) has been installed and used in the public space (1), at no time is it possible to disseminate personal data by exploiting the initial image data and only counting data are communicable to means of remote exploitation of the camera (2) after the process according to the invention has been applied.

[0087] As is self-evident, and as follows from the foregoing, the invention is not limited to the particular embodiments just described; on the contrary, it encompasses all variants thereof.

Claims

A method for the optimal implementation of an image processing device intended for counting people and / or objects in a monitored environment, said device comprising an element which will be disabled by the last step of the method in order to guarantee the anonymity of the people present in the monitored environment, comprising the following steps: image capture means are used, arranged so as to allow the capture of images of people and / or objects in a monitored environment, the image capture means providing over time digital data representative of the images, called initial data, from a device called a video sensor; the image capture means are equipped with processing units in order to process the initial data; the processing units may be either on the same printed circuit board as the image capture means, or remote from them.External operating means are used to temporarily collect digital data resulting from the processing of initial data by the processing unit in order to optimize the placement and orientation of the image capture devices for counting applications; high-speed wireless and / or wired communication means are used for communication between the processing unit and the external operating means; low-speed wireless and / or wired communication means, incompatible with the transmission of a video stream, even compressed, are used for communication between the processing unit and remote operating means, arranged to collect and then process the counting results.and is essentially characterized by: in a first step, the power supplies for the high-speed wireless and / or wired communication links are deactivated and the power supplies for the image capture device are activated simultaneously via a single hardware power supply controlled by the processing unit; in a second step, according to the process, the processing means average over time n initial data points, n being a natural number greater than or equal to 1, to form a new set of digital data over time called averaged data, which has the effect of erasing from the digital data everything that is representative of movement in the environment to be monitored, in particular created by people moving through it, and of keeping only the digital data representative of static objects ("called background") in an anonymized manner; in a third step, according to the process,The processing unit deactivates the hardware power supply to the image capture means, which simultaneously activates the power supply for the high-speed communication means via a single hardware link of the processing unit that controls their power supply input pin(s). The processing means communicate the averaged data to the external operating means, with the exception of any digital data subsequent to the second step, and this from the beginning of this third step, the image capture system being deactivated at this time. In a fourth step according to the process, the processing means activate the image capture means and simultaneously deactivate the high-speed communication means via the single hardware link of the processing unit that controls these power supplies.The processing unit could then only communicate the counting results to remote operating equipment via low-speed communication channels. A method according to claim 1 wherein the value n used for averaging the initial data is greater than or equal to 1 and preferably greater than 50. A method according to any one of the preceding claims, wherein the processing unit is equipped with a communication module enabling communication via radio waves with external operating means. A method according to any one of the preceding claims, wherein the processing unit is equipped with a communication module enabling wired communication with external operating means. A method according to any one of the preceding claims characterized in that the operating means control and authenticate the operation of the image capture system. A method according to any one of the preceding claims wherein, when the processing unit is equipped with a communication module, when the communication module is removed during step 3 according to the method, then the counting step (4) is automatically selected.

Images