Secure autonomous counting device

Abstract

The invention relates to a device for estimating the number of people passing through k predefined counting positions as they move in an environment: an image-capturing system acquires, in the environment to be monitored, images of this environment and of the people moving therein, and delivers digital image data to first processing means, which first processing means then create a set referred to as restricted data; the first processing means use a neural network to estimate the position and movement of the people based on the restricted data; when these people pass through one or more counting positions, the counter associated with each counting area is incremented; after a chosen time period, the first processing means of the device communicate the values of all of the counters to second processing means, and the first processing means then reset these counters.

Description

SECURE AUTONOMOUS COUNTING DEVICE.

[0001] The present invention relates to the field of image processing and more particularly to electronic devices which, from the capture of images, called initial images, are implemented for counting and in particular for counting people.

[0002] There are many applications for counting, for example to make public transport routes more fluid (optimization of bus passages, "smart city"), to improve road traffic flow (triggering a red light), to optimize public lighting (presence or absence of people in the lit area) or to identify free spaces in a car park (counting "vehicle" objects).

[0003] For people counting, the analysis of movements (“flows”) makes it possible to alert, among other things, that a space will be over capacity or to estimate the number of people passing in front of a specific point of sale and thus to ensure the relevance of the location chosen for this point of sale or in the case of a train station or metro station to determine the entrances and / or exits which are preferentially used by people but also to determine the preferential flows between these entrances / exits.

[0004] Patent CN111860162B describes such a video method for counting people present in a crowd in an environment, including a step of extracting physical characteristics and this on the best image definition.

[0005] Such a step makes it possible to authenticate individuals within a crowd, which goes against the constraints of respecting individual freedoms.

[0006] Furthermore, this patent does not allow for the estimation and / or measurement of the movement of people in the visualized environment, as the count is performed on each image without any link to the previous one(s).

[0007] Patent CN110135325B also describes a video method for counting people that implements a neural network.

[0008] The process determines, from the initial images, potential positions of people's heads and applies a magnification ("zoom"): again, this step makes it possible to authenticate individuals present in a crowd of people.

[0009] Patent JP5834254B2 describes another video counting system intended to be placed inside a store and to analyze the movement of people in that environment, but it does not allow for a precise analysis of the flow of people moving within the environment.

[0010] Indeed, it assumes operation with one entrance and one exit, which is perfectly legitimate for a small shop but will be unsuitable for a much larger space, such as a metro station for example where the environment to be monitored may have several entrances and / or several exits and movement of people in all directions between these entrances and / or exits.

[0011] Patent CN109376637B describes a method and video counting system using neural networks trained for this purpose.

[0012] On the other hand, the neural networks described are trained to extract the positions and faces of people in the image from the initial data using 3 networks, which, on the one hand, proves costly in computing resources and, on the other hand, does not satisfy the constraints related to respecting individual freedoms since the data extracted in this way allows, once again, individual identification of people in the environment.

[0013] Patent WO2018059408A1 describes a method and device using, once again, neural networks and their training for counting applications and in particular applications for counting people crossing predefined lines in an environment ("cross-line counting method").

[0014] However, the use of the steps and devices presented in the method in question entails the following problems: The image data is processed at the initial resolution of the image sensor ("original frame images"), and no method is disclosed for reducing its size and / or resolution; furthermore, this implies that some or all of the operations are performed from the initial resolution of the image sensor. When a step is performed to determine the presence of people in the environment to be monitored, based on the image data from its initial resolution, this step again generates new image data, which may, perhaps occasionally, allow for the individual identification of all the people whose presence has thus been detected in the environment to be monitored. The components necessary for the implementation of the device, as described in the patent, make it particularly bulky: indeed, the keyboard,Mouse, screen, etc., and even hard disk are described as "connected" to the device, which makes it impossible to place it simply or discreetly, or in a self-contained and / or very low-power electronic system. The neural network targets the head of people and not the entire body to perform certain steps ("position information of the pedestrian head"). The detection of people in images using a neural network, as described by the patent, does not corroborate its results obtained with other physical elements such as motion detection based on inter-image changes or colorimetric criteria, which are particularly relevant for obtaining much more reliable and anonymous tracking of detected individuals over time, because these elements can be performed using a much lower image resolution than that provided by the image sensor.

[0015] Furthermore, this patent refers, for the detection of people by use of a neural network, to patent CN105590094A, which in fact also presents the same problems during its implementation because it also uses the neural network without control with other physical elements.

[0016] In order to meet the constraints related to respecting individual freedoms, it is therefore relevant to design a system which, at no time, stores images in non-volatile memory and which never transmits any image on the electronic networks to which it may be connected (Wi-Fi connection, Ethernet connection, USB connection, etc.) during its operation as a counting system, which facilitates its use in public spaces from a legal point of view and which finally applies different calculation processes from an image definition deliberately degraded compared to the initial image data in order to make it impossible to authenticate the people present in the environment, at each shot, and from the first stage of the system's operation.

[0017] The present invention makes it possible to solve the problems mentioned above.

[0018] To this end, the invention relates to an image processing device intended for applications involving counting people and / or objects, comprising:

[0019] - a camera system, arranged to allow the capture of images of people and / or objects in an environment to be monitored, the camera means providing at successive moments digital data representative of the images called initial data,

[0020] - the first means of operation in the form of an electronic card called a processing unit and having at least one computing unit in order to process this initial data and create a new set of data called counting results,

[0021] - remote operating equipment, arranged to collect and then process the counting results,

[0022] and is essentially characterized by:

[0023] - The initial processing methods subsample the initial data to create a restricted dataset,

[0024] - The first methods of operation use non-volatile memory in which k different counting zones have been stored, k being an integer greater than or equal to 1, to define k different counting zones, these zones having been defined beforehand during a phase called phase

[0025] implementation,

[0026] - The first means of exploitation use an electronic neural network arranged to exploit restricted data, in a so-called counting phase in order to determine if defined people and / or objects move over time in the environment and then to estimate the number of defined people and / or objects intercepting over time each of the k counting zones defined in the environment to be monitored from the restricted data and thus create the counting results,

[0027] - the initial operating means transmit the counting results to remote operating means, excluding all initial and restricted data,

[0028] - The image capture system communicates regularly with the processing unit to prevent any misappropriation of the digital data representing the images and to interrupt their transfer in case of misappropriation and / or malfunction.

[0029] In preferred embodiments, one or both of the following arrangements are used: the device is arranged so that the initial operating means erase the original image data once the restricted data has been created and stored; the device is arranged so that the image capture system can be on the same printed circuit board as the processing unit or remote; the device is arranged so that when the image capture system is remote from the processing unit and connected to it by means of a coaxial cable using carrier current and modulation to carry the digital data from the sensor to the processing unit; the device is arranged so that the processing unit has temporary access, and only during the implementation phase, to a special electronic transport network in order to communicate restricted data.Access to this electronic transport network becoming physically impossible during the counting phase, the device is arranged so that the particular electronic transport means uses a radio frequency connection, the device is arranged so that the particular electronic transport means uses a wired connection, the device is arranged so that the values ​​reached by the digital counters C, i ; i being the index of the counting position Z i Among the k counting positions, after a delay DY, values ​​are communicated via a transmission network and then reset to zero after this operation. The device is arranged so that, when the counter C i has not reached a threshold noted S comp after the DY delay, then the C counter i is set to 0 and the value reached is communicated by the device for the counting zone Z iis zero, the device is arranged so that the means of transport used to communicate the values ​​of the meters in the metering zones is a wired connection, the device is arranged so that the means of transport used to communicate the values ​​of the meters in the metering zones is a wireless connection, the device is arranged so that its power supply can be provided by a solar panel, the device is arranged so that its power supply can be provided by a wind turbine, in the case where the device is powered by a solar panel, the solar panel is sized so as to be able to operate the device, but also to recover more energy in order to store it so that the device can operate, including when there is no sun, in the case where the device is powered by a wind turbine,The wind turbine is sized to power the device, but also to capture more energy for storage so that the device can function even when there is no wind. In cases where the device is powered by a solar panel or a wind turbine, an energy storage system is planned to capture the surplus energy produced.

[0030] The invention includes, apart from these main provisions, certain other provisions which are preferably used at the same time and which are discussed in more detail below.

[0031] In what follows, we will describe some preferred embodiments of the invention with reference to the figures attached hereto in a manner which is of course not limiting.

[0032] In the drawings,

[0033] This is a schematic view of the implementation of the device according to the invention.

[0034] This is a schematic view of the decision-making steps carried out by the first means of operation to perform the count at a given time t.

[0035] We describe a system set up for taking pictures of a set (3) of people (31) (32) (33) and / or predefined objects, for example "car", "motorcycle", "bicycle", etc., present in an environment (1) to be monitored and comprising a system of image capture (2)("camera") arranged to take pictures of these people and / or these predefined objects present in the environment (1).

[0036] When the system is used for object searches, the nature of these objects, for example: car, motorcycle, etc., is predefined or "predefined" by the user of the counting system described.

[0037] As is known per se, the image capture system or camera (2) comprises at least one processor (21), for example, a microprocessor, such as the NXP iMX8M Mini, with 1, 2, or 4 cores depending on the version, and operating frequencies of 800 MHz or higher. The power supply (5) is provided by a device known per se, such as a MeanWell LRS-75-12 power supply. The program code (250) required for the operation of the processor (21) is executed from LPDDR-type random access memory (RAM) connected to the processor, for example, a Micron MT53D512M32D2DS memory (25) with 2 GB of RAM. The processor can also operate with smaller memory capacities of 1 GB, 512 MB, or even 256 MB. At least one non-volatile memory (23) is connected to the processor (21).It contains the processor's program code to enable autonomous startup and also allows for the storage of additional information such as camera operating parameters (exposure time, gain, resolution, among other non-exhaustive examples). This memory (23) is, for example, a MICRON MT25QU256ABA1EW7-0SIT memory, with 32 MB of non-volatile memory. In a preferred embodiment, when the processor (21) uses an advanced real-time operating system such as Linux, it is advantageous to use a second type of non-volatile memory with a larger capacity: the processor (21) is then connected to an eMMC (embedded multimedia card) with a memory capacity greater than or equal to 2 GB. By definition, 1 GB = 10. 9 bytes and 1 MB = 10 6bytes. For example, a Kioxia THGBMJG6C1LBAIL memory (24) with a capacity of 8 GB. This electronic assembly, comprising the processor (21) and the various RAM (25) and non-volatile (23) and / or (24) memories, constitutes the processing unit of the system described.

[0038] The camera (2) also includes an imaging sensor (22). The sensor (22) delivers images in the form of digital data, hereafter referred to as initial image data, and is normally equipped with an optic (220).The sensor (22) is connected in a manner known per se to the processor (21), via a connection (28), this connection being, for example, a direct parallel connection between the pins of the processor (21) and those of the sensor (22) or a serial connection between the pins of the processor (21) and those of the sensor (22) such as I2C, SPI, HiSPI, FPDLink or equivalent, GSML or APIX or even a connection on differential pairs (“differential link”) with a specific communication protocol such as mipi csi (Mobile Industry Processor Interface camera serial interface), lvds (low voltage differential signaling), sub-lvds, slvs-ec (Scalable Low Voltage Signaling with Embedded Clock) or also the 2-wire LAN communication protocol using not a coaxial cable but a pair of wires and a specific protocol “Windowed OFDM (Orthogonal Frequency Division Multiplexing)”, among the non-restrictive examples of possible implementations.

[0039] The image sensor (22) is for example an OnSemi AS0149 component delivering images with a resolution of up to 1280x960 image elements also called pixels in the form of digital data, and normally equipped with optics (220), for example with M12 mount, adapted to the size of said sensor.

[0040] The image sensor (22) can also be a component sensitive to the near-infrared spectrum such as a monochrome OnSemi AR0144 sensor or even sensitive to the so-called thermal infrared spectrum (whose wavelengths are between 7 mm and 14 mm or even 20 mm) such as a Lynred ATI320 sensor.

[0041] The AS0149, AR0144 and ATI320 image sensors deliver their digital data to the processor via a differential link using the mipi protocol, sometimes referred to as the mipi "link".

[0042] It goes without saying that if the sensor (22) has a differential link communication interface using a protocol other than mipi, the communication interfaces dedicated to this type of differential link will be used on the processor (21).

[0043] Furthermore, in a preferred embodiment according to the invention and in order to secure the operation of the device, the processor (21) performs the programming of the sensor registers through a microcontroller (50), for example an STM32G071GBUX having its own non-volatile memory and its own volatile memory.

[0044] In the case of this programming of the sensor registers, the microcontroller (50) copies the programming pins, including the dedicated I2C and / or SPI pins of the processor (21) to the sensor (22) transparently.

[0045] On the other hand, in order to secure the operation of the device and to control the operation of the sensor (22) and the microcontroller (50) as expected, the processor (21) can then regularly, for example once every 10 seconds, send additional commands to the microcontroller (50), intended for the latter and not for the sensor (22), in order to test the presence and correct operation of this microcontroller by analyzing the responses provided by the microcontroller (50) to these specific requests from the processor (21).

[0046] For example, the processor (21) and the microcontroller (50) can use a process called "Rolling Code" (https: / fr.wikipedia.org / wiki / Code_tournant) when sending these test commands for presence and / or operation.

[0047] Therefore, if the microcontroller (50) malfunctions or provides incorrect or incompatible responses with the sequence allowed by the "Rolling Code" process, for example due to substitution by a third party for any reason (accidental or malicious), then the processor (21) can detect this malfunction and thus invalidate all its own results, including counting, subsequent and / or stop the transfer of digital data as long as the responses are erroneous.

[0048] In one embodiment according to the invention, the optic (220) is chosen so as to view a field of view suitable for shooting predefined people and / or objects (3) in the environment (1): for example, an M12 Sunex DSL377A-650-F2.8 optic, compatible with the component (22) OnSemi AS0149 and whose horizontal field of view is 113°.

[0049] In another particular embodiment, a sensor (42) and an optic (420) are used, for example respectively of the same nature as the sensor (22) previously described and the optic (220) previously described, placed on a separate electronic board "remote sensor board" (4), preferably of smaller dimensions than the processing unit which can advantageously be located a few meters away from the processing unit for a more discreet implementation for example.

[0050] We will also refer to the video sensor (42) as a remote sensor later on.

[0051] In this particular embodiment, the initial data and the power supply are carried on the same coaxial cable and the card (4) has suitable filtering allowing the initial data and the power supply to be concatenated, for example by a capacitive coupling known in itself.

[0052] The initial data is for example sent to the processor by means of a coaxial connection or "link" with the power supplied by the coaxial link and the data transported on a power modulation for example of type FPDLink or equivalent, or GSML or a differential link of type mipi csi, lvds, sub-lvds, slvc-ec, or a GSML link or an APIX link (Apix being a coaxial modulation) among other non-restrictive examples of embodiment and the power of the card (4) containing the sensor (42) is supplied to the remote sensor card via a single coaxial cable (29), such as a cord: Johnson SMA #415-0029-3.0 whose central part carries both these initial image data and the power supply, this principle being called phantom power supply.

[0053] In a particular embodiment, when the serial link used is of the FPDlink type, the processing unit (2) can be equipped with a so-called deserialization component such as a component manufactured by Texas Instruments DS90UB954 and the remote sensor board (4) can be equipped with an associated serialization component DS90UB953 thus creating an operational FPDLink serial link between the electronic board (4) and the computing entity.

[0054] Indeed, the initial digital image data, delivered according to the "mipi" standard by the sensor, are transformed by the serialization component DS90UB953, transported via the coaxial cable, then deserialized by the deserialization component DS90UB954 and delivered to the processor (21) again in the form of initial image data according to the mipi standard (270).

[0055] In one particular embodiment, the remote sensor board (4) is itself equipped with operating means such as a microcontroller (43), for example an STM32G071GBUX having its own non-volatile memory and its own volatile memory.

[0056] The processor (21) can communicate with the microcontroller (43) using the communication link with the help of the device (29).

[0057] Indeed, it is planned to be able to transfer the programming data of the sensor (22) through the communication link (29), for example using an FPDLink or equivalent.

[0058] In this particular embodiment, the processor (21) can then regularly, for example once every 10 seconds, send commands to the microcontroller (43) in order to test the presence and correct operation of the latter by analyzing the responses provided by the microcontroller (43) to these requests.

[0059] Therefore, if the microcontroller (43) malfunctions or if the remote sensor board (4) provides incorrect or incompatible responses with the sequence allowed by the "Rolling Code" process, for example due to substitution by a third party for any reason (accidental or malicious), then the processor (21) can detect this malfunction and thus invalidate its own results, including subsequent counting results, and / or stop the transfer of digital data as long as the responses are erroneous.

[0060] From the initial image data, the processor (21) of the computing entity (2) resizes the initial image data to create a new set called restricted data, this step according to the process being noted (91).

[0061] In a preferred embodiment, the definition of the restricted data is chosen to include 320x240 image elements or pixels, which effectively lowers the resolution considerably compared to the original image data since the resolution of the original image data is 1280x960 pixels when an OnSemi AS0149 image sensor is used, i.e. 16 times fewer image elements contained in the restricted data.

[0062] In a particular embodiment, to perform this transformation from 1280x960 to 320x240 elements, the processor (21) can locally apply subsampling; the most direct way being to use a mathematical process consisting for example of selecting a value every n values, n being an integer strictly greater than 1 for example n=4, in the horizontal dimension of the initial data and a value every p values, p being an integer strictly greater than 1, for example p=4, in the vertical dimension of the initial data.

[0063] When the processor has created and stored this restricted data in its RAM, the initial image data is permanently erased from the processor's RAM (25) and the processor can no longer access this initial image data and therefore cannot use it for any mathematical or memory operation.

[0064] In this particular embodiment, the size of the restricted image data is therefore reduced by a factor of 16 compared to the size of the initial image data; it is of course possible to choose other subsampling ratios by choosing values ​​other than a ratio of 4 and other methods to perform this operation.

[0065] As an example, when the image capture system (2) views an area of ​​4 m x 2 m of the environment (1), the head of a person present in this environment (31) (32) (33) initially represents a data set of approximately 100 x 100 pixels among the initial image data, the previous operation transforms this data set into a new set of 25 x 25 pixels and individual identification of the person is then no longer possible.

[0066] Indeed, in a document from the company Axis (https: / www.axis.com / dam / public / 53 / 47 / fd / pixel-density-en-US-364241.pdf) a minimum value of 40 x 40 pixels is defined for satisfactory identification of people from digital shots of their face.

[0067] In a system setup stage, prior to the actual counting stage, the processing unit can thus be equipped with a radio communication module to facilitate the configuration stage, for example, without limitation: Wifi, Bluetooth, LoRa, ISM (433 MHz, 868 MHz or 915 MHz or 2.4 GHz).

[0068]

[0069] In one particular embodiment, the processing unit is equipped with a Wifi communication module, for example: a TP-Link AC600 WiFi USB Key module (26) and thus send restricted image data via a USB connection between the processor (21) and this module (26) to the user of a computer, for example a laptop PC (7) with a Wifi connection (70) connection parameters with the module (26) and equipped with image viewing software (71) thus allowing the device (2) according to the invention and / or the device (4) according to the invention to be optimally positioned for this computer user (7), when used, for image capture in the medium (1) to be monitored.

[0070] In a particular embodiment according to the invention, as long as the Wifi link (70) is thus activated, the system is no longer allowed to perform any counting operation whatsoever.

[0071] In a preferred embodiment according to the invention, k counting zone positions of the middle (11) (12) are defined as follows, k being an integer greater than or equal to 1, then the processor of the processing unit records the characteristics in a set (Z) of these k counting positions (dimensions, position, shape, etc.) in its non-volatile memory (23) or, if present and available in its non-volatile memory (24) then physically disconnects the Wifi module (26) so that its use is no longer possible until a possible future new step of setting up the system in a different position and this before any counting step.

[0072] The shape of each of the counting zones is defined as a polygon with a finite number pol of points, pol being a natural number greater than 2, and all the characteristics of each polygon are defined in this set (Z).

[0073] The characteristics of a counting position of the set (Z) are denoted Z c with c a natural number ranging from 1 to k.

[0074] Each Z zone c is associated by the processor with a digital counter C c , initially zero and which is increased by one unit by the system each time a person or predefined object intercepts in the medium (1) the counting zone represented by Z c (11) (12).

[0075] In a particular embodiment, the user of the computer temporarily connected via Wifi to the counting system directly removes the USB Wifi module from the system according to the invention to reauthorize the operations necessary for counting (detection of the physical absence of the USB Wifi module on the system by the processor).

[0076] In a particular embodiment, the system itself disconnects the power supply to the Wifi module, thus rendering it completely inoperative during the operations necessary for counting, and then reauthorizes the operations necessary for counting.

[0077] The electronic neural network (240) implemented in the system for the invention is now described as the constitution of one or more layers of interconnected electronic neurons.

[0078] A neuron is associated with a transfer function, subsequently called synaptic weight, between all its inputs and its output.

[0079] A layer of neurons represents the juxtaposition of several neurons sharing the same inputs, each therefore having its own synaptic weight and its own output.

[0080] Each neuron in a layer can thus be connected to one or more neurons in one or more previous layers, the last layer being the "output layer" of the neural network (240).

[0081] In the case of the neural network implemented by the invention, this output layer delivers, from the restricted data, a finite list, which can be limited to 50 for example, of numerical quantities comprising: a set (R) of areas of interest or "rectangles" from among the restricted data characterized by an origin, a width, and a height from among the restricted data; and the probability of the presence of a predefined person or object in each of these areas of interest R. i of the set (R), i being a natural number greater than or equal to 1 when the processor's computation steps have determined at least one area of ​​interest among the restricted data.

[0082] The set (R) can, of course, include other information elements such as color information for example.

[0083] For example, the processor (21) can calculate, from the restricted image data and the origin, the width and height of each of the regions of interest R i , a colorimetric histogram of the region of interest R i , comprising h elements each representing a specific color, for example: black, blue, brown, grey, green, orange, pink, purple, red, white, yellow, h being an integer greater than or equal to 1, for example 11.

[0084] For each of the h elements representing a specific color, the processor can thus calculate the proportion of the specific color relative to the (h-1) other elements and / or calculate the centroid of the specific color in the region of interest R i .

[0085] These additional elements can then be added to the list of numerical quantities delivered by the neural network for characterization operations performed later by the processor during the counting steps.

[0086] The synaptic weights of the neural network (240) therefore characterize its operation and the optimal values ​​of these weights are determined during a step prior to the implementation of the system according to the invention, called learning.

[0087] The learning stage can be carried out using a dedicated development environment such as "TensorFlow" (available at the internet address (URL): tensorflow.org) and this learning takes place entirely outside the described device, for example using a PC, specific software for the chosen development environment and a database of reference images and annotations containing, in practice, at least 10,000 images and annotations, including photos of people or the defined object from several viewpoints, different lighting and at different shooting distances and for each of these reference images, the positions and dimensions of the people or the defined object are known and stored in the annotation database.

[0088] Databases and annotations containing fewer than 10,000 images and annotations may be used, but this can negatively impact the quality or relevance of the synaptic weights thus created.

[0089] The synaptic weights (240) are calculated during this learning and are then stored in a non-volatile memory (23) or (24) of the processing unit (2), preferably in the large-capacity non-volatile memory (24).

[0090] Indeed, with a database of reference images and annotations containing 10,000 images and annotations, the indicated learning constitutes a network containing on the order of 400,000 synaptic weights, which represents approximately 1.6 million bytes (unit of information on 8 bits) when each of these synaptic weights is represented by 32 bits of information, which is compatible with the non-volatile memories of the processing unit.

[0091] Each synaptic weight has a real, unbounded value, and these 32 bits of information can therefore represent either an integer (between -2,147,483,648 and 2,147,483,647) or a signed floating-point number (minimum absolute value approximately 1.175 x 10⁻³ -38 , maximum absolute value approximately 1.7 x 10 38 ).

[0092] Of course, if greater precision or dynamics were required for the functioning of the neural network, each of the synaptic weights could be stored with a larger number of bits of information, for example 64 bits.

[0093] This set of synaptic weights constituting the electronic neural network can then be used directly by the processor (21) to perform mathematical operations from the restricted image data and provide each new restricted data set (R) for the entire medium (1) or a restricted part of this medium (1), in this case we speak of an operation called "crop".

[0094] In a particular embodiment, when setting up the system by an operator and when the initial image data or restricted data can be viewed with a Wifi link, the power supply to the non-volatile memory (24) where all the synaptic weights necessary for counting are stored is turned off by the processor when the Wifi link is activated and the processor can then no longer access the synaptic weights present in the non-volatile memory and therefore can no longer perform the steps necessary for counting; as soon as the Wifi link is deactivated, the processor turns the power supply back on to the non-volatile memory (24) and then the processor can again perform all the counting steps.

[0095] In another particular embodiment and in order to reduce the duration of the operations performed by the processor using the neural network, the latter is read from a non-volatile memory (23) or (24) and then stored in the volatile memory (25) of the system.

[0096] In order to detect their presence and track the movements of people and / or defined objects in the environment, the processor must perform another step from the restricted image data.

[0097] For each new set of restricted image data delivered by the sensor (22) or the sensor (42) at a time t t The processor performs a comparison with previous restricted image data, delivered at an earlier time denoted t s , for each image element or pixel (motion detection, step according to the process noted (92)), s and t being natural numbers with t > s.

[0098] For each pixel P of the restricted image data at time t t , if the difference between the new restricted image data and the previous restricted data from time t s is greater than a threshold S pixel , S pixel being a real number preferably greater than or equal to 20, then the processor stores the position in the image and its value in light intensity and / or its color, in volatile memory to create a new set (B), each pixel of this new set (B) being characteristic of changes in the medium (1).

[0099] At this stage, noted (93), according to the invention, these changes can be linked to a movement of predefined people and / or objects in the environment but also to other modifications of the environment such as variations in brightness for example: the processor must therefore perform new operations to characterize more reliably the movements to be considered for counting.

[0100] From this new set of pixels denoted (B), the processor determines, according to a principle known in itself, such as "clustering" or by using a "filling by diffusion" process, the pixels connected orthogonally and / or diagonally to form new sets, denoted bs, tables of numerical data representative of the pixels connected orthogonally and / or diagonally, these sets being subsequently called "blobs".

[0101] These bs sets are, again, of smaller dimensions than the restricted image data. This step in the process is denoted (94).

[0102] When these new bs sets are created, the processor applies operations to calculate: a horizontal and a vertical size in the restricted image data; a position in the restricted image data; an estimate of the blob's displacement between the position in the restricted image data and the previous restricted image data, and thus deduce a displacement velocity, the time interval between the two sets being fixed by the frame rate delivered by the sensor and having the inverse of this frame rate; and using the previously defined electronic neural network, determine the probability that the blob is indeed representative of the movement of a person or a predefined object in the medium (1) by comparing each blob bs, whose origin, height, and width are known, to each region of interest of the set (R) whose origin,The height, width, and probability of presence of a predefined person or object are known.

[0103] Consider a blob bs k belonging to the set (B) of blobs bs, k being a natural number here between 1 and the total number of blobs in the set (B).

[0104] When the position of this blob bs k corresponds with the position of an element R i areas of interest (R) and that the probability of presence is greater than a fixed threshold S blob , S blob being a real value strictly greater than 0, less than or equal to 1, for example 0.9 then bs k is considered by the processor as representative of the presence of a predefined person or object in the environment and this set bs k is subsequently used by the processor.

[0105] When for a blob bs kbelonging to the set of blobs bs, this probability is less than the threshold set S blob then bs k is not considered representative by the processor of the presence of a predefined person or object in the environment and is therefore removed from the set bs, the processor erasing the bs data k of his fleeting memory.

[0106] After this step of the invention, the system has stored in its volatile memory (25) all the data sets (bs n ), each element of (bs n ) being representative of a predefined person or object, n being a natural number greater than or equal to 1 when at least one predefined person or object has been determined to be present in the medium (1) and n less than or equal to the number of blobs in the initial set bs.

[0107] For each new set of restricted image data, the processor compares this new set containing all the data (bs n) created at a time t r to the equivalent set (bs m ) derived, through the same steps, from the previous restricted image data created at an earlier time t q , m being an integer greater than or equal to 1 when at least one predefined person or object has been determined to be present and moving in the environment, r and q being natural numbers with r > q.

[0108] The comparison between an element of the set (bs n ) and an element of the set (bs m ), is done by comparing different characteristics of each blob such as, for example, but not limited to, its length and width dimensions, the average value of its pixels, its average position in the image and therefore that for all the characteristics compared between an element of the set (bs n ) and an element of the set (bs m), the difference is less than a specific threshold for each characteristic, for example 2 for length and width dimensions, 10 for the average pixel values ​​and 10 for the position in the image, then the processor performs the step of associating an element of (bs n ) to an element of (bs m ).

[0109] When the processor associates a blob with A of the whole of the (bs n ) of time t r with a blob bs B of the whole of the (bs m ) of time t q , then this characterizes an effective movement of the person or predefined object in the medium (1), step according to the process noted (95), A and B being natural numbers respectively less than n and m.

[0110] In a particular embodiment, when the set of regions of interest (R) also includes a colorimetric histogram comprising the h color elements; these h elements can then be used as additional criteria to perform the association between blobs in the set (bs n ) and blobs from the set (bs m ).

[0111] If during consecutive images the processor does not associate a blob from the set of (bs n ) with a blob from a set (bs m ) previous, d being a natural number greater than or equal to 1, then the processor erases from its volatile memory (25) in a step denoted (97) the set of data representing (bs m ) the oldest and it can no longer be used by the processor (21).

[0112] For each blob associated with a previous blob, the processor determines, in a step denoted (97), whether it intercepts the determined count positions of the set (Z) by comparing the positions of the blob in the restricted image data with those of each count position Z c .

[0113] When the processor calculates a correspondence between the position of the blob and that of the counting area Z c , so he increases the C counter by one unit c specific to zone Z c , representing the number of predefined people and / or objects that have reached this Z zone c This step in the process is noted (98).

[0114] In a step prior to counting, the processor assigns to all Z zones c a zero value in their counter C c .

[0115] It should be noted that the counting system according to the invention does not record any data in its non-volatile memory during counting.

[0116] Therefore, when the system's power supply is cut off, all the data present in volatile memory is completely erased after a few seconds, this duration typically corresponding to the discharge time of the various capacitors needed, according to a principle known in itself, for filtering the system's power supply (2).

[0117] After a delay denoted DY chosen by the user of the counting system, for example, 1 hour, 1 day, 1 week, the processor uses one of the transport networks (80) to which it is connected, such as an Ethernet link or an RS232 serial link or USB, for example, and which are preferably available directly on the processor pins (21), to communicate to secondary operating means (8), for example a PC, the value reached by the different counters C c of each Z counting zone c of the set (Z) and then all the counters of these counting zones (11) (12) are reset to a zero value.

[0118] It should be noted that only the values ​​of the different C counters c can be transmitted in this way and in particular, the device does not provide any operating step to transfer data restricted by these links (80) (Ethernet, USB and RS232).

[0119] In a particular embodiment, when a counter of a counting zone Z c did not reach a value S comp , called the counting threshold, during the chosen time period DY, then the counter C c is considered zero, a possible value of S comp being 5 for example, and the value 0 is then communicated by the processor to the user of the counting system as the value of counter C c of zone Z c considered during the DY period.

[0120] It should be noted that no initial image data or even restricted data is communicated to the second means of operation (8) by the processor (21) on this link (80) to which it is connected.

[0121] Indeed, link (80) in this particular case is a very low bandwidth link, less than 1Mbits / s, and is totally incompatible with sending images in a video stream.

[0122] Therefore, only the numerical values ​​reached by the counters are communicated, which is well compatible with digital links (80) using very low rates (or very low bandwidth) and the system also performs all operations locally, in a completely autonomous manner, without using remote computing resources.

[0123] In a particular embodiment, the system according to the invention is powered by a solar panel (6) whose surface area is less than 1m² 2 Indeed, with such a surface area, a solar panel provides a peak power of around 150Wp / m² 2but the measured electrical consumption of the system in operation with the processor (21), its RAM (25) and non-volatile memory (23) and / or (24), the sensor (22) and / or the sensor (42), the serialization (41) / deserialization (27) means implemented for the establishment of the FPDlink or equivalent is on the order of or less than 5W and is therefore compatible with a solar panel power supply.

[0124] In a particular embodiment according to the invention, when the solar panel is used, a charge regulator and a battery will preferably be used to store the surplus energy delivered by the solar panel compared to the electrical consumption of the device and thus also have power available even when the solar panel no longer provides energy (at night among other times): an ECO-WORTHY brand kit with a peak power of 240 W can be used in this embodiment.

[0125] It should be noted that the system's power consumption is significantly lower than that of a laptop computer, whose minimum power consumption in operation is around 50W, even before carrying out the counting steps and using a digital image sensor, and also lower than the power consumption of electronic cards such as nvidia jetson nano, widely used for image processing using neural networks, and whose power consumption in operation is around 20W.

[0126] In a particular embodiment, the system according to the invention is powered by a wind turbine (9), for example a VEVOR brand 3-blade wind turbine whose peak power is 400 W.

[0127] When the wind turbine is used, a charge regulator and a battery will preferably be used to store the surplus energy delivered by the wind turbine compared to the electrical consumption of the device and thus also have this electrical supply available even when the wind turbine is no longer providing energy (in the absence of wind among other things).

[0128] As is self-evident, and as follows from the foregoing, the invention is not limited to the particular embodiments just described; on the contrary, it encompasses all variants thereof.

Claims

Image processing device intended for people and / or object counting applications, comprising: a camera system, arranged to allow the capture of images of people and / or objects in an environment to be monitored; the camera means providing, at successive times, digital data representative of the images, called initial data; initial processing means in the form of an electronic card called a processing unit and having at least one computing unit to process this initial data and create a new set of data called counting results; remote processing means, arranged to collect and then process the counting results.and is essentially characterized by: initial processing means in the form of an electronic card called a processing unit, having at least one computing unit to perform subsampling of the initial data in order to create a restricted dataset; these initial processing means use non-volatile memory in which k different counting zones have been stored, k being an integer greater than or equal to 1, to define k different counting zones, these zones having been defined beforehand during a phase called the implementation phase; these processing means use an electronic neural network arranged to process the restricted data,In a so-called counting phase, in order to determine if defined people and / or objects move over time within the environment, and then to estimate the number of defined people and / or objects intercepting each of the k defined counting zones in the monitored environment over time, based on restricted data, these operating means transmit the counting results to remote operating means, excluding all initial and restricted data. The image acquisition system communicates regularly with the processing unit to prevent any diversion of the digital data representing the images and to interrupt their transfer in case of diversion and / or malfunction. Device according to claim 1), characterized in that the first means of operation erase the initial image data once the restricted data has been created and stored. Device according to claim 1), characterized in that the image capture system can be on the same printed circuit board as the processing unit or remote. Device according to claim 3), characterized in that the image capture system can be remote from the processing unit and connected to it by means of a coaxial cable using a carrier current and modulation. Device, according to any one of the preceding claims, characterized in that the processing unit has temporary access, and only during the implementation phase, to a particular electronic transport network in order to communicate restricted data, access to this electronic transport network becoming physically impossible during the counting phase. Device, according to claim 5), characterized in that the particular electronic means of transport uses a radio connection. Device, according to claim 5), characterized in that the particular electronic means of transport uses a wired connection. Device, according to any one of the preceding claims, characterized in that the values ​​reached by the digital counters C i ; i being the index of the counting position Z i among the k counting positions, after a delay DY are communicated by means of a transport network and then reset to a zero value after this operation. Device, according to claim 8), characterized in that, when the counter C i has not reached a threshold noted S comp after the DY delay, then the C counter i is set to 0 and the value reached is communicated by the device for the counting zone Z i is zero. Device, according to claim 8) or 9), characterized in that the means of transport used to communicate the values ​​of the meters of the counting areas is a wired link. Device, according to claim 8) or 9), characterized in that the means of transport used to communicate the values ​​of the meters of the counting areas is a radio link. Device, according to any one of the preceding claims, characterized in that its power supply is provided by a solar panel. Device, according to claim 12), characterized in that the solar panel is sized so as to be able to operate the device, but also to recover more energy in order to store it so that the device can operate, including when there is no sun. Device, according to any one of claims 1 to 11, characterized in that its power supply is provided by a wind turbine. Device, according to claim 14), characterized in that the wind turbine is sized so as to be able to operate the device, but also to recover more energy in order to store it so that the device can operate, including when there is no wind. Device, according to any one of claims 12) to 15) characterized in that an energy storage device is provided to recover the surplus energy produced by the solar panel and / or the wind turbine.

Images