Localized triggering of human activity analysis

Abstract

The present description relates to determining human activity in an environment (ENV) in which at least one sensor is configured to measure a parameter relating to human activity. This process comprises: - providing a device for detecting a human presence in the environment (GW, TV, BB1, PRN, BB2), and - activating the sensor (C11, C12, C13) if a human presence has been detected in the environment (RO1).

Description

Localized triggering of human activity analysis Technical field

[0001] This description relates to the field of sensors, in particular for the purpose of measuring human activities. Prior art

[0002] Identifying and analyzing human activities in connected environments uses information about the ambient context of the environment, and can therefore exploit various sensors, such as: - those measuring environmental information (temperature, brightness, CO2 emissions, etc.), - those measuring door, window, cupboard or other openings, - those measuring movement (for example infrared sensors), - those measuring images more generally such as cameras, - those measuring sounds more generally such as microphones, etc.

[0003] Very often, sensors of different types can be installed together to increase the relevance of the information captured in an environment and in particular the precision of possible identification of human activities.

[0004] These data sources are most often localized: they can only record events occurring within their "field of vision". Thus, to be able to identify human activities in a large environment, potentially consisting of several rooms in a home for example, it is generally necessary to multiply the data sources, to cover the environment as much as possible.

[0005] In order to identify human activities in the environment, it is necessary to continuously analyze the data returned by the sensors, using specific algorithms. In particular, the sensors must return data for each change measured in the environment that can be linked to a change in activity.

[0006] These constraints of measurement, analysis, and multiplicity of sensors, imply significant consumption of electricity, radio frequency band, and computational resources. Often, the sensors in these installations operate on batteries, and methods are sought to save their consumption. Summary

[0007] This description improves the situation.

[0008] To this end, it proposes a method for activating a determination of human activity in an environment in which at least one sensor is configured to measure at least one parameter relating to a human activity, the method comprising: - Implement a device for detecting human presence in the environment, and - Activate the sensor if human presence has been detected in the environment.

[0009] Such an implementation makes it possible to save the power supply energy of the sensor, as well as the energy for processing data from the sensor if no human presence is detected for at least a certain time.

[0010] Here, the term "environment" means a spatial environment, such as a building (a home, a professional building or other), or even an augmented reality room, or other.

[0011] A human activity that can be determined is, for example, a person's current occupation in the environment (focused on a work task, or following entertainment on a screen, or others). For example, a sensor can measure eye movements of a user, and this measurement data is transmitted to a human activity recognition device, capable of interpreting the eye movements as characterizing the viewing of entertainment or, on the contrary, the reading of a work document, for example.

[0012] Thus, the implementation of the above method not only saves the energy to be supplied to the sensor, but it also saves the computing resources necessary for the operation of this human activity recognition device, as long as no human presence is detected in the environment.

[0013] Thus, in one embodiment, the sensor (at least) can be deactivated in the event of non-detection of human presence after a time delay.

[0014] In this way, the power supply to the sensor(s) can be stopped if no human presence is detected for a certain chosen time (a few minutes for example).

[0015] In one embodiment, the human presence detection device may be configured to locate, in the environment, the detected human presence.

[0016] Such an embodiment makes it possible to activate, for example, one or more sensors which are in a particular area of ​​the environment where a human presence has been detected and located, while one or more sensors in other areas of the environment can remain inactive.

[0017] Thus, in an embodiment where the environment comprises a plurality of monitoring zones, with at least one sensor assigned to each zone and configured to measure at least one parameter relating to human activity in its zone, the method may comprise: - Implement the human presence detection device in each area of ​​the environment, and - Selectively activate at least one sensor assigned to an area, if human presence has been detected in that area.

[0018] This saves power and data processing energy from other environmental sensors, if no human presence is detected in other areas of the environment.

[0019] In an embodiment where a set of a plurality of sensors is assigned to at least one area of ​​the environment, the method may comprise activating the sensors of said set in the event of detection of human presence in said at least one area, for a measurement of human activity in said at least one area.

[0020] Thus in this embodiment, the activation of the sensors of the area in which a human presence has been detected concerns all the sensors which have been assigned to this area, in order to typically obtain a relevant analysis of the human activity in this area.

[0021] For example, all the data that these sensors can collect can be used to precisely determine human activity taking place in this area.

[0022] In such an embodiment, the method may then comprise a transmission of the data measured by said at least one sensor to a human activity recognition device.

[0023] For example, the human activity recognition device may be configured to operate by machine learning that is a function of human presence detection locations (in English “place-based” type machine learning).

[0024] In one embodiment, the human presence detection device may be configured to emit / receive radio frequency radiation into the environment.

[0025] For example, in such an embodiment, a disturbance of this radiofrequency radiation can be determined to deduce a human presence at least partially obstructing this radiation.

[0026] Thus, in such an embodiment, the human presence detection device can be configured to measure an attenuation of a radiofrequency signal between at least one transmitter and at least one receiver, and deduce from the measured attenuation a temporary presence of an obstacle between the transmitter and the receiver, the temporary presence being assimilated to a human presence in the environment.

[0027] For example, in such an embodiment, the human presence detection device may be configured to emit / receive Wi-Fi type radiofrequency radiation.

[0028] Such a technique, called "Wifi Sensing", is an optimal choice to activate sensors that are relevant to measure at least one parameter related to human activity, without the need to use conventional presence sensors which are expensive. Wifi Sensing can be implemented with pre-existing hardware such as a gateway and fixed objects, connected to the gateway via a Wifi link, and therefore without the need for other sensors. It offers the possibility of detecting human presence and locating this presence with sufficient precision for the needs of monitoring human activity.

[0029] Alternatives to such an embodiment are possible. For example, according to another embodiment, the device for detecting the presence of a person may comprise one or more motion sensors, or the two embodiments may be combined. Thus, for example, three embodiments for the human presence detection device may be envisaged: - WiFi sensing type detection, or - detection using motion sensors that detect human presence, or - a combination of these two achievements.

[0030] In an embodiment where said at least one transmitter and at least one receiver comprise a gateway of a local network and a plurality of objects connected to the gateway, this plurality of connected objects can be chosen to locate a human presence detected by measuring the attenuation of the radiofrequency signal between the gateway and each connected object.

[0031] For example, trilateration involving the gateway and several connected objects in the environment can help locate the detected human presence in a given area. In this case, the sensor(s) assigned to this area can be activated.

[0032] In such an implementation, each connected object and the gateway can be assigned fixed positions in the environment.

[0033] Thus, the connected objects chosen to carry out detection and localization can be assigned to fixed, chosen positions, as is typically the case with a connected television set, or a connected printer, or even connected lamps for example.

[0034] The present description also relates to a device for activating a determination of human activity in an environment in which at least one sensor is configured to measure at least one parameter relating to a human activity, the activation device being: - connected to the sensor and to a human presence detection device, and - configured to implement the process presented above.

[0035] Thus typically, such an activation device may be configured to activate the sensor if a human presence has been detected in the environment by the human presence detection device.

[0036] The present description also relates to a device for detecting human presence in an environment in which at least one sensor is configured to measure at least one parameter relating to human activity, for the implementation of the above method.

[0037] The human presence detection device and the activation device described above may be part of a single device or may be separate devices.

[0038] Such a human presence detection device may be, for example, a gateway of a local radiofrequency network (for example, a Wi-Fi network), or possibly another terminal of a local network (via a radiofrequency link other than Wi-Fi, for example, such as Bluetooth®), such as, for example, a mobile terminal of the smartphone type of a user, running a computer application for implementing the above method.

[0039] This description also relates to a system comprising at least: - a sensor configured to measure at least one parameter relating to human activity, - a device for detecting human presence in an environment in which the sensor is located, and - an activation device, connected to the sensor and to the human presence detection device, and configured to implement the method presented above.

[0040] This activation device may be integrated into the human presence detection device (for example in the form of a module programmed into a gateway capable of detecting human presence and locating it), or may be separate (for example integrated into a server, a gateway or a terminal on the local network or other).

[0041] The system may further comprise a human activity recognition device, connected to the activation device for implementing human activity recognition when the sensor(s) are activated.

[0042] This human activity recognition device can be integrated into the aforementioned system (for a local human activity analysis solution) or be separate (for remote analysis based on data sent by the sensor(s)).

[0043] According to another aspect, there is provided a computer program comprising instructions for implementing all or part of a method as defined herein when this program is executed by a processing circuit (for example the processing circuit of the aforementioned activation device). According to another aspect, there is provided a non-transitory recording medium, readable by a processing circuit, on which such a program is recorded. Brief description of the drawings

[0044] Other features, details and advantages will become apparent upon reading the detailed description below, and upon analyzing the attached drawings, in which: Fig. 1

[0045] Figure 1 shows an example of a system for implementing the above method, according to one embodiment. Fig. 2

[0046] Figure 2 shows an example of steps of a method as defined above, according to one embodiment. Fig. 3

[0047] Figure 3 illustrates a definition of the zones RO1, RO2 of a given environment ENV, and the sensors which are assigned to each of these zones, according to an exemplary implementation. Fig. 4

[0048] Figure 4 shows elements of a device for detecting and locating a human presence, these elements being able to be such as a router (for example a GW gateway of a local network) connected to terminals (for example connected objects such as a connected TV set, or even a connected PRN printer), according to an exemplary embodiment. Fig. 5

[0049] Figure 5 illustrates an awakening of sensors (in white) for a given area (in which a human presence has been detected), while the sensors of the other areas of the environment remain deactivated (in black). Fig. 6

[0050] Figure 6 shows an example of a processing circuit of an activation device of a system of the type defined above, according to one embodiment. Description of the embodiments

[0051] Figure 1 shows a human presence detection device based on radiofrequency radiation in the form, in this embodiment, of a gateway GW of a local area network LAN, to which are connected connected objects such as one or more connected lamps BB1, BB2, etc., for example arranged in respective rooms RO1, RO2, etc. of an environment ENV, as well as a connected television set TV in one room, a connected printer PRN in another room. As presented further below with reference to Figures 3 and following, radiofrequency radiation between the gateway GW and each connected object (TV, BB1 in the example illustrated in Figure 1) can be disturbed by the body of a user UT when located near these connected objects, in fact absorbing part of the radiation linking the gateway GW to these connected objects TV, BB1. The disturbance of each of these radiations can be measured to detect a human presence UT in this room, and these disturbances can be compared with each other to locate this presence. In this case, in the example illustrated in Figure 1, the radiofrequency radiation is more disturbed for the connected objects in room RO1 (which are the television set TV and the lamp BB1) than for those in room RO2: it can typically be deduced that the human presence UT was detected in room RO1, and not in room RO2.

[0052] In this case, only sensors C11, C12, C13, etc. of room RO1 can be activated while sensors C21, C22, C23, etc. of room RO2 can be deactivated (or remain deactivated if applicable).

[0053] A radio frequency range suitable for carrying out such detection is that of Wifi (a few dozen meters) and this detection technique by disturbing Wifi radiation is then called “Wifi Sensing”.

[0054] So-called "Wifi Sensing" techniques aim to detect human presence, or even gestures, by analyzing disturbances induced by the human body on the propagation of Wifi waves, typically between a router and a terminal (typically between a gateway of a local network and a connected object in the local network). The terminal can be any type of connected object and preferably fixed, for example, in a predefined area. In practice, it can be a television or a printer, connected, or even connected light bulbs, for example, in usually fixed positions.

[0055] The main advantage of Wifi Sensing is that it exploits the transmission of Wifi waves, penetrating all materials in connected environments (therefore without significant additional installation or consumption costs). This technique also allows detection with a wide field of vision (being able to pass through walls and other obstacles within a reasonable radio frequency range of a few dozen meters). It thus makes it possible to detect the presence of people in an environment such as a home or a professional premises and in particular to observe movements, in such an environment, of these people, for example.

[0056] Furthermore, it is possible to roughly locate human presence, particularly when several connected objects are placed in the environment.

[0057] However, this technology is far from being sophisticated enough to be used to identify any specific human activity. Simply detecting a presence, or identifying a person's posture or movement, is not informative enough to identify complex activities of that person, which depend more on a more general context.

[0058] A solution is then proposed here aimed at exploiting the information from Wifi Sensing to optimally activate one or more sensors in the environment, in order to improve the detection of ongoing human activities, while controlling energy consumption. reasonable use of these sensors, as well as computational consumption of human activity analysis processes in the observed environment.

[0059] The proposed solution implements a method for triggering the analysis of human activity locally from one or more locally activated sensors, following the detection of a human presence, for example by Wifi Sensing. We refer to Figure 2 illustrating as an example the steps of such a method.

[0060] In a first step S1 of the method, the environment is decomposed into a list of distinct zones (typically rooms separated by walls). In the next step S2, the sensors of the environment are associated with each zone, according to their position and detection field (typically, a sensor installed in a room is associated with this room for example). This decomposition is fixed by a user such as an installer, or by any other method of automatic decomposition of an environment by zones.

[0061] This situation is illustrated in Figure 3, which shows a definition of the zones of a given environment (here rooms delimited by partitions such as walls).

[0062] In another step of the method S3, a human presence detection device, of the Wifi Sensing device type, which may typically be composed of a router such as a GW gateway (circle in Figure 4) and terminals which may be connected objects (triangles in Figure 4), is configured so that it can cover each area RO1, RO2, etc. adequately, in terms of human presence detection. The detection coverage between the router and a terminal is generally an ellipse whose foci are the router and the terminal.

[0063] The following steps of the process can then be as follows: - In the nominal state, the environmental sensors are “at rest” at step S4 and configured to measure and transmit no information, if no human presence is detected; - The detection of a human presence at step S5 in one of the zones of the environment, via Wifi Sensing, triggers the awakening of the sensors (C11, C12, C13) assigned to this zone (RO1). At step S6, all the sensors of the zone concerned RO1 are then “awakened” (in white in Figure 5): a wake-up order is transmitted to each of them via the communication network to which they are connected (for example the local LAN network), this network being used among other things for the uploading of measurements. The sensors associated with the other zones remain at rest (in black in Figure 5); - The awakened sensors send back their measurements at step S7 according to their nominal operating mode and their capacities. This information passes over the communication network to which they are connected, for example via the GW gateway and to a SER server for processing the data sent back by the sensors and connected to the GW gateway by a wide area network WAN. Thus, an activation device, for example in the form of a processing circuit connected to the sensors and which may comprise for example the GW gateway, transmits the measurements from the sensors to a human activity recognition device, such as the SER server, as soon as a presence has been detected in an area. All the sensors associated with this area are awakened. The SER activity recognition device then has access to the data sent back by the sensors of the area and can use them to identify in step S8 a particular human activity in the awakened area, for example to transmit its inferences to other systems which exploit them; - When a human presence is no longer detected in the zone at step S9, after a delay of a few minutes for example, a rest order is transmitted to each of the sensors in the zone at step S10.

[0064] In one embodiment, the GW gateway comprises the aforementioned detection device and activation device. Thus, the GW gateway is configured to detect human presence and to activate the sensors in the area when human presence is detected.

[0065] An example of human activity recognition implemented in step S8 may be, but is not limited to, a “Place-based” type machine learning approach, as described in the following article: “Human activity recognition using place-based decision fusion in smart homes. In International and interdisciplinary conference on modeling and using context”, J. Cumin, G. Lefebvre, F. Ramparany, JL Crowley.

[0066] The above process works in the same way if several presences are detected in different areas simultaneously: the sensors of each area concerned are awakened and the information is transmitted, as well as the data measured by the respective sensors, to the human activity recognition device.

[0067] This sensor wake-up process can be instantiated: - locally on one of the local network devices in the environment, such as the GW gateway or on another local network device such as a connected computer or a user terminal connected to the network for example, - or on a remote machine such as the SER server communicating with the sensors of the local network of the environment, for example via the GW gateway, as illustrated in the example in figure 1.

[0068] Thus, in one embodiment, the human activity recognition can be performed locally on one or more devices in the network and in another embodiment, the activity recognition is performed on a remote machine SER in the "cloud".

[0069] In either of these embodiments, the activation or deactivation of the sensors can be carried out locally (at a unit connected to the local network) or remotely (at a unit connected to the SER server for example). Figure 6 illustrates by way of example the first embodiment in which an activation device, here in the form of a processing circuit (for example integrated into the GW gateway), can comprise: - A first INT1 interface, for RF radiofrequency communication (for example via Wifi connection) with various connected objects in the environment, - A second interface INT2, for controlling sensors C11, C12, C13, etc., C21, C22, C23, etc., arranged in the environment, on the one hand to activate or deactivate at least one set of appropriate sensors (C11, C12, C13) in the event of detection of human presence or absence in an area (R01) to which this set of sensors is assigned, and on the other hand to receive the data from these sensors and transmit them for example to a remote processing server SER via a wide area network WAN to which a third communication interface INT3 is connected, - A PROC processor capable of controlling the communication interfaces INT1, INT2, INT3, and - A MEM memory storing at least instructions of a computer program and accessible by the PROC processor to implement the above method when the PROC processor of the processing circuit executes the program instructions.

[0070] The subject of this description may find applications in the recognition of human activities in connected environments having radiofrequency detection means (for example Wifi), for example in “intelligent home” or “smart home” installations (Connected Home, Protected Home, etc.), or in intelligent tertiary buildings (such as connected professional premises or even medicalized homes or hospitals), or even in augmented reality spaces.

Claims

Claims 1. Method for activating a determination of human activity in an environment in which at least one sensor is configured to measure at least one parameter relating to a human activity, the method comprising: - Implement (S5) a device for detecting human presence in the environment, and - Activate (S6) the sensor if a human presence has been detected in the environment.

2. Method according to claim 1, in which the sensor is deactivated in the event of non-detection of a human presence after a time delay (S9).

3. Method according to one of the preceding claims, in which the human presence detection device is configured to locate, in the environment, the detected human presence.

4. Method according to claim 3, in which the environment comprises a plurality of monitoring zones (RO1, RO2), with at least one sensor (BB1, BB2) assigned to each zone and configured to measure at least one parameter relating to human activity in its zone, the method comprising: - Implement (S3) the human presence detection device in each area of ​​the environment, and - Selectively activate at least one sensor assigned to an area, if human presence has been detected in that area.

5. Method according to claim 4, in which a set of a plurality of sensors (C11, C12, C13; C21, C22, C23) is assigned to at least one zone (RO1; RO2) of the environment, the method comprising an activation of the sensors of said set in the event of detection of a human presence in said at least one zone, for a measurement of human activity in said at least one zone.

6. Method according to one of the preceding claims, comprising a transmission (S7, S8) of the data measured by said at least one sensor to a human activity recognition device.

7. Method according to one of the preceding claims, in which the human presence detection device is configured to emit / receive radiofrequency radiation into the environment.

8. Method according to claim 7, in which the human presence detection device is configured to measure a radiofrequency signal attenuation between at least one transmitter and at least one receiver, and to deduce from the measured attenuation a temporary presence of an obstacle between the transmitter (GW) and the receiver (TV, BB1), the temporary presence being assimilated to a human presence (UT) in the environment (ENV).

9. Method according to one of claims 7 and 8, in which the human presence detection device is configured to emit / receive Wi-Fi type radiofrequency radiation.

10. Method according to one of claims 8 and 9, wherein, said at least one transmitter and at least one receiver comprising a gateway (GW) of a local area network (LAN) and a plurality of objects connected to the gateway, said plurality of connected objects is chosen to locate a human presence detected by measuring the attenuation of the radiofrequency signal between the gateway and each connected object.

11. Method according to claim 10, in which each connected object (BB1, BB2, TV, PRN) and the gateway (GW) are assigned fixed positions in the environment.

12. Device for activating a determination of human activity in an environment in which at least one sensor is configured to measure at least one parameter relating to a human activity, the activation device being connected to the sensor and to the human presence detection device, and being configured to implement the method according to one of claims 1 to 11.

13. Device for detecting human presence in an environment in which at least one sensor is configured to measure at least one parameter relating to human activity, for implementing the method according to one of claims 1 to 11.

14. System comprising at least: - a sensor (C11, C12, C13, C21, C22, C23) configured to measure at least one parameter relating to a human activity, - a device for detecting human presence in an environment (GW, TV, BB1, PRN, BB2) in which the sensor is located, and - an activation device (PROC, MEM, INT1, INT2, INT3) according to claim 12.

15. Computer program comprising instructions for implementing the method according to one of claims 1 to 11, when said program is executed by a processor.

Images