Method and computing device for inferring via a machine learning algorithm a temperature in the vicinity of a person

Abstract

Method and computing device for inferring via a machine learning algorithm a temperature in the vicinity of a person. The computing device stores a predictive model of the machine learning algorithm. The computing device determines a two-dimensional (2D) thermal image comprising temperature measurements and a position of the person within the 2D thermal image. The 2D thermal image is determined based on data received from a thermal sensor located above an area where the person is located. The computing device executes the machine learning algorithm, using the predictive model for generating output(s) based on inputs. The inputs comprise the temperature measurements of the 2D thermal image and the position of the person within the 2D thermal image. The output(s) comprise the inferred temperature in the vicinity of the person, the inferred temperature being at a distance above a ground of the area where the person is located.

Description

METHOD AND COMPUTING DEVICE FOR INFERRING VIA A MACHINE LEARNING ALGORITHM A TEMPERATURE IN THE VICINITY OF A PERSONTECHNICAL FIELD

[0001] The present disclosure relates to the field of building automation. More specifically, the present disclosure presents a method and computing device for inferring via a machine learning algorithm a temperature in the vicinity of a person.BACKGROUND

[0002] Systems for controlling environmental conditions, for example in buildings, are becoming increasingly sophisticated. An environment control system may at once control heating and cooling, monitor air quality, detect hazardous conditions such as fire, carbon monoxide release, intrusion, and the like. Such environment control systems generally include at least one environment controller, which receives measured environmental values, generally from sensors, and in turn determines set-points or command parameters to be sent to controlled appliances.

[0003] One aspect of environmental conditions control is the regulation of the temperature in an area of a building (e.g. in a meeting room, an open space, a corridor, etc.). The regulation of the temperature is based on temperature measurements in the area under control. One or more temperature sensors are usually deployed in the area, for measuring the current temperature in the area. If the measured temperature is not adequate, commands are sent to an equipment (e.g. an heating, ventilating, and I or air-conditioning (HVAC) system) to adjust the temperature in the area. Examples of temperature sensors include a standalone temperature sensor, a temperature sensor included in a smart thermostat, etc.

[0004] However, the temperature is generally not uniform in an area. For instance, in the case where the area is a room of a building, the temperature is generally not the same on the floor, on the ceiling and at a given height between the floor andthe ceiling. Furthermore, the temperature is not the same in an area of the room close to a window, by contrast to an area of the room close to a heating or air conditioning source.

[0005] In the case where one or more persons are present in the area, the temperature perceived by the person(s) should be taken into consideration for regulating the temperature in the area, in order to maximize the thermal comfort of the person(s). For example, the temperature in the vicinity of a person located in the room is 19 degrees Celsius, but the temperature measured by a smart thermostat controlling the temperature in the room is 21 degrees Celsius. A regulation of the room temperature based on the temperature measurement by the smart thermostat may erroneously determine that 21 degrees Celsius is an adequate temperature for the room, while the person is not comfortable with a temperature of 19 degrees Celsius in his vicinity.

[0006] Therefore, there is a need for a new method and computing device for inferring via a machine learning algorithm a temperature in the vicinity of a person.SUMMARY

[0007] According to a first aspect, the present disclosure relates to a method for inferring via a machine learning algorithm a temperature in the vicinity of a person. The method comprises storing a predictive model of the machine learning algorithm in a memory of a computing device. The method comprises determining, by a processing unit of the computing device, a two-dimensional (2D) thermal image comprising temperature measurements. The 2D thermal image is determined based on data received from a thermal sensor located above an area where the person is located. The method comprises determining, by the processing unit of the computing device, a position of the person within the 2D thermal image. The method comprises executing, by the processing unit of the computing, device the machine learning algorithm using the predictive model for generating at least one output based on inputs. The inputs comprise thetemperature measurements of the 2D thermal image and the position of the person within the 2D thermal image. The at least one output comprises the inferred temperature in the vicinity of the person, the inferred temperature being at a distance above a ground of the area where the person is located.

[0008] According to a second aspect, the present disclosure relates to a non-transitory computer readable medium comprising instructions executable by a processing unit of a computing device. The execution of the instructions by the processing unit of the computing device provides for inferring via a machine learning algorithm a temperature in the vicinity of a person by implementing the aforementioned method.

[0009] According to a third aspect, the present disclosure relates to a computing device. The computing device comprises at least one communication interface, memory for storing a predictive model of the machine learning algorithm, and a processing unit. The processing unit determines a two-dimensional (2D) thermal image comprising temperature measurements. The 2D thermal image is determined based on data received from a thermal sensor located above an area where the person is located. The processing unit determines a position of the person within the 2D thermal image. The processing unit executes the machine learning algorithm using the predictive model for generating at least one output based on inputs. The inputs comprise the temperature measurements of the 2D thermal image and the position of the person within the 2D thermal image. The at least one output comprises the inferred temperature in the vicinity of the person, the inferred temperature being at a distance above a ground of the area where the person is located.

[0010] In a particular aspect, the machine learning algorithm implements a neural network, and the predictive model comprises weights of the neural network.

[0011] In another particular aspect, the thermal sensor is an infrared (IR) camera.

[0012] In still another particular aspect, the 2D thermal image is a 2D array oftemperature measurements.

[0013] In yet another particular aspect, the position of the person within the 2D thermal image is defined by two sets of 2D coordinates.

[0014] In a particular aspect, the area is located in a building, the ground being a floor of the area, the thermal sensor being positioned on a ceiling above the area.

[0015] In another particular aspect, the inputs further comprise at least one of the following: a humidity level measurement in the area, a carbon dioxide level measurement in the area, an external temperature measurement outside the area, an external humidity level measurement outside the area, and a period of time.

[0016] In still another particular aspect, one or more commands for controlling an appliance are generated based at least on the inferred temperature. The one or more commands are transmitted to the controlled appliance. In a particular embodiment, the controlled appliance is an heating, ventilating, and I or air-conditioning (HVAC) system.

[0017] In yet another particular aspect, determining the 2D thermal image comprises receiving the 2D thermal image from the thermal sensor via the communication interface of the computing device.BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Embodiments of the disclosure will be described by way of example only with reference to the accompanying drawings, in which:

[0019] Figure 1 represents an environment control system comprising a computing device and a thermal sensor;

[0020] Figures 2A and 2B represent a person within an area where the thermal sensor of Figure 1 is deployed;

[0021] Figures 3 and 4 represent a thermal image of the area illustrated in Figures 2A and 2B;

[0022] Figure 5 represents a method for inferring via a machine learning algorithm a temperature in the vicinity of a person;

[0023] Figure 6 is a schematic representation of the machine learning algorithm executed by the computing device of Figure 1 according to the method of Figure 5; and

[0024] Figure 7 illustrate an exemplary implementation of the machine learning algorithm of Figure 6 by a neural network.DETAILED DESCRIPTION

[0025] The foregoing and other features will become more apparent upon reading of the following non-restrictive description of illustrative embodiments thereof, given by way of example only with reference to the accompanying drawings.

[0026] Various aspects of the present disclosure generally address one or more of the problems related to accurate and reliable determination of the temperature at a precise position within an area (for instance in the context of environment control systems for buildings). More particularly, the present disclosure aims at determining the temperature in the vicinity of a person (e.g. a person located in a room of a building) by processing a thermal image (generated via a thermal sensor positioned above an area where the person is located) by means of a machine learning algorithm.

[0027] The following terminology is used throughout the present specification:

[0028] Environment: condition(s) (temperature, pressure, oxygen level, light level, security, etc.) prevailing in a controlled zone or place, such as for example in a building.

[0029] Environment control system: a set of components which collaborate for monitoring and controlling an environment.

[0030] Environmental data: any data (e.g. information, commands) related to an environment that may be exchanged between components ofan environment control system.

[0031] Environment control device (ECD): generic name for a component of an environment control system. An ECD may consist of an environment controller, a sensor, a controlled appliance, etc.

[0032] Environment controller: device capable of receiving information related to an environment and sending commands based on such information.

[0033] Environmental characteristic: measurable, quantifiable or verifiable property of an environment (a building). The environmental characteristic comprises any of the following: temperature, pressure, humidity, lighting, CO2, flow, radiation, water level, speed, sound; a variation of at least one of the following, temperature, pressure, humidity and lighting, CO2 levels, flows, radiations, water levels, speed, sound levels, etc., and / or a combination thereof.

[0034] Environmental characteristic value: numerical, qualitative or verifiable representation of an environmental characteristic.

[0035] Sensor: device that detects an environmental characteristic and provides a numerical, quantitative or verifiable representation thereof. The numerical, quantitative or verifiable representation may be sent to an environment controller.

[0036] Controlled appliance: device that receives a command and executes the command. The command may be received from an environment controller.

[0037] Environmental state: a current condition of an environment based on an environmental characteristic, each environmental state may comprise a range of values or verifiable representation for the corresponding environmental characteristic.

[0038] Zone of a building: the expression ‘zone of a building’ is used throughout the present specification to refer to the interior of a whole building or a portion of the interior of the building such as, without limitation: a story, a room, an aisle, etc.

[0039] Area: the expression area is used throughout the present specification to refer to an entire zone of a building, a portion of a zone of a building, an outdoor place, etc.

[0040] Reference is now made concurrently to Figures 1 and 3, where Figure 1 represents an environment control system comprising a computing device 100 is illustrated. For example, the computing device 100 is an environment controller 100 exchanging data with other environment control devices (ECDs). The environment controller 100 is responsible for controlling the environment of a zone of a building. The environment controller 100 receives from sensors (e.g. from a thermal sensor 200) environmental characteristic values measured by the sensors. The environment controller 100 generates commands based on the received environmental characteristic values. The generated commands are transmitted to controlled appliances (not represented in the Figure 1 for simplification purposes), to control operations of the controlled appliances.

[0041] The zone under the control of the computing device 100 is not represented in Figure 1 for simplification purposes. As mentioned previously, the zone may consist of a room, a story, an aisle, etc. However, any type of zone located inside any type of building is considered being within the scope of the present disclosure.

[0042] Details of the computing device 100, the thermal sensor 200 and a training server 300 will now be provided.

[0043] As illustrated in Figure 1 , the computing device 100 comprises a processing unit 110, memory 120, and a communication interface 130. The computing device 100 may comprise additional components, such as another communicationinterface 130, a user interface 140, a display 150, etc.

[0044] The processing unit 110 comprises one or more processors (not represented in Figure 1 ) capable of executing instructions of a computer program. Each processor may further comprise one or several cores. The processing unit 110 executes a machine learning algorithm 112 and a control module 114, as will be detailed later in the description.

[0045] The memory 120 stores instructions of computer program(s) executed by the processing unit 110 (for implementing the machine learning algorithm 112, the control module 114, etc.), data generated by the execution of the computer program(s), data received via the communication interface 130 (or another communication interface), etc. Only a single memory 120 is represented in Figure 1 , but the environment controller 100 may comprise several types of memories, including volatile memory (such as a volatile Random Access Memory (RAM), etc.) and non-volatile memory (such as a hard drive, electrically-erasable programmable read-only memory (EEPROM), flash, etc.).

[0046] The communication interface 130 allows the computing device 100 to exchange data with remote devices (e.g. with the thermal sensor, controlled appliance(s) not represented in Figure 1 , the training server 300, etc.) over a communication network (not represented in Figure 1 for simplification purposes). For example, the communication network is a wired communication network, such as an Ethernet network; and the communication interface 130 is adapted to support communication protocols used to exchange data over the Ethernet network. Other types of wired communication networks may also be supported by the communication interface 130. In another example, the communication network is a wireless communication network, such as a Wi-Fi network; and the communication interface 130 is adapted to support communication protocols used to exchange data over the Wi-Fi network. Other types of wireless communication network may also be supported by the communication interface 130, such as a wireless mesh network, Bluetooth®, Bluetooth® Low Energy (BLE), etc. In stillanother example, the environment controller 100 comprises two communication interfaces 130. The environment controller 100 communicates with the thermal sensor 200 and controlled appliance(s) via a first communication interface 130 (e.g. a Wi-Fi interface); and communicates with other devices (e.g. the training server 300) via a second communication interface 130 (e.g. an Ethernet interface). Each communication interface 130 usually comprises a combination of hardware and software executed by the hardware, for implementing the communication functionalities of the communication interface 130.

[0047] Details of the thermal sensor 200 are not represented in Figure 1 for simplification purposes, since thermal sensors are well known in the art. The thermal sensor 200 generally comprises a processing unit, memory, a communication interface and a thermal sensing component. An example of thermal sensor 200 consists of an infrared (IR) camera comprising an IR sensing component. However, other types of thermal sensors may be used in place of an IR camera.

[0048] In the case of an IR camera, the IR sensing component of the IR camera generates raw sensor data. The processing unit of the IR camera executes a conversion algorithm to convert the raw sensor data into corresponding temperature measurements. The communication interface of the IR camera transmits sensor data to the computing device 100, the sensor data comprising the temperature measurements.

[0049] The present disclosure addresses thermal sensors 200 generating a two-dimensional (2D) thermal image comprising temperatures measurements (e.g. a 2D array of temperature measurements), which is transmitted to the computing device 100. Alternatively, the thermal sensor 200 generates raw sensor data, which are transmitted to the computing device 100, and further processed by the computing device 100 to generate the 2D thermal image of temperature measurements.

[0050] For example, as illustrated in Figure 3, the thermal image 500 is a 2D arrayof temperature measurements, comprising M columns and N rows, as follows:[Ti T2... TMTI,2T22■■■ TM 2TI,N T2IN ■ ■ ■ TM N]

[0051] Column i (with i comprised between 1 and M) and row j (with j comprised between 1 and N) comprises temperature measurement Tjj.

[0052] The data structure representative of the thermal image comprising temperature measurements is not limited to the aforementioned 2D array of temperature measurements. Other data structures may be used for representing the thermal image.

[0053] Details of the training server 300 are not represented in Figure 1 for simplification purposes. The training server 300 generally comprises a processing unit, memory and a communication interface. The training server 300 may comprise additional components, such as a user interface, a display, another communication interface, etc.

[0054] The role of the training server 300 is to generate a predictive model used by the machine learning algorithm 112. The predictive model is generated by the processing unit of the training server 300 and transmitted to the computing device 100 via the communication interface of the training server 300.

[0055] Reference is now made to Figures 1, 2A and 2B, where Figures 2A and 2B respectively represent front and top views of an area 400 where a person 430 is located.

[0056] The area 400 is generally located inside a building, although the present disclosure is also applicable to an outdoor area. For example, the area 400 comprises at least a portion of a room, a corridor, an open-space, etc.

[0057] The thermal sensor 200 is located above the area 400 where the person430 is located. The person 430 is at least partially inside a field of view 201 of the thermal sensor 200.

[0058] Figure 2A illustrates an area 400 having a floor 420 where the person is standing. A ceiling 410 located above the area 400 is also represented in Figure 2A. The thermal sensor 200 is affixed to the ceiling 410.

[0059] Although the person 430 is represented as standing in Figure 2A, the person 430 may be in a different position (e.g. sitting on a chair, etc.). Furthermore, although a single person 430 is represented in Figure 2A, more than one person may be located in the area 400 and at least partially inside the field of view 201 of the thermal sensor 200. Additionally, one or more objects (e.g. furniture) may be located in the area 400 and at least partially inside the field of view 201 of the thermal sensor 200. For example, the person 430 may be sitting on a chair and working at a desk, the chair and desk being located in the area 400 and at least partially inside the field of view 201 of the thermal sensor 200.

[0060] In the rest of the description and for generalization purposes, we will refer to the floor 420 of the area 400 as the ground 420 of the area 400. The person 430 and optionally other objects (e.g. furniture) present in the area 400 are standing I laying above the ground 420 of the area 400.

[0061] The thermal sensor 200 measures temperatures on the ground 420 of the area 400 within the field of view 201. However, when a person (e.g. 430) and I or an object is in the field of view 201 , the thermal sensor 200 measures temperatures on the person (e.g. 430) and I or the object.

[0062] The temperature measurements performed by the thermal sensor 200 are used to infer a temperature in the vicinity of the person 430 and at a distance above the ground 420. The inferred temperature is determined via the machine learning algorithm 112, as will be detailed later in the description.

[0063] The distance above the ground 431 (illustrated in Figure 2A) may vary and is defined during a training phase of the machine learning algorithm 112. Forexample, the machine learning algorithm 112 is trained to determine the inferred temperature at one meter above the ground. Similarly, the vicinity 432 (illustrated in Figure 2B) of the person 430 may vary and is defined during the training phase of the machine learning algorithm 112. For example, the machine learning algorithm 112 is trained to determine the inferred temperature at a horizontal distance of one meter from the person 430 (the vicinity 432 of the person 430 is defined by an elliptical shape centered on the person 430 as illustrated in Figure 2B).

[0064] Reference is now made to Figures 2A, 2B, 3 and 4. As mentioned previously, Figure 3 illustrates the thermal image 500 determined based on measurements by the thermal sensor 200. Figure 4 represents a position 510 of the person 430 in the thermal image 500.

[0065] A thermal image is usually represented with pixels, each pixel having a color representative of the corresponding temperature measurement. However, for illustration purposes, the thermal image 500 is represented in Figures 3 and 4 has a set of squares instead of a set of pixels, each square representing an element of the 2D array of temperature measurements. The horizontal axis and the vertical axis in Figures 3 and 4 respectively correspond to the columns and rows of the 2D array of temperature measurements. For example, the square corresponding to the temperature measurement Tjj is at position i on the horizontal axis and position j on the vertical axis. The 2D array of temperature measurements comprising M columns and N rows, the representation of the thermal image 500 extends on the horizontal axis from position 1 to M and extends on the vertical axis from position 1 to N.

[0066] The implementation illustrated in Figures 3 and 4 is based on a reference system where the horizontal and vertical axes respectively extend from 1 to M and 1 to N. A person skilled in the art would readily adapt this implementation to another reference system, for example a reference system where the horizontal and vertical axes respectively extend from 0 to (M-1 ) and 0 to (N-1 ). In any of these reference systems.

[0067] The determination of the position 510 of the person 430 in the thermal image 500 is out of the scope of the present disclosure. Different types of algorithms (known by a person skilled in the art) may be used for this purpose, including a machine learning algorithm complementary to the machine learning algorithm 112 used for determining the inferred temperature.

[0068] The data used for identifying the position 510 of the person 430 may vary from one implementation to another. Following is an exemplary implementation. The position 510 of the person 430 in the thermal image 500 is approximated by a rectangular shape in the thermal image. The rectangular shape is identified by two sets of 2D coordinates. Figure 4 illustrates the rectangular shape corresponding to the position 510 of the person 430 in the thermal image 500 being identified by the 2D coordinates of an upper left square 511 (representative of a first pixel of the thermal image 500) and the 2D coordinates of a lower right square 512 (representative of a second pixel of the thermal image 500). For instance, the thermal image 500 has a resolution of 160 by 120 (M=160 and 1^=120). The position 510 of the person 430 is identified by coordinates (40,80) and (50,110). Alternatively, the position 510 of the person 430 can also be identified by coordinates (40,110) and (50,80), corresponding to respective lower left square and upper right square (not represented in Figure 4).

[0069] If the thermal image 500 comprises several persons 430, the position 510 of each person 430 is determined. Using the determined position of a given person, a corresponding inferred temperature can be determined in the vicinity of the given person, by means of a method described in the following paragraphs.

[0070] Reference is now made concurrently to Figures 1, 2A, 2B, 3, 4 and 5, where Figure 5 represents a method 600 using the machine learning algorithm 112 to inter the temperature in the vicinity 432 of the person 430. At least some of the steps of the method 600 are implemented by the computing device 100. The present disclosure is not limited to the method 600 being implemented by the computing device 100 represented in Figure 1 , but is applicable to any type ofcomputing device capable of implementing the steps of the method 600.

[0071] A dedicated computer program has instructions for implementing at least some of the steps of the method 600. The instructions are comprised in a non-transitory computer readable medium (e.g. the memory 120) of the computing device 100. The instructions provide for using the machine learning algorithm 112 to inter the temperature in the vicinity 432 of the person 430, when executed by the processing unit 110 of the computing device 100. The instructions are deliverable to the computing device 100 via an electronically-readable media such as a storage media (e.g. CD-ROM, USB key, etc.), or via communication links (e.g. via a communication network through the communication interface 130).

[0072] The instructions of the dedicated computer program executed by the processing unit 110 implement the machine learning algorithm 112 and the control module 114. The machine learning algorithm 112 (e.g. a neural network) is adapted for inferring output(s) based on inputs using the predictive model (generated by the training server 300), as is well known in the art. The control module 114 provides functionalities allowing the computing device 100 to interact with and control other devices (e.g. the thermal sensor 200, the training server 300, a controlled appliance 700 only represented in Figure 5, etc.).

[0073] The method 600 comprises the step 605 of generating the predictive model. Step 605 is performed by the processing unit of the training server 300. This step will be further detailed later in the description.

[0074] The method 600 comprises the step 610 of transmitting the predictive model generated at step 605 to the computing device 100, via the communication interface of the training server 300. Step 610 is performed by the processing unit of the training server 300.

[0075] The method 600 comprises the step 615 of receiving the predictive model from the training server 300, via the communication interface 130 of the computing device 100. Step 615 is performed by the processing unit 110 of the computingdevice 100.

[0076] The method 600 comprises the step 620 of storing the predictive model in the memory 120 of the computing device 100. Step 620 is performed by the processing unit 110 of the computing device 100.

[0077] The method 600 comprises the step 625 of determining the two-dimensional (2D) thermal image 500 comprising temperature measurements. The 2D thermal image 500 is determined based on data received from the thermal sensor 200 (located above the area 400 where the person 430 is located). Step 625 is performed by the processing unit 110 of the computing device 100.

[0078] In a first exemplary implementation, the thermal sensor 200 is capable of directly generating the 2D thermal image 500. In this case, the 2D thermal image 500 is simply received from the thermal sensor 200 via the communication interface 130 at step 625. In a second exemplary implementation, the 2D thermal image 500 is generated by the processing unit 110 based on the data received from the thermal sensor 200 via the communication interface 130 at step 625.

[0079] A digital representation of the 2D thermal image 500 may vary, based on a particular implementation of the method 600. In a previously mentioned implementation, the 2D thermal image 500 is in the form of a 2D array of temperature measurements. In another implementation, the data structure representative of the 2D thermal image 500 is not in the form of a 2D array of temperature measurements. For example, the data structure is a vector comprising M * N entries (which may be converted into a 2D array of temperature measurements by the processing unit 110).

[0080] The method 600 comprises the step 630 of determining the position 510 of the person 430 within the 2D thermal image 500. Step 630 is performed by the processing unit 110 of the computing device 100.

[0081] The method 600 comprises the step 635 of executing the machine learning algorithm 112. The machine learning algorithm 112 uses the predictive model(stored at step 620) for generating one or more outputs based on inputs. The execution of the machine learning algorithm 112 is performed by the processing unit 110 of the computing device 100. This step will be further detailed later in the description.

[0082] The inputs comprise the temperature measurements of the 2D thermal image 500 and the position 510 of the person 430 within the 2D thermal image 500.

[0083] Additional input(s) may be used. For example, at least one of the following additional inputs is used: a humidity level measurement in the area 400, a carbon dioxide level measurement in the area 400, an external temperature measurement outside the area 400, an external humidity level measurement outside the area 400, and a period of time, etc.

[0084] The one or more outputs comprise the inferred temperature in the vicinity 432 of the person 430. The inferred temperature is at a distance (e.g. one meter) above the ground 420 of the area 400 where the person 430 is located.

[0085] The method 600 comprises the optional step 640 of generating one or more commands for controlling one or more controlled appliances 700, based at least on the inferred temperature (generated at step 635). Step 640 is performed by the processing unit 110 of the computing device 100.

[0086] The method 600 comprises the optional step 650 of transmitting the one or more commands (generated at step 630) to the one or more controlled appliances 700, via the communication interface 130. Step 640 is performed by the processing unit 110 of the computing device 100.

[0087] A single controlled appliance 700 is illustrated in Figure 5 for simplification purposes. An exemplary controlled appliance 700 is an HVAC system. Examples of commands include a command for controlling a speed of a fan, a command for controlling a pressure generated by a compressor, a command for controlling a rate of an airflow through a valve, etc.

[0088] At least some of the steps 615, 620, 625, 630, 640 and 645 of the method 600 are performed by the control module 114 executed by the processing unit 110.

[0089] Referring now concurrently to Figures 1, 5 and 6, a schematic representation of the machine learning algorithm 112 is illustrated in Figure 6, representing the inputs and the outputs used by the machine learning algorithm 112 when performing step 635 of the method 600.

[0090] Reference is now concurrently to Figures 1, 2A, 2B, 4, 5, 6 and 7, where Figure 7 represents a neural network 800 implemented by the machine learning algorithm 112. The neural network 800 illustrated in Figure 7 is for illustration purposes only. A person skilled in the art will readily understand that other implementations of the neural network 800 may be used for performing step 635 of the method 600.

[0091] The neural network 800 includes an input layer for receiving the temperature measurements of the 2D thermal image 500 and the position 510 of the person 430 within the thermal image 500, followed by a flattening layer, followed by a plurality of fully connected layers. The last layer among the plurality of fully connected layers is an output layer for outputting the inferred temperature in the vicinity 432 of the person 430.

[0092] In general, neural networks used in the processing of images have a first layer adapted to receive array(s) of dimensions 2 or greater, representing the images. The flattening layer is adapted for converting the data of the image into a 1 -dimensional array (also referred to as a vector) for inputting it to the next layer.

[0093] In the previously mentioned exemplary implementation, the digital representation of the 2D thermal image 500 consists of a 2D array of temperature measurements (having M columns and N rows), as follows:[Ti T2... TM-iTI,2T22■■■ TM 2TI,N T2,N ■ ■ ■ TM N]

[0094] The flattening layer comprises M * N neurons for receiving the temperature measurements, each neuron receiving one temperature measurement among the M * N temperature measurements of the 2D array.

[0095] In the previously mentioned exemplary implementation, the position 510 of the person 430 within the thermal image 500 is defined by the 2D coordinates (e.g. X! and Y of the upper left square 511 and the 2D coordinates (e.g. X2and Y2) of the lower right square 512. The flattening layer further comprises four neurons for receiving the coordinates X^ YtX2and Y2.

[0096] The flattening layer can be considered as an input layer with respect to the following fully connected layers. The operations of the fully connected layers are well known in the art. The number of fully connected layers is an integer greater than 2, including the output layer (Figure 7 represents four fully connected layers, including the output layer, for illustration purposes only). The number of neurons in each fully connected layer may vary. During the training phase of the neural network, the number of fully connected layers and the number of neurons for each fully connected layer are selected, and may be adapted experimentally. The output layer comprises one neuron for outputting the inferred temperature in the vicinity 432 of the person 430.

[0097] In an alternative exemplary implementation (not represented in the Figures), the input layer receives a digital representation of the thermal image 200 consisting of a (one dimension) vector comprising the M * N temperature measurements of the thermal image 200. In this case, the neural network 800 does not include the flattening layer represented in Figure 7. The input layer comprises M * N (for the temperature measurements) + 4 (for the coordinates X^ Y X2and Y2of the person) neurons connected to the first layer of the fully connected layers.

[0098] In another alternative exemplary implementation (not represented in theFigures), the neural network 800 includes the input layer for receiving the thermal image 500, followed by a convolutional layer, optionally followed by a pooling layer, followed by the flattening layer, followed by a plurality of fully connect layers. The convolutional layer and the optional flattening layer are only applied to the thermal image 500. The coordinatesX2and Y2are directly transferred by the input layer to the flattening layer.

[0099] A convolutional layer is well known in the art. The convolutional layer applies a convolution to the 2D thermal image represented by the 2D array of temperature measurements having M columns and N rows, to generate a corresponding convoluted 2D array. The 2D convolutional layer is defined by the following parameters: a two-dimensions filter and a stride. The convoluted 2D array comprise M’ columns and N’ rows, with M greater or equal than M’ and N greater or equal than N’.[00100JA pooling layer is also well known in the art. If the convolutional layer is followed by a pooling layer, the output of the convolutional layer being the convoluted 2D array, the pooling layer generates a corresponding pooled 2D array. The pooling layer is defined by the following parameters: a two-dimensions filter, a stride and a pooling algorithm. The convoluted 2D array comprising M’ columns and N’ rows, the pooled 2D array comprise M” columns and N” rows, with M’ greater than M” and N’ greater than N”. The role of the pooling layer is to reduce the size of the 2D array generated by the convolutional layer.

[0101] The convolutional layer (or the pooling layer if it is present) is followed by the flattening layer, which operates in a manner which has been previously described in relation to Figure 7. Each value of the 2D array generated by the convolutional layer (or by the pooling layer if it is present) is allocated to a dedicated neuron of the flattening layer.

[0102] Instead of a single convolutional layer (and a corresponding optional pooling layer), several consecutive convolutional layers (and corresponding optional pooling layers) may be included in the neural network 800, as is well known in theart.TRAINING PROCEDURE

[0103] Following is a description of a procedure for training a neural network (e.g. the one illustrated in Figure 7) to generate the inferred temperature in the vicinity 432 of the person 430. The training procedure is implemented by the training server 300. The training procedure can be adapted by a person skilled in the art to other types of machine learning algorithms 112. The training procedure is adapted to an implementation of the neural network supporting step 635 of the method 600 represented in Figure 5.

[0104] The processing unit of the training server 300 executes a neural network training engine (not represented in the Figures). The neural network training engine implements functionalities of a neural network, allowing to generate a predictive model ready to be used by the neural network 800 (implemented by the computing device 100) at the end of the training, as is well known in the art.[00i05]The training procedure comprises a step of initializing a predictive model used by the neural network implemented by the neural network training engine. The predictive model comprises various parameters which depend on the characteristics of the neural network implemented by the neural network training engine. The predictive model is stored in the memory of the training server 300.[00i06]The initialization of the predictive model comprises defining a number of layers of the neural network, a functionality for each layer (e.g. convolutional layer, flattening layer, fully connected layer, etc.), initial values of parameters used for implementing the functionality of each layer, etc. The initialization of the predictive model is performed by a person highly skilled in the art of neural networks.

[0107] For example, the initialization of the parameters of a fully connected layer includes determining the number of neurons of the fully connected layer and determining an initial value for the weights of each neuron. Different algorithms (well documented in the art) can be used for allocating an initial value to theweights of each neuron. For example, each weight is allocated a random value within a given interval (e.g. a real number between -0.5 and +0.5), which can be adjusted if the random value is too close to a minimum value (e.g. -0.5) or too close to a maximum value (e.g. +0.5). In another example, the initialization of the parameters of a convolutional layer includes determining the size and values of the filter, and the value of the stride.[OO1O8]A comprehensive description of the initialization of the predictive model is out of the scope of the present disclosure, since it is well known in the art of neural networks and is different for each type of layer (e.g. a fully connected layer, a flattening layer, a convolutional layer, a pooling layer, etc.).

[0109] The training procedure comprises an initial step of generating training data. The training data comprise a plurality of 2D thermal images 500 generated by the thermal sensor 200 and a corresponding plurality of positions 510 of the person 430 within the 2D thermal images 500. For each 2D thermal image 500 and corresponding position 510, a training temperature is measured in the vicinity 432 of the person 430 at the distance 431 above the ground 420 of the area 400 where the person 430 is located. The training temperature measured during the training procedure corresponds to the inferred temperature generated at step 635 of the method 600.[OOiiojThe training procedure comprises a step (I) of executing the neural network implemented by the neural network training engine, using the predictive model to generate output(s) based on inputs.[OOiiijThe inputs comprise a given 2D thermal image 500 and a corresponding given position 510 of the person 430 within the given 2D thermal image 500 (selected among the training data). The output(s) comprise an inferred temperature in the vicinity 432 of the person 430 at the distance 431 above the ground 420 of the area 400 where the person 430 is located. As mentioned previously, a corresponding training temperature has been measured for this given 2D thermal image 500 and corresponding given position 510 during the generation of thetraining data.[00ii2]The neural network implemented by the neural network training engine corresponds to the neural network (e.g. neural network 800 illustrated in Figure 7) executed by the computing device 100 at step 635 of the method 600 (same number of layers, same functionality for each layer, same parameters used for implementing the functionality of each layer, etc.).[00ii3]The training procedure comprises a step (II) of adjusting the predictive model of the neural network implemented by the neural network training engine, to minimize a difference between the inferred temperature and the measured training temperature.[00ii4]As is well known in the art, this step aims at minimizing a difference between outputs calculated by the neural network and expected outputs of the neural network, through the adjustment of the predictive model of the neural network (more specifically by adjusting parameters of the neural network comprised in the predictive model). For example, for a fully connected layer of the neural network, the adjustment comprises adjusting the weights associated to the neurons of the fully connected layer. In another example, for a convolutional layer of the neural network, the adjustment comprises adjusting the values of the filter used by the convolutional layer.

[0115] Various algorithms may be used for minimizing the difference between the calculated (inferred) outputs and the expected outputs. For example, the predictive model is adjusted so that a difference between the inferred temperature and the measured training temperature is lower than a threshold (e.g. a difference of only 1 % is tolerated).[00ii6]As is well known in the art, steps (I) and (II) are repeated for several instances of a given 2D thermal image 500 and a corresponding given position 510 selected among the training data. The training data need to be large enough to properly train the neural network.

[0117] The aforementioned steps of the training procedure correspond to step 605 of the method 600 represented in Figure 5. At the end of the training procedure, the neural network is considered to be properly trained (the predictive model of the neural network has been adjusted so that a difference between the expected outputs and the calculated outputs has been sufficiently minimized). The predictive model, comprising the adjusted parameters of the neural network, is transmitted to the computing device 100, as illustrated by step 610 of the method 600 represented in Figure 5. Test data are optionally used to validate the accuracy of the predictive model. The test data are different from the training data used for the training procedure.

[0118] Various techniques well known in the art of neural networks can be used for performing step (II). For example, the adjustment of the predictive model of the neural network at step (II) uses back propagation. Other techniques, such as the usage of bias in addition to the weights (bias and weights are generally collectively referred to as weights in the neural network terminology), reinforcement training, etc., may also be used.[00ii9]The training procedure may be implemented in two steps for a deployment at different customer premises. A generic predictive model is generated in a controlled training environment according to the aforementioned training procedure. At each specific customer premises, the generic predictive model is deployed, and a specific predictive model adapted to the characteristics of the specific customer premises is generated, by refining the generic predictive model. Ideally, the neural network automatically adjusts the generic predictive model and converges towards the adapted specific predictive model.

[0120] Fine-tuning of the generic predictive model into the specific predictive model can be achieved through reinforcement learning. As is well known in the art, a reinforcement signal is used for adapting the predictive model when reinforcement learning is implemented. The value of the reinforcement signal is one of positive reinforcement (also referred to as a positive reward) or negative reinforcement(also referred to as a negative reward).[00i2i]Although the present disclosure has been described hereinabove by way of non-restrictive, illustrative embodiments thereof, these embodiments may be modified at will within the scope of the appended claims without departing from the spirit and nature of the present disclosure.

Claims

WHAT IS CLAIMED IS:

1. A method for inferring via a machine learning algorithm a temperature in the vicinity of a person, the method comprising: storing a predictive model of the machine learning algorithm in a memory of a computing device; determining by a processing unit of the computing device a two-dimensional (2D) thermal image comprising temperature measurements, the 2D thermal image being determined based on data received from a thermal sensor located above an area where the person is located; determining by the processing unit of the computing device a position of the person within the 2D thermal image; and executing by the processing unit of the computing device the machine learning algorithm using the predictive model for generating at least one output based on inputs, the inputs comprising the temperature measurements of the 2D thermal image and the position of the person within the 2D thermal image, the at least one output comprising the inferred temperature in the vicinity of the person, the inferred temperature being at a distance above a ground of the area where the person is located.

2. The method of claim 1 , wherein the machine learning algorithm implements a neural network, and the predictive model comprises weights of the neural network.

3. The method of claim 1 , wherein the thermal sensor is an infrared (IR) camera.

4. The method of claim 1 , wherein the 2D thermal image is a 2D array of temperature measurements.

5. The method of claim 4, wherein the position of the person within the 2D thermal image is defined by two sets of 2D coordinates.

6. The method of claim 1 , wherein the area is located in a building, the ground being a floor of the area, the thermal sensor being positioned on a ceiling above the area.

7. The method of claim 1 , wherein the inputs further comprise at least one of the following: a humidity level measurement in the area, a carbon dioxide level measurement in the area, an external temperature measurement outside the area, an external humidity level measurement outside the area, and a period of time.

8. The method of claim 1 , further comprising generating one or more commands for controlling an appliance based at least on the inferred temperature; and transmitting the one or more commands to the controlled appliance.

9. The method of claim 8, wherein the controlled appliance is an heating, ventilating, and I or air-conditioning (HVAC) system.

10. A non-transitory computer readable medium comprising instructions executable by a processing unit of a computing device, the execution of the instructions by the processing unit of the computing device providing for inferring via a machine learning algorithm a temperature in the vicinity of a person by: storing a predictive model of the machine learning algorithm in a memory of a computing device; determining by the processing unit of the computing device a two-dimensional (2D) thermal image comprising temperature measurements, the 2D thermal image being determined based on datareceived from a thermal sensor located above an area where the person is located; determining by the processing unit of the computing device a position of the person within the 2D thermal image; and executing by the processing unit of the computing device the machine learning algorithm using the predictive model for generating at least one output based on inputs, the inputs comprising the temperature measurements of the 2D thermal image and the position of the person within the 2D thermal image, the at least one output comprising the inferred temperature in the vicinity of the person, the inferred temperature being at a distance above a ground of the area where the person is located.

11. A computing device comprising: at least one communication interface; memory for storing a predictive model of the machine learning algorithm; and a processing unit for: determining a two-dimensional (2D) thermal image comprising temperature measurements, the 2D thermal image being determined based on data received from a thermal sensor located above an area where the person is located; determining a position of the person within the 2D thermal image; and executing the machine learning algorithm using the predictive model for generating at least one output based on inputs, the inputs comprising the temperature measurements of the 2D thermal image and the position of the person within the 2D thermal image, the at least one output comprising the inferred temperature in the vicinity ofthe person, the inferred temperature being at a distance above a ground of the area where the person is located.

12. The computing device of claim 11 , wherein the machine learning algorithm implements a neural network, and the predictive model comprises weights of the neural network.

13. The computing device of claim 11 , wherein the thermal sensor is an infrared (IR) camera.

14. The computing device of claim 11 , wherein determining the two-dimensional (2D) thermal image comprises receiving the two-dimensional (2D) thermal image from the thermal sensor via the at least one communication interface.

15. The computing device of claim 11 , wherein the 2D thermal image is a 2D array of temperature measurements.

16. The computing device of claim 15, wherein the position of the person within the 2D thermal image is defined by two sets of 2D coordinates.

17. The computing device of claim 11 , wherein the area is located in a building, the ground being a floor of the area, the thermal sensor being positioned on a ceiling above the area.

18. The computing device of claim 11 , wherein the inputs further comprise at least one of the following: a humidity level measurement in the area, a carbon dioxide level measurement in the area, an external temperature measurement outside the area, an external humidity level measurement outside the area, and a period of time.

19. The computing device of claim 11 , wherein the processing unit further generates one or more commands for controlling an appliance based at least on the inferred temperature; and the processing unit transmits the oneor more commands to the controlled appliance via the at least one communication interface.

20. The computing device of claim 19, wherein the controlled appliance is an heating, ventilating, and / or air-conditioning (HVAC) system.

Images