A method for improving RF sensing based control

Abstract

A method (600) for controlling a device (200) in a control system (100); wherein the control system (100) comprising a controller (150), the device (200) and a plurality of nodes (300, 300', 300''); the method (600) comprising the controller (150) - controlling the device (200) with a control mode determined by radio frequency, RF, sensing performed by at least two nodes (300', 300'') out of the plurality of nodes (300, 300', 300''); - applying an adjustment to the control mode when detecting a first node (300') out of the at least two nodes (300', 300'') unreachable by the controller (150) and / or a second node (300, 300'') out of the plurality of nodes (300, 300', 300''); wherein the adjustment comprises: extending an operation duration of the device (200); and / or changing an operation configuration of the device (200).

Description

[0001] 2024PF80319

[0002] 1

[0003] A METHOD FOR IMPROVING RF SENSING BASED CONTROL

[0004] FIELD OF THE INVENTION

[0005] The invention relates to the field of a control system based on radio frequency sensing. More particularly, various methods, apparatus, and systems are disclosed herein related to improving reliability of a RF sensing based control system.

[0006] BACKGROUND OF THE INVENTION

[0007] Radio frequency (RF) based sensing is a technique widely used for motion detection, which involves one or more transmitters arranged for transmitting RF signals on a regular basis, such as according to a certain transmission frequency, and one or more receivers for receiving the RF signals, such as upon knowing the corresponding transmission frequency. These RF signals, which may also be used for data communication, when passing through a sensing volume, are affected by presence / movement of a person within the sensing volume. By monitoring such variations of RF signals, it is possible to infer presence or motion of the person. RF based sensing may also be extended to other applications such as location detection, fall detection, gesture detection, or vital signs detection, which are also based on monitoring how RF signals are affected in the sensing volume.

[0008] RF sensing based motion detection has been used in various applications. For example, in a smart home automation scenario RF sensing can be used to detect motion in a room and trigger the lights, fans, and other appliances to turn on or off automatically. In a retail analysis scenario, RF sensing can be used to track the movement of customers in a store and analyse shopping patterns to improve sales and marketing strategies.

[0009] One of the key benefits of using RF sensing for automated control is its ability to detect human presence without the need for physical contact or line of sight. This makes it ideal for applications where hands-free operation is desired, such as in smart homes, hospitals, or industrial settings. Passive RF sensing is also a non-intrusive technology, which means that it does not require any special clothing or equipment to be worn by the user. This makes it a convenient and comfortable option for users. In addition, RF sensing based automation systems are generally cost-effective, easy to install and maintain, and require little power to operate. Overall, RF sensing based automation offers an efficient solution for 2024PF80319

[0010] 2 many different applications, making it a popular choice for both commercial and consumer use.

[0011] US20140341109A1 relates to a method for managing gateway communication.

[0012] SUMMARY OF THE INVENTION

[0013] RF sensing based presence or motion detection has been commonly used for automatic control in smart home or smart building environments. By detecting presence or motion, the system can trigger various actions, such as turning lights on or off, adjusting heating or cooling, or even unlocking doors. However, there are some potential unreliability issues associated with these systems. One common issue is interference from other RF devices, communication channel fading, and / or environmental changes. Another potential issue is the limited range of the RF signal, which may not be able to detect people in certain areas of a room. To mitigate these issues, it is important to carefully design the system and regularly test and maintain it to ensure its proper functioning.

[0014] It is thus recognized by the inventors that it is beneficial to adjust the RF sensing based control mode upon detecting that one or more nodes used for RF sensing are unreachable, and to make such detection as quickly as possible to minimize any negative impact to user experience.

[0015] More particularly, the goal of this invention is achieved by a method as claimed in claim 1, by a controller as claimed in claim 11, by a system as claimed in claim 14, and by a computer program as claimed in claim 15.

[0016] In accordance with a first aspect of the invention a method is provided. A method for controlling a device based on presence or motion detection in a control system; wherein the control system comprising a controller, the device and a plurality of nodes configured to perform RF sensing for presence or motion detection of individuals or objects in smart home or smart building environments; the method comprising the controller: controlling the device with a control mode determined by radio frequency, RF, sensing performed by at least two nodes out of the plurality of nodes; applying an adjustment to the control mode when detecting a first node out of the at least two nodes unreachable by the controller and / or a second node out of the plurality of nodes; wherein the adjustment comprises: extending an operation duration of the device; and / or 2024PF80319

[0017] 3 changing an operation configuration of the device.

[0018] As one implementation, the control system may comprise a controller, a device to be controlled, and at least two nodes to set up a RF sensing pair for presence or motion detection. Of course, the division of roles is based on functionality, and physically some roles can be combined in one physical entity. For example, a controller may also be configured as a node and participate in RF sensing, and a node out of the plurality of nodes may be connected to or integrated in the device to be controlled.

[0019] In practice, a control system such as a smart home or building automation system, a smart lighting control system may comprise a plurality of nodes deployed in the space and a plurality of devices to be controlled. Presence or motion detection-based control focuses on identifying the presence of individuals or objects in a given area for turning on lights, adjusting temperature, or triggering other automated actions. Each of the plurality of nodes may be co-located with a device to be controlled, such as connected to or integrated in the device. Alternatively, some or all of the nodes may be deployed as a standalone sensing device. And the controller is the central spot to collect all the sensing information and to determine the corresponding control mode for each device respectively.

[0020] One way to perform RF sensing based presence detection is to observe the variations of received signal strength between two RF nodes in the sensing or detection area. Received Signal Strength Indicator (RSSI) is a measure of the power level of the received signal in wireless communication. Alternatively, RF sensing may also be based on Channel State Information (CSI), which describes how a signal propagates from a transmitter to a receiver with a combined effect of scattering, fading, and attenuation with distance. There are several potential causes for RSSI and CSI fluctuations in practice. For example, RSSI and CSI will fluctuate when there are obstructions on the signal propagation path. The physical obstructions may be walls, doors, furniture, people, or another object that obstruct the wireless signal, leading to attenuation and fluctuations in RSSI and CSI. The fluctuations may also be caused by reflection and multipath effects. Since a human body also absorbs or reflects wireless signals, RF sensing based presence detection is designed to recognise such changes caused by human presence.

[0021] In a communication or control network, when a node cannot be reached or is unable to establish a connection with another node or network, it is commonly referred to as "unreachable". This term means that the node is not responding to the signals or messages sent to it. Alternative terms used in the field can be "out of range", "disconnected", "unavailable", or "off the network". 2024PF80319

[0022] 4

[0023] Upon detecting that the first node is unreachable, to prevent the situation that the RF sensing also fails to detect motion and / or presence in the field, the system takes precaution to apply an adjustment to the RF sensing based default control mode. By extending an operation duration of the device and / or changing an operation configuration of the device, it allows time for the first node to recover (e.g., when the unreachable status resulted from occasional interference) and avoids the situation that the control system appears to be completely unresponsive to a user behavior.

[0024] The change on the operation configuration may be used to reduce the power consumption of the device, such that the device may be adjusted to operate in a low-power mode during the extended operation time. For example, when the device is a TV, the changes can be turning off certain advanced features, and to adjust the volume, brightness, contrast, and colour settings for power saving. When the device is a lamp, it may be tuned to a lower or lowest dimming level for power saving, instead of completely turning off after a hold period without detecting any motion in the sensing area.

[0025] Beneficially, the detection of the first node unreachable is determined by at least one of missing a message or a response from the first node by the second node; missing a response from the first node to a polling message sent by the controller; detecting a degraded communication link with the first node by the controller and / or the second node.

[0026] In a communication or control system, polling is widely used to regulate the communication between a controller, a bridge, or an access point and a plurality of nodes or devices in the network. Polling may also be used by a controller, a bridge, or an access point to check the status of a plurality of nodes or devices in the network, such as for network diagnostics. Missing one or more response from the first node to a polling message may dive the controller a clear indication that the first node may confront some malfunction or unreliable connections.

[0027] The first node may be configured to send messages regularly to another node to perform RF sensing, which messages may be sent with a relatively short time interval to detect any change of the presence in the sensing area timely. Missing one or more messages by the other node of the sensing pair may keep the other node aware that the first node is unreachable. 2024PF80319

[0028] 5

[0029] In addition to RF sensing, the first node may also participate other communication activities with one or more nodes out of the plurality of nodes. Missing a message or a response from the first node may also be an indication to the one or more nodes about the unreachable status of the first node.

[0030] In one example, the method further comprises at least one of sending a further polling message from the controller to the first node; increasing a transmission frequency of sending polling messages from the controller to the first node.

[0031] Since polling messages are typically sent with a fixed time interval to check the status of all the nodes in the system, it is beneficial to adjust the frequency of sending polling messages to the first node, such that the controller can detect any changes in the status of the first node more quickly (such as when it becomes reachable again).

[0032] In one example, the first node is attached to, connected to, or integrated in the device.

[0033] It may also be the case that the other node out of the at least two nodes, which set up the RF sensing pair, is attached to, connected to, or integrated in the device. In a further option, a further node, located close to the at least two nodes, is attached to, connected to, or integrated in the device. It is beneficial that the control system comprises a plurality of devices to be controlled and each node out of the plurality of nodes is attached to, connected to, or integrated in a device in the control system, such that the cost for deploying the plurality of nodes is minimized. Alternatively, it is also possible that the plurality of nodes is deployed independently from the one or more devices to be controlled.

[0034] Preferably, the device is one of a lighting device, a TV, audio equipment, a fan, an air conditioner, or another home appliance.

[0035] Beneficially, the method further comprises: providing an unreachable status of the first node to a remote server or a user device.

[0036] The user device may be a smartphone, a tablet, s mart watch, or a laptop.

[0037] In one example, the control system comprises a further device located near the device, and the method further comprises: extending an operation duration of the further device; and / or changing an operation configuration of the further device.

[0038] In the case of the first node attached to or integrated in the device, the device may rely on the first node for communicating with the controller instead of having another 2024PF80319

[0039] 6 communication means. When the first node is unreachable, the adjustment to change the control mode of the device may not be delivered and then it may be especially helpful to adjust the control of the further device located near the device. This solution will help compensate for the negative impact that an unresponsive device may have on the user. For example, when a light turns off after a hold period due to failure to detect the presence of a user, it would be helpful if another light next to it remained on or turned on.

[0040] In one example, the method further comprises: restoring the control mode for controlling the device when detecting the first node reachable again.

[0041] Beneficially, the detection of the first node reachable again is determined by at least one of receiving a message or a response from the first node by the second node; receiving a response from the first node to a polling message sent by the controller; detecting an improved communication link with the first node by the controller and / or the second node.

[0042] In one example, the method further comprises: selecting another node, different from the first node, out of the plurality of node to perform RF sensing for controlling the device.

[0043] It may happen that after a long-time interval that the first node is not reachable again. And then, it is beneficial to select another node to take over the RF sensing function from the first node. The other node may be selected based on its relative location in the sensing area and / or its connectivity to the other nodes.

[0044] In accordance with a second aspect of the invention a controller is provided. A controller for controlling a device based on presence or motion detection in a control system, the control system comprising the controller, the device and a plurality of nodes configured to perform RF sensing for presence or motion detection of individuals or objects in smart home or smart building environments; the controller comprising: a communication unit configured to communicate with the device and the plurality of nodes; and a processor configured to:

[0045] • control the device with a control mode determined by radio frequency, RF, sensing performed by at least two nodes out of the plurality of nodes; 2024PF80319

[0046] 7

[0047] • apply an adjustment to the control mode when detecting a first node out of the at least two nodes unreachable by one or more other nodes out of the plurality of nodes; wherein the adjustment comprises:

[0048] • to extend an operation duration of the device; and / or

[0049] • to change an operation configuration of the device.

[0050] A controller may be a bridge, a gateway, or a central controller of a smart home or smart building system.

[0051] The communication unit of the controller may be configured to support more than one communication protocols, such as to communicate with the device and the plurality of nodes via a same network or via different communication protocols.

[0052] A device is an electronic device, which may be a lighting device, a wall switch, a speaker, a television, a thermostat, a power outlet plug, a smart home assistant device, and / or a smart home appliance.

[0053] A node may be a radio capable to carry out RF sensing, such as to transmit and / or receive RF signals.

[0054] Preferably, the detection of the first node unreachable is determined by at least one of missing a message or a response from the first node by the second node; missing a response from the first node to a polling message sent by the controller; detecting a degraded communication link with the first node by the controller and / or the second node.

[0055] Advantageously, the communication unit is further configured to: send a further polling message to the first node; and / or increase a transmission frequency of sending polling messages to the first node.

[0056] The controller may be further configured to provide an unreachable status of the first node to a remote server or a user device.

[0057] Beneficially, the controller is connected to the device and the plurality of nodes wirelessly, such as according to a wireless communication standard. The wireless communication standard may be related to a standard of Wi-Fi, Zigbee, Thread, Matter, or Bluetooth Low Energy. 2024PF80319

[0058] 8

[0059] In one example, the RF sensing among the plurality of nodes may use InterP AN messages according to a Zigbee standard, Bluetooth beacons according to a Bluetooth Low Energy standard, or other peer to peer direct communication messages.

[0060] For RF sensing purpose, preferably the messages may be transmitted by a first node out of a sensing pair on a regular basis, to detect the presence or motion in the sensing area timely. Missing one or more messages, the other one or more nodes may consider the first node is unreachable.

[0061] In accordance with a third aspect of the invention a control system is provided. A control system comprising a controller according to the present invention, a device, and a plurality of nodes configured to perform radio frequency, RF, sensing, wherein the controller is configured to control the device with a control mode determined by RF sensing performed by at least two nodes out of the plurality of nodes.

[0062] The control system may be a smart lighting control system, a smart home or building automation system, which may comprise a plurality of nodes deployed in the space and a plurality of devices to be controlled. Presence or motion detection-based control focuses on identifying the presence of individuals or objects in a given area for turning on a light / TV / ait conditioner, adjusting temperature, or triggering other automated actions. Each of the plurality of nodes may be co-located with a device to be controlled, such as connected to or integrated in the device. Alternatively, some or all of the nodes may be deployed as a standalone sensing device. And the controller is the central spot to collect all the sensing information and to determine the corresponding control mode for each device respectively.

[0063] BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In the drawings, like reference characters generally refer to the same parts throughout the different figures. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.

[0065] Fig. 1 illustrates an example implementation of a control system according to the present invention;

[0066] Fig. 2 shows an example of the control system according to the present invention;

[0067] Fig. 3 shows another example of the control system according to the present invention;

[0068] Fig. 4 shows an example of implementing the present invention in a lighting control system; and 2024PF80319

[0069] 9

[0070] Fig. 5 shows a flow chart of a method according to the present invention.

[0071] DETAILED DESCRIPTION OF EMBODIMENTS

[0072] The embodiments set forth below represent information to enable those skilled in the art to practice the embodiments. Upon reading the following description in light of the accompanying drawings, those skilled in the art will understand the concepts of the disclosure and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure.

[0073] Fig. 1 illustrates an example implementation of a control system 100 according to the present invention. The control system 100 comprises a controller 150, at least a device 200 to be controlled, and a plurality of nodes 300, 300’, 300”.

[0074] The device 200 may be one of a lighting device, a TV, audio equipment, a fan, an air conditioner, or another home appliance.

[0075] The plurality of nodes 300, 300’, 300” are configured to perform radio frequency (RF) sensing, such as to detect motions or presence of one or more persons in the sensing area.

[0076] The first node 300’ may be attached to, connected to, or integrated in the device 200. Alternatively, the first node 300’ may be standalone from the device, and the device 200 may be located in a sensing area covered by the at least two RF sensing nodes 300’, 300”.

[0077] Alternatively, the device 200 may be located outside the sensing area covered by the at least two RF sensing nodes 300’, 300” but is controlled based on the sensing result obtained for the sensing area. To take lighting control as one example, it may be an option to create an RF sensing area in the garden using a few garden lights (integrated with sensing nodes) and let the detection result control a light inside the house or a light near the front door. When someone enters the garden, it may trigger an action, such as to alert the owner. A light near the front door may also help the owner to be aware that someone is approaching. According to the present invention, by identifying an unreachable garden light might trigger the front door light to remain on longer.

[0078] The controller 150 comprises a communication unit 151 configured to communicate with the device 200 and the plurality of nodes 300, 300’, 300”; and a processor 152 configured to: 2024PF80319

[0079] 10 control the device 200 with a control mode determined by radio frequency, RF, sensing performed by at least two nodes 300’, 300” out of the plurality of nodes 300, 300’, 300”; apply an adjustment to the control mode when detecting a first node 300’ out of the at least two nodes 300’, 300” unreachable by the controller 150 and / or a second node 300, 300” out of the plurality of nodes; wherein the adjustment comprises:

[0080] - to extend an operation duration of the device 200; and / or

[0081] - to change an operation configuration of the device 200.

[0082] As one implementation option, the control system 100 may be connected to a wireless LAN access point 17, e.g., via Ethernet or Wi-Fi. The wireless LAN access point 17 is connected to the Internet 20. The system 100 is able to communicate with the plurality of nodes 300, 300’, 300”, and the device 200, using a wired communication mode and / or a wireless communication mode, such as a short-range wireless communication technology.

[0083] The plurality of nodes 300, 300’, 300” may follow a same wireless communication standard. The wireless communication standard may be related to a standard of Zigbee, Thread, or Bluetooth Low Energy. The nodes are able to perform RF sensing to detect motions, presence, and other physical activities by measuring the changes in radio frequency signals caused by those activities. The plurality of nodes may also carry out data communication for the control system in addition to RF sensing.

[0084] RF sensing for presence detection or motion detection used for automated control focuses on identifying the presence of individuals or objects in a given area for purposes such as turning on lights, adjusting temperature, or triggering other automated actions. One way to perform RF sensing based presence detection is to observe the variations of received signal strength between two RF nodes in the sensing or detection area. Received Signal Strength Indicator (RSSI) is a measure of the power level of the received signal in wireless communication. Other parameters may be used for RF sensing comprise Channel State Information (CSI), phase shift, and Doppler shift.

[0085] In a communication or control system, when a node is unreachable, it typically means that other nodes cannot communicate with or get any response from that node. This may happen because the node is moving far away or obstructed by physical barriers such as walls or furniture. It may also happen if there is interference from other wireless signals or if the node is malfunctioning. When a node is unreachable, it cannot transmit or receive data, which can cause disruptions in the network and affect overall performance. 2024PF80319

[0086] 11

[0087] The detection of the first node 300’ unreachable may be determined by at least one of: missing a message or a response from the first node 300’ by the second node 300, 300”; missing a response from the first node 300’ to a polling message sent by the controller 150; detecting a degraded communication link with the first node 300’ by the controller 150 and / or the second node.

[0088] Since the first node 300’ is configured to perform RF sensing for the control of the device 200 and when the first node 300’ is not reachable, the corresponding RF sensing function will also be impaired.

[0089] In one example, the first node 300’ is configured to send messages periodically to the other node 300” of the at least two nodes, and the other node 300” is configured to detect the messages from the first node 300’. RF sensing is performed based on periodically detected messages, such as changes in RSSI, CSI, phase shift, and / or Doppler shift between messages. Adjacent messages for RF sensing may be sent with a time interval of less than one second, such as every 200ms or 400ms. The shorter the time interval, the control system is more sensitive to the change of presence status, and lower latency to the control of the device. When the other node 300” fails to detect one or more periodic messages from the first node 300’, it may report to the controller that the first node is unreachable. And then, the system may not be able to react to any motion or presence in the sensing area. To reduce the impact to a user, it is then necessary to adjust the control of the device, such as to extend an operation duration of the device 200; and / or to change an operation configuration of the device 200. The extended operation duration may help the control system to wait for the first node 300’ to be back to operation or to reconfigure the RF sensing pairs, such as to select another pair of nodes 300 to perform RF sensing for the control of the device 200. In the meanwhile, it may also be possible to apply a different operation configuration of the device 200, such as for power saving when the control system is not certain if there is people presence.

[0090] As an example, the device 200 is a lighting device. It is then beneficial to apply a longer hold time to keep light 200 on and / or to change the dimming level of the light 200, such as keeping the light on at least a minimum dimming level during the extended hold time as we don’t know if someone is there. This may help to reduce unnecessary energy waste.

[0091] Another possibility to detect the unreachable of the first node 300’ is that the controller 150 fails to receive a response from the first node 300’ to a polling message. 2024PF80319

[0092] 12

[0093] In a communication system, polling is widely used to regulate the communication between a controller, a bridge, or an access point and a plurality of nodes or devices in the network. Polling may also be used by a controller, a bridge, or an access point to check the status of a plurality of nodes or devices in the network, such as for network diagnostics.

[0094] The controller 150 of the present invention may be configured to send polling messages regularly to each of the plurality of nodes 300, 300’, 300”. The polling messages may be sent in unicast to each node sequentially. Depending on the number of nodes in the network and the transmission frequency of the polling messages, a node may be polled every few seconds. For the controller to determine that a certain node, such as the first node, is unreachable may also take several seconds when it fails to receive a response to the polling message from that node.

[0095] In response to the unreachable status of the first node 300’, the communication unit 151 of the controller 150 may be further configured to send a further polling message to the first node 300’; and / or increase a transmission frequency of sending polling messages to the first node 300’. In this manner, it will help the controller to detect any change to the status of the first node more quickly, such as when the first node comes back online or becomes reachable again in the network.

[0096] In the example of FIG. 1, a user of a user device 19 is able to use an app running on the user device 19 to control one or more of device 200 via the system / controller 150. The user device 19 may be, for example, a mobile device such as a mobile phone, a tablet or a smart watch. In the example of FIG. 1, user device 19 is connected directly to the wireless LAN access point 17. Alternatively, user device 19 may be connected to the Internet 100 remotely, e.g. via an LTE or 5G mobile communication network.

[0097] There may also be a remote server 18 connected to the control system 100 via the Internet.

[0098] The controller may be configured to provide the unreachable status of the first node 300’ to the remote server 18 or the user device 19.

[0099] It is also beneficial to restore the control mode for controlling the device 200 when detecting the first node 300’ reachable again, and then RF sensing or presence detection-based control of the device 200 is recovered.

[0100] The detection of the first node 300’ reachable again may be determined by at least one of: receiving a message or a response from the first node 300’ by the second node 300, 300”; receiving a response from the first node 300’ to a polling message sent by the 2024PF80319

[0101] 13 controller 150; detecting an improved communication link with the first node 300’ by the controller 150 and / or the second node 300, 300”.

[0102] Fig. 2 shows an example of the control system 100 according to the present invention.

[0103] There may be multiple active sensing pairs in the control system 100, where for each sensing pair a dash line is used to indicate the sending and receiving of periodic messages used for RF sensing. In the example, each sensing pair is illustrated as a one-to one pairing. It may also be the case that the sensing pairs are built up in a one to multiple manner, such that one node is configured to send messages periodically and more than one node are configured to detect the messages for RF sensing. In a further option, most or all nodes are configured to participate in RF sensing, such that the RF sensing pairs are established in a multiple-to-multiple or even all-to-all fashion. For example, it may be the case that in a small network, all the nodes are in the direct communication range of each other. And then, each node may have a timeslot for sending sensing messages, and all other nodes will listen to detect the sensing messages. In this way, a most enriched RF sensing fingerprint may be generated.

[0104] The control system may have more than one device 200, 200’ to be controlled, such as in a home or building automation system. In this example, the control system 100 comprises a further device 200’ located near the device 200. The further device 200’ may be placed as a standalone device or is connected with a node 300. In this example, the first node 300’ is attached to or integrated in the device 200. Upon detecting the unreachable of the first node 300’, the controller may be further configured to: extend an operation duration of the further device 200’; and / or change an operation configuration of the further device 200’ .

[0105] With the first node attached to or integrated in the device 200, the device may rely on the first node 300’ for communicating with the controller instead of having another communication means. Since the first node is unreachable, the adjustment to change the control mode of the device 200 may not be delivered and then it may be an option to adjust the control of the further device 200’ located near the device 200. This may help to reduce the negative experience to a user.

[0106] Fig. 3 shows another example of the control system 100 according to the present invention. The controller 150, the one or more devices 200, 200’, and the plurality of nodes 300, 300’, 300”may set up a wireless mesh network based on one of the aforementioned wireless communication standards and communicate with each other either 2024PF80319

[0107] 14 via a direct communication link or a multi-hop link. This helps to improve the reliability of the system, and an unreachable status of a node may be detected by more than one neighboring node, such as to enable quick diagnostic and the control system can apply a remedy solution immediately.

[0108] Fig. 4 shows an example of implementing the present invention in a lighting control system 100. In this example, the controller 150 may be a bridge device, and each of the plurality of nodes is connected to or integrated in a lighting device or a smart bulb. Note that it may also be the case that not all the nodes are connected or integrated in a lighting device. In another application scenario, a node may be connected to or integrated in another electronic device to be controlled. In a further option, a node may also be a standalone sensing node and provide sensing information to the controller.

[0109] The plurality of nodes or devices may be connected to the bridge in a wireless network, such as a Zigbee network, a BLE mesh network, a Thread network, or another mesh network.

[0110] The bridge 150 may be configured to regularly communicate with all nodes / devices / lights in the system to detect if the nodes are still responding and “reachable” via the mesh network and then synchronise to their current state. This action is known as “polling” and occurs roughly with a low frequency, e.g., every 3 seconds. For larger networks (e.g. 50 lights or more) it may take minutes before a light is checked again, and typically only after multiple “does not respond” experienced from a light is marked as “unreachable”. Therefore, for the lighting control scenario, the problems are:

[0111] • Informing users of “unreachable” lights is slow, while the messages among the nodes participating in RF sensing are sent more frequently, such as every 400 ms.

[0112] • For a room while lighting is controlled by RF sensing, it is more important to know that a light used for sensing is unreachable as motion / presence detection will also be affected.

[0113] • Depending on sensing devices being unreachable, certain lights might not be controlled.

[0114] According to the present invention, the following solutions are disclosed to improve the lighting control system:

[0115] • A default behaviour and updated automation configuration based on “unreachability”, like a longer hold time to keep lights on or a switch to at least a minimum level as we don’t know if someone is there. 2024PF80319

[0116] 15

[0117] • Improved “unreachable” detection by using the absence of a few cyclic (400 ms) RF sensing measurements from the same device to detect communication issues earlier.

[0118] • Improved / more-efficient “unreachable” detection by “polling” sensing devices with another frequency and including knowledge that humans in an area (detected by RF sensing) might degrade regular Zigbee communication causing an “unreachable” device.

[0119] FIG. 4 also shows a simplified view of the software modules in the bridge. The software modules are implemented in the processor or MCU of the bridge device. A bridge is controlled via a “bridge interface”, which may be according to a connected lighting interface protocol (CLIP). Commands received by the bridge are decoded via an interface daemon and distributed to software components via a “broker”. Such a command to switch on a light is of interest to “light control” which listens to this and will send to respective command over the mesh network to one or to a group of lights. Likewise, a “polled” change in light state is reported by “light control” via the “broker”. And if an APP or other client is connected to the bridge interface, the interface daemon will encode and report this.

[0120] Home automations may be configured in a similar way (e.g., bridge interface - > interface daemon -> broker -> automation engine). RF sensing may also be activated in a similar way (e.g., bridge interface -> interface daemon -> broker -> RF sensing).

[0121] When RF sensing is active, the RS SI, CSI or other parameters of the RF- sensing messages the nodes received from their peers will be collected to provide a measurement overview. By obtaining this overview by the bridge, an algorithm can run based on the overview to detect presence or motion, i.e., a person near a node influenced the RF communication. This algorithm is implemented in the “RF sensing” module shown in the figure.

[0122] The RF sensing result is then reported (via the broker) to the automation engine, which depending on the configuration decides to send a command (via the broker) to light control to change light settings. Note that, the nodes involved in the cyclic sensing might not be same devices that change their light setting. Secondly, since FIG. 4 is a simplification and skips components that implement e.g. the HTTP protocol of the bridge interface.

[0123] With the present invention, the following changes will be applied to different software modules:

[0124] Automation engine

[0125] It implements a new feature and listens to “unreachable” events from “light control”. 2024PF80319

[0126] 16

[0127] It adjusts its behaviour and its parameters. E.g. “unreachable” while the light is on, doubles the duration the light stays on. This avoids switching lights off when a human is present. Also, if “unreachable” lasts longer than the duration to stay on, it dims to a minimum acceptable level, until the devices are reachable again. As a result “unreachability” of sensors doesn’t put users into the dark.

[0128] Light control (when interfacing to RF sensing)

[0129] These light control component gets, directly or indirectly via the “broker”, additional information. Missing RF sensing messages from device X to Y is reported to “light control”. If nobody hears from device X for a few cycles, the device is considered “unreachable”.

[0130] Light control

[0131] This component adjusts its “polling” pattern to increase polling frequency of a specific node / device if RF sensing data keeps indicating the node / device is not sending data, so it can quickly detect if the device comes back (e.g., that device might have suffered a power outage or some interference). Likewise, if RF sensing data indicates the device is sending data again, it could be marked as “reachable”. And the typical RF sensing based control can take over.

[0132] Some further improvements can be made:

[0133] If the “automation engine” is informed of “unreachable” by light-control “polling” as well as “communication issues” from RF sensing and can differentiate them then its response might be different.

[0134] The RF sensing algorithm may have a “health” status which is based how many RF sensing measurements are missed. This could be used as a combined “area has reachability problems” next to a specific “device is unreachable”. A bad “health” might increase the duration to keep lights on, as we are not sure a user has left the area.

[0135] Fig. 5 shows a flow chart of a method 600 according to the present invention. A method 600 for controlling a device 200 in a control system 100; wherein the control system 100 comprising a controller 150, the device 200 and a plurality of nodes 300, 300’, 300”; the method 600 comprising the controller 150: controlling, in step S601, the device 200 with a control mode determined by radio frequency, RF, sensing performed by at least two nodes 300’, 300” out of the plurality of nodes 300, 300’, 300”; 2024PF80319

[0136] 17 applying, in step S603, an adjustment to the control mode when detecting in step S602 a first node 300’ out of the at least two nodes 300’, 300” unreachable by the controller 150 and / or a second node 300, 300” out of the plurality of nodes 300, 300’, 300”; wherein the adjustment comprises: extending an operation duration of the device 200; and / or changing an operation configuration of the device 200.

[0137] Beneficially, the detection of the first node 300’ unreachable is determined by at least one of missing a message or a response from the first node 300’ by the second node 300, 300”; missing a response from the first node 300’ to a polling message sent by the controller 150; detecting a degraded communication link with the first node 300’ by the controller 150 and / or the second node 300, 300”.

[0138] The method 600 may further comprise: providing an unreachable status of the first node 300’ to a remote server or a user device.

[0139] The described operations may be implemented as program code means of a computer program and / or as dedicated hardware of a node, a network device, or a controller device. For example, when the computer program product comprising instructions is executed on a processor 152 of a controller 150 according to the present invention, cause the controller 150 to carry out the steps of the method 600 according to the present invention.

Claims

2024PF8031918CLAIMS:

1. A method (600) for controlling a device (200) based on presence or motion detection in a control system (100); wherein the control system (100) comprising a controller (150), the device (200) and a plurality of nodes (300, 300’, 300”) configured to perform radio frequency, RF, sensing for presence or motion detection of individuals or objects in smart home or smart building environments; the method (600) comprising the controller (150): controlling (S601) the device (200) with a control mode determined by RF sensing performed by at least two nodes (300’, 300”) out of the plurality of nodes (300, 300’, 300”); applying (S603) an adjustment to the control mode when detecting (S602) a first node (300’) out of the at least two nodes (300’, 300”) unreachable by the controller (150) and / or a second node (300, 300”) out of the plurality of nodes (300, 300’, 300”); wherein the adjustment comprises: extending an operation duration of the device (200); and / or changing an operation configuration of the device (200).

2. The method (600) of claim 1, wherein the detection of the first node (300’) unreachable is determined by at least one of: missing a message or a response from the first node (300’) by the second node (300, 300”); missing a response from the first node (300’) to a polling message sent by the controller (150); detecting a degraded communication link with the first node (300’) by the controller (150) and / or the second node (300, 300”).

3. The method (600) of claim 2 further comprising at least one of: sending a further polling message from the controller (150) to the first node(300’);2024PF8031919 increasing a transmission frequency of sending polling messages from the controller (150) to the first node (300’).

4. The method (600) of any one of the previous claims, wherein the first node (300’) is attached to, connected to, or integrated in the device (200).

5. The method (600) of any one of the previous claims, wherein the device (200) is one of a lighting device, a TV, audio equipment, a fan, an air conditioner, or another home appliance.

6. The method (600) of any one of the previous claims further comprising: providing an unreachable status of the first node (300’) to a remote server or a user device.

7. The method (600) of any one of the previous claims, wherein the control system (100) comprising a further device (200’) located near the device (200), and the method (600) further comprises: extending an operation duration of the further device (200’); and / or changing an operation configuration of the further device (200’).

8. The method (600) of any one of the previous claims further comprising: restoring the control mode for controlling the device (200) when detecting the first node (300’) reachable again.

9. The method (600) of claim 8, wherein the detection of the first node (300’) reachable again is determined by at least one of: receiving a message or a response from the first node (300’) by the second node (300, 300”); receiving a response from the first node (300’) to a polling message sent by the controller (150); detecting an improved communication link with the first node (300’) by the controller (150) and / or the second node (300, 300”).

10. The method (600) of any one of the previous claims 1-7 further comprising:2024PF8031920 selecting another node (300), different from the first node (300’), out of the plurality of node to perform RF sensing for controlling the device (200).

11. A controller (150) for controlling a device (200) based on presence or motion detection in a control system (100), the control system (100) comprising the controller (150), the device (200) and a plurality of nodes (300, 300’, 300”) configured to perform radio frequency, RF, sensing for presence or motion detection of individuals or objects in smart home or smart building environments; the controller (150) comprising: a communication unit (151) configured to communicate with the device (200) and the plurality of nodes (300, 300’, 300”); and a processor (152) configured to:• control the device (200) with a control mode determined by RF sensing performed by at least two nodes (300’, 300”) out of the plurality of nodes (300, 300’, 300”);• apply an adjustment to the control mode when detecting a first node (300’) out of the at least two nodes (300’, 300”) unreachable by the controller (150) and / or a second node (300, 300”) out of the plurality of nodes; wherein the adjustment comprises:• to extend an operation duration of the device (200); and / or• to change an operation configuration of the device (200).

12. The controller (150) of claim 11, wherein the detection of the first node (300’) unreachable is determined by at least one of: missing a message or a response from the first node (300’) by the second node (300, 300”); missing a response from the first node (300’) to a polling message sent by the controller (150); detecting a degraded communication link with the first node (300’) by the controller (150) and / or the second node.

13. The controller (150) of claim 12, wherein the communication unit (151) is further configured to: send a further polling message to the first node (300’); and / or2024PF8031921 increase a transmission frequency of sending polling messages to the first node (300’).

14. A control system (100) comprising a controller (150) according to any one of the previous claims 11-13, a device (200), and a plurality of nodes (300, 300’, 300”) configured to perform radio frequency, RF, sensing, wherein the controller (150) is configured to control the device (200) with a control mode determined by RF sensing performed by at least two nodes (300’, 300”) out of the plurality of nodes (300, 300’, 300”).

15. A computer program product comprising instructions which, when the computer program product is executed on a processor (152) of a controller (150) according to any one of the claims 11-13, cause the controller (150) to carry out the steps of the method (600) according to any one of the claims 1-10.

Images