System and method for determining emotional state of user based on one or more body and / or physiological parameters

Abstract

A method and system for determining an emotional state of a user based on a radio frequency (RF)-based sensing system is disclosed. The system includes a presence detection unit configured to receive presence data and determine a presence of a user and a visitor in an environment. The system also includes a controller configured to receive first and second inputs indicative of the presence of a user and a visitor, respectively. The controller is further configured to set the RF-based sensing system to a first mode of operation when the user is present, and to switch the RF-based sensing system to a second mode of operation when the user and the visitor are present, where the RF-based sensing system operates according to a second set of RF sensing parameters when set to the second mode of operation, and where the RF-based sensing system operates according to a second set of RF sensing parameters when set to the second mode of operation. The second set of RF sensing parameters enables sensing of one or more body and / or physiological parameters of the user. The controller is further configured to obtain an RF signal when the RF-based sensing system is set to a second mode of operation, determine one or more body and / or physiological parameters of the user based on the RF signal, and determine that the emotional state of the user is an elevated emotional state based on the one or more body and / or physiological parameters, wherein the elevated emotional state is an emotional state that deviates from a baseline emotional state of the user.

Description

Technical Field

[0001] The present invention relates to a method for determining an emotional state of a user based on a radio frequency based sensing system. The present invention also relates to a system for determining an emotional state of a user. The present invention also relates to a computer program for determining an emotional state of a user based on a radio frequency based sensing system. Background Art

[0002] Mistreatment of the elderly includes intentional or neglectful behavior by a caregiver or trusted person that harms a vulnerable elderly person. Mistreatment of the elderly may occur in a combination of forms including emotional abuse, physical abuse, sexual abuse, financial exploitation, and neglect and abandonment. In addition, mistreatment of the elderly is associated with a variety of serious consequences, including increases in rates of physical injury, depression, emotional distress, functional decline, emergency room utilization, hospitalization, and morbidity and mortality. The public health impact of elder abuse may be far-reaching, as exposure to these phenomena produces many and different physical and psychological consequences.

[0003] Child abuse and maltreatment are recognized as important social problems with significant consequences for affected children, their families and society at large. Incidents of child abuse and maltreatment are often underreported as children are often reluctant to report the abuse. Summary of the invention

[0004] The inventors have recognized that when a user experiences abuse by a visitor (e.g., a caregiver or trusted person), he / she may show signs of physiological arousal (e.g., elevated levels of one or more physical and / or physiological parameters, such as elevated heart rate, elevated respiration rate, increased body movement, pupil dilation) in response to abuse-related stimuli. However, prior art solutions for monitoring such physical and / or physiological parameters (e.g., photoplethysmography, electrocardiography, plethysmography, etc.) are unobtrusive, while remote camera-based solutions may pose serious privacy and legal issues. Detecting such physiological arousal may be performed unobtrusively using radio frequency (RF)-based sensing. However, when a visitor is present, the RF sensing (configuration) parameters may need to be adjusted to optimally monitor the user's physical and / or physiological parameters.

[0005] In general, RF-based sensing can be performed for a number of different applications and different sensing targets, each of which requires a different configuration of radio frequency sensing. For example, if the sensing target refers to a simple presence detection task for controlling a light in a room, the accuracy of the detection is not necessarily very high, and the algorithm for radio frequency sensing may be based on the RSSI values ​​of different radio frequency signals, for example. However, more advanced applications (such as breathing monitoring of a user in a specific area of ​​the environment) may often require much higher accuracy together with a correspondingly higher spatial resolution. In this case, RF sensing can be performed based on an algorithm using the CSSI of the RF signal, where an additional specific signal path is weighted to be stronger than other signal paths. In general, the difference in RF sensing parameters and the specific configuration of RF sensing for specific applications and sensing targets can be regarded as different RF sensing modes. Therefore, as becomes apparent from the example, different operating modes of the RF-based sensing system can be used to optimally monitor the user's body and / or physiological parameters. For example, more resources may be required. For example, additional power and bandwidth may be required to better monitor the activities of users and visitors, certain (configuration) parameters (such as carrier frequency) may need to be adjusted to avoid interference, and so on. The inventors have recognised that by adjusting RF sensing parameters in the presence of a visitor, monitoring of physical and / or physiological parameters can be more accurate. It is therefore an object to provide a more accurate system for determining a user's emotional state based on the presence of a visitor.

[0006] According to a first aspect, the object is achieved by a system for determining an emotional state of a user, the system comprising: a radio frequency (RF) based sensing system comprising one or more nodes arranged for transmitting and / or receiving RF signals for RF based sensing; a presence detection unit configured to receive presence data and determine the presence of a user in an environment based on the received presence data, and further configured to determine the presence of a visitor in the environment based on the received presence data; and a controller configured to:

[0007] - receiving a first input indicating the presence of a user in the environment, and further receiving a second input indicating the presence of a visitor in the environment;

[0008] - setting the RF-based sensing system to a first operating mode when the user is present, wherein when set to the first operating mode, the RF-based sensing system operates according to a first set of RF sensing parameters;

[0009] - switching the RF-based sensing system to a second operating mode when the user and the visitor are present, wherein when set to the second operating mode, the RF-based sensing system operates according to a second set of RF sensing parameters that is different from the first set of RF sensing parameters, wherein the second set of RF sensing parameters enables sensing of one or more physical and / or physiological parameters of the user;

[0010] - when the RF-based sensing system is set to a second operating mode, obtaining an RF signal;

[0011] - determining one or more physical and / or physiological parameters of the user based on the RF signals; and

[0012] - Determining whether the user's emotional state is an elevated emotional state based on one or more physical and / or physiological parameters, wherein an elevated emotional state is an emotional state that deviates from the user's baseline emotional state.

[0013] When people experience abuse from visitors (e.g., caregivers or trusted people), certain physical and / or physiological parameters may change in response to stimuli associated with abuse. For example, when the elderly, infants / young children experience abuse from trusted people / caregivers, pets experience abuse from trusted trainers / caregivers, employees experience any form of emotional or physical abuse from their employers, etc., certain physical and / or physiological parameters (such as heart rate, respiratory rate, body movements, etc.) may change in response to the situation experienced. For example, when experiencing physical abuse from a trusted caregiver, an infant may show an elevated level of pain, which is manifested as an elevated heart rate or respiratory rate. In another example, a pet that suffers physical abuse from a trainer may show similar manifestations of physical and / or physiological parameters. Therefore, such changes in physical and / or physiological parameters may indicate an elevated emotional state of the user. In the context of the present disclosure, the term "elevated emotional state" may refer to an emotional state that deviates from the user's baseline emotional state. An emotional state may be defined by one or more values ​​indicating arousal and / or valence levels, and a baseline emotional state may be a baseline value indicating a baseline level of arousal and / or valence. Examples of elevated emotional states may include, but are not limited to, anger, stress, excitement, depression, happiness, sadness, etc. For example, when a user is angry or afraid, the user's heart rate may increase significantly, but in a disgusted emotional state, the heart rate may decrease significantly. In a pleasant emotional state, heart rate variability (HRV) may be much greater than in a fearful, neutral or angry emotional state. Additionally or alternatively, the user's breathing rate may change based on emotional changes. For example, when a user (adult, baby, pet, etc.) experiences emotional and / or physiological stress or when he (she) feels excited, an increase in breathing rate may be noticed, and the user's anxiety level may affect the breathing rate, especially the exhalation time. Wireless connection nodes arranged to transmit and / or receive radio frequency RF signals can be used for RF-based sensing, such as motion and presence detection, and can also be used for vital sign monitoring, such as heart rate and respiratory rate monitoring.

[0014] The RF-based sensing system can be configured to operate in different modes of operation, for example, by using different RF sensing (configuration) parameters and / or algorithm sets at different times. For example, a first set of RF sensing (configuration) parameters can be used to detect basic motion when no visitor is present, a different set of RF sensing parameters can be used to detect gestures and falls of a user, a second set of RF sensing parameters can be used to determine breathing rate if a visitor is present, and so on. Compared to simple presence detection or people counting tasks, RF-based physiological parameter (e.g., heart rate, breathing rate, etc.) monitoring generally requires more resources, such as increased data rates, increased number of nodes, more computationally demanding time-frequency signal analysis, and so on. For example, time drift in RSSI / CSI can be used for breathing rate detection, because breathing motion events require a long time to determine, such as about 20 human breaths per minute, while when trying to determine occupancy, gestures, or falls to the ground, such high-resolution time processing may be unnecessary, because these events require at most 1 or 2 seconds. Additionally and / or alternatively, certain parameters of the RF-based sensing system can be advantageously adjusted to avoid interference from external sources (e.g., visitors).

[0015] The controller is configured to set the RF-based sensing system to a first operating mode when only the user is present, wherein when set to the first operating mode, the RF-based sensing system operates according to a first set of RF sensing parameters. For example, when only the user is present, the RF-based sensing system can be configured to detect the presence of the user in the environment, perform gesture determination of the user, count the number of people in the environment, etc. Upon detecting a visitor (e.g., a caregiver, a trusted person, etc.), the controller can switch the RF-based sensing system to a second operating mode, wherein when set to the second operating mode, the RF-based sensing system operates according to a second set of RF sensing (configuration) parameters that are different from the first set of RF sensing parameters, wherein the second set of RF sensing parameters is used to sense one or more physical and / or physiological parameters of the user that indicate or reflect the emotional state of the user (e.g., heart rate, breathing rate, pupil size, etc.). By switching from the first operating mode to the second operating mode, wherein the RF sensing parameters of the second operating mode are selected for monitoring one or more physiological parameters of the user when the visitor is present, a more efficient and accurate way of monitoring one or more physiological parameters of the user is provided.

[0016] The controller can be configured to adjust / select the second RF sensing parameter set so that the performance of the RF-based sensing system is optimized for sensing one or more physical and / or physiological parameters of the user. The one or more physical and / or physiological parameters may include one or more of heart rate, breathing rate, and body movement. Since the second RF sensing parameter set is adjusted for the task of monitoring the physical and / or physiological parameters of interest, the performance and reliability of the RF-based sensing system are optimized for the application of interest.

[0017] The controller can be configured to receive input indicating the position and / or orientation of one or more nodes relative to a user and / or a visitor, and adjust / select a second set of RF sensing parameters such that interference from the presence of a visitor is avoided. For example, the controller can be configured to select a node mounted on a ceiling over a wall-mounted node because the field of view of the wall-mounted node may be blocked by the visitor. Because the controller can be configured to adjust / select a second set of RF sensing parameters such that interference from the presence of a visitor in sensing one or more physical and / or physiological parameters of a user is avoided, the performance of the system for sensing one or more physical and / or physiological parameters of a user is improved.

[0018] The presence detection unit may be configured to determine the presence of a user and / or a visitor in the environment based on RF signals transmitted and / or received by the RF-based sensing system when the RF-based sensing system is set to a first operating mode. For example, the controller may be configured to select / adjust a first RF sensing parameter set for presence detection or people counting. Since the RF-based sensing system can be used to detect the presence of a user and / or a visitor and can also be used to monitor one or more physical and / or physiological parameters of a user, the complexity of existing systems for determining the emotional state of a user can be reduced.

[0019] The presence detection unit can determine the presence of a user and / or visitor based on input from a user interface for manually inputting the presence of a user or the start of a visit by a visitor to a user. For example, a user, a visitor, or a person working around a user (e.g., a hospital staff member) can simply press a button when a user enters the environment and / or when a visit begins. Additionally and / or alternatively, the presence detection unit can determine the presence of a user and / or visitor based on presence data from one or more occupancy sensor devices located in the environment. This allows automatic recognition of the presence of a user and / or visitor in the environment.

[0020] The controller may be configured to determine whether the user's emotional state is an elevated emotional state by applying a machine learning model to one or more physical and / or physiological parameters. The machine learning model may be trained to determine elevated emotional states using one or more physical and / or physiological parameters as input. For example, a neural network, probability, regression model, etc. may be trained using one or more physical and / or physiological parameters as input and using labeled instances of elevated emotional states (i.e., labeled as nervous, angry, happy, etc.) as output.

[0021] Certain user activities (such as going to the toilet, going out, sleeping activities, etc.) may be associated with / indicate the user's elevated emotional state. The controller may also be configured to receive activity data indicating user activities over an extended observation period, extract features indicating user activities over the observation period based on the received activity data, and determine whether the user's emotional state is an elevated emotional state by applying a machine learning model (such as a support vector machine, a neural network, an optical gradient boosting machine (LGBM), etc.) to the extracted features. This improves the accuracy of the system in determining whether the user's emotional state is an elevated emotional state.

[0022] Long-term abuse or abuse-related trauma may be reflected in changes in patterns of certain activities of the user (e.g., changes in toileting, going out, and sleeping). The controller can be configured to receive activity data indicating the user's activities during an observation period, extract features indicating the user's activities during the observation period based on the received activity data, and determine the occurrence of an abuse event by applying a machine learning model to the extracted features. The machine learning model can be trained to detect the probability of an abuse event based on the features extracted during the observation period. By detecting changes in behavior over time, early symptoms or traumatic effects due to abuse can be detected, thereby enabling timely intervention to be provided to the user and stopping the abuse.

[0023] The controller may be configured to output a signal indicating the user's emotional state to the target device. For example, if the user's emotional state is an elevated emotional state, the user, hospital, nursing facility, etc. may be notified by a message at a device such as a personal smart phone or other digital interface (e.g., a web user interface). Additionally and / or alternatively, the controller may be configured to output a collection of the user's emotional state in digital or graphical form, for example, on a dashboard with key emotional state indicators and / or one or more physical and physiological parameters. Such a dashboard may be presented on a digital user interface (e.g., a web user interface).

[0024] The controller may be configured to receive data indicating a visitor's visitor identification and to associate the user's (elevated) emotional state with the visitor identification. The data may include video or voice recordings that allow identification of visitors based on, for example, facial and / or voice recognition, clothing recognition of people working in a care facility / hospital, etc. Additionally or alternatively, the data may include a visitor's weight estimate, respiratory rate, and / or heart rate recording that allows identification of visitors based on the visitor's weight, respiratory rate, and / or heart rate pattern. Associating the user's emotional state with the visitor identification allows identification of candidate / possible abusers. For example, in an assisted living facility / hospital, specific caregivers who may intentionally or unintentionally mistreat an elderly person may be identified by associating users showing signs of abuse with identifications of specific visitors / caregivers in many different rooms in the assisted living facility / hospital.

[0025] The set of RF sensing parameters may include one or more of: receive sensitivity, orientation of one or more nodes, number of nodes at different relative positions, connectivity of one or more nodes to one or more other nodes, one or more frequency channels, transmit power, transmit beamforming, receive beamforming, and bandwidth.

[0026] The orientation of one or more nodes may include the orientation of one or more nodes relative to a specific volume, one or more other nodes, a user, a visitor, or a combination thereof. For example, an RF sensing node may use beamforming to adjust its detection area. The coverage of the environment may be increased by increasing the power of the RF sensing node. The connectivity of one or more nodes to one or more other nodes may include a data transfer rate between nodes, such as a data transfer rate required for real-time heart rate monitoring. For example, if the data transfer rate between nodes is lower than the rate required for real-time heart rate detection, the RF sensing parameters may be adjusted to increase the data transfer rate. The connectivity of one or more nodes to one or more other nodes may also include, for example, the number of retries for transmitting data. For example, if a higher number of retries is required for transmitting data, the node may have worse connectivity. The number of nodes at different relative positions may refer to the number of nodes at different relative positions relative to the user in the environment, or the number of nodes at different relative positions of a group of nodes in the environment. A group of nodes may, for example, include a group or all nodes arranged in a room, a floor, a house, etc. For example, a group of nodes may include a node mounted on a ceiling and four table nodes. If two of the four table nodes are located in an area of ​​the room where the user's view is blocked (e.g., due to the presence of visitors), then in the second operating mode, only the ceiling-mounted node and the two table nodes are selected for RF-based sensing. This improves the performance of sensing one or more physical and / or physiological parameters of the user.

[0027] In many cases, the user may continue to show elevated levels of one or more physical and / or physiological parameters for a period of time after the visitor ends the visit. The controller may also be configured to receive data indicating the end of the visitor's visit to the user, and switch the RF-based sensing system to the first operating mode a period of time after the visit ends. The time period may be fixed or dynamic based on the emotional state of the user determined during the visit. Monitoring one or more physical and / or physiological parameters of the user for a period of time after the visit ends allows for an estimation of the user's recovery time after the visit ends, for example, the time it takes for the user to go from an elevated emotional state to a baseline emotional state after the visit.

[0028] According to a second aspect, the object is achieved by a method for determining an emotional state of a user based on an RF-based sensing system, the RF-based sensing system comprising one or more nodes arranged for transmitting and / or receiving RF signals for RF-based sensing, the method comprising the following steps:

[0029] - receiving a first input indicating the presence of a user in the environment;

[0030] - receiving a second input indicating the presence of a visitor in the environment;

[0031] - setting the RF-based sensing system to a first operating mode when the user is present, wherein when set to the first operating mode, the RF-based sensing system operates according to a first set of RF sensing parameters;

[0032] - switching the RF-based sensing system to a second operating mode when the user and the visitor are present, wherein when set to the second operating mode, the RF-based sensing system operates according to a second set of RF sensing parameters that is different from the first set of RF sensing parameters, wherein the second set of RF sensing parameters is adjusted for sensing one or more physical and / or physiological parameters of the user;

[0033] - when the RF-based sensing system is set to a second operating mode, obtaining an RF signal;

[0034] - determining one or more physical and / or physiological parameters of the user based on the RF signal;

[0035] - Determining whether the user's emotional state is an elevated emotional state based on one or more physical and / or physiological parameters, wherein an elevated emotional state is an emotional state that deviates from the user's baseline emotional state.

[0036] According to a third aspect, the object is achieved by a computer program configured to perform a method for determining an emotional state of a user based on an RF based sensing system.

[0037] It should be understood that the systems, methods, and computer program products may have similar and / or the same embodiments and advantages as the lighting device described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and additional objects, features and advantages of the disclosed systems, devices and methods will be better understood through the following illustrative and non-limiting detailed description of embodiments of the devices and methods with reference to the accompanying drawings, in which:

[0039] Figure 1 schematically illustrates a block diagram of a system for determining an emotional state of a user;

[0040] Figure 2 An example of a system for determining an emotional state of a user is schematically shown;

[0041] Figure 3 A flow chart of a method for determining an emotional state of a user is schematically shown.

[0042] All the figures are schematic, not necessarily to scale, and generally show only parts which are necessary in order to elucidate the invention, wherein other parts may be omitted or merely suggested. DETAILED DESCRIPTION

[0043] Figure 1 An example of a system 100 for determining an emotional state of a user is shown. The system 100 includes a radio frequency (RF) based sensing system 102 including one or more nodes 122, 124 arranged to transmit and / or receive RF signals 18 for RF based sensing. The one or more nodes 122, 124 may be, for example, in the form of a Wi-Fi transceiver node 122 arranged to transmit RF signals 18 to a receiver node 124, in the form of a radar sensor 122 arranged to transmit and receive RF signals 18, in the form of a group of radar sensors 122, 124 arranged to transmit and receive RF signals 18, and the like.

[0044] The system 100 also includes at least one data processor or controller 106. The system 100 may also include at least one data repository or storage or memory 108 for storing computer program code instructions.

[0045] The system 100 also includes a presence detection unit 104, which is configured to receive presence data, such as an RF signal, an input signal from a user interface, data from one or more occupancy sensors (e.g., a camera, an audio recording device, a PIR sensor, a CO2 sensor, a vibration sensor, an RFID tag, etc.), and determine the presence of a user in the environment based on the received presence data. The presence detection unit 104 is also configured to determine the presence of a visitor of the user in the environment based on the received presence data. The presence detection unit 104 can be integrated in the controller 106, or can be a separate device (e.g., a wearable device in the form of a button or touch screen so that the user can input the start of a visit; a sensing arrangement, e.g., a camera, a presence sensor, etc. configured to sense the presence of a person in the environment).

[0046] Figure 2 An example of a system for determining an emotional state of a user is schematically shown. The controller 106 is configured to receive a first input indicating the presence of a user 222 in an environment 220. The environment 220 may be a room that may be limited by walls, floors and ceilings, an open space arrangement, a corridor, etc. When the user 222 is present in the environment 220, the controller 106 is configured to set the RF-based sensing system to a first operating mode. When set to the first operating mode, the RF-based sensing system 202 including nodes 204, 208, 206 operates according to a first RF sensing (configuration) parameter set. For example, a parameter set of the RF-based sensing system may be selected to perform the task of detecting the presence of the user 222. Additionally or alternatively, a parameter set of the RF-based sensing system may be selected to perform a person count in the environment 220, a determination of the gesture of the user 222, etc. The controller 106 is also configured to receive a second input indicating the presence of a visitor 224 in the environment 220. When a visitor 224 is detected in the environment 220, the controller 106 is configured to switch the RF-based sensing system 202 to a second operating mode. When set to the second operating mode, the RF-based sensing system 202 operates according to a second set of RF sensing (configuration) parameters that are different from the first set of RF sensing parameters, wherein the second set of RF sensing parameters enables sensing of one or more physical and / or physiological parameters of the user 222. The controller 106 is configured to obtain RF signals and determine one or more physical and / or physiological parameters of the user, such as heart rate, respiration rate, body mass index, etc., based on the RF signals.

[0047] For example, a Doppler radar sensing system 202 including a set of nodes 204, 206, 208 can be used for respiratory monitoring by transmitting a signal 18 and by receiving a reflected signal 18 having a Doppler shift caused by the periodic movement of the user's chest. The breathing rate of the user 222 can be extracted from the Doppler shift. Respiratory rate monitoring may require the radar sensing system 202 to use a higher transmission power relative to the transmission power of the nodes for accurate monitoring when the system is operating in a first operating mode and / or with a specific frequency bandwidth (because the respiratory rate falls within a specific range). Therefore, the RF sensing parameters of the RF sensing system 202 can be appropriately switched from a first RF sensing parameter set to a second RF sensing parameter set configured for the respiratory rate monitoring task.

[0048] Alternatively or additionally, the RF sensing system 202 may include 802.15.4-compliant or Wi-Fi-compliant wireless sensor nodes 204, 206, 208, and the received signal strength (RSS) signal 18 transmitted between the wireless sensor nodes 204, 206, 208 may be used for respiratory rate or heart rate monitoring. A greater number of RF sensor nodes (and therefore more resources) may be required for accurate monitoring compared to the first operating mode. In another example, only RF sensor nodes in close proximity to the chest of the user 222 may be used for accurate monitoring. Wireless sensor nodes are typically susceptible to interference from other devices; therefore, the RF-based sensing system may switch to an operating frequency different from the operating frequency in the first operating mode to avoid interference from other devices.

[0049] The controller 106 is configured to determine whether the emotional state of the user 222 is an elevated emotional state based on one or more physical and / or physiological parameters. For example, if the user 222 experiences abuse by the visitor 224 (the visitor may be a health care provider (e.g., a nurse, a doctor, etc.), a trusted person, etc.), certain physical and / or physiological parameters (such as heart rate, breathing rate, body movement, etc.) may change in response to stimuli associated with the abuse. Thus, such physical and / or physiological parameters may indicate / associated with an elevated emotional state of the user 222. For example, the user 222 may experience an elevated heart rate compared to the user's baseline heart rate measurement. Thus, the controller may determine that the user's emotional state is a stressed emotional state in response to the presence of the visitor 224.

[0050] The presence of visitor 224 in environment 220 may hinder the accuracy of RF-based sensing system 202 in monitoring one or more physical and / or physiological parameters of user 222. For example, visitor 224 may be located in the line of sight between sensor node 206 and user 222, which may reduce the accuracy of RF-based system 202 for monitoring one or more physical and / or physiological parameters of user 222. Controller 106 may be configured to receive input indicating the position and / or orientation of one or more nodes relative to visitor 224 and user 222, and select a second set of RF sensing parameters to avoid interference caused by the presence of visitor 224, thereby sensing one or more physical and / or physiological parameters of user 222. For example, controller 106 may be configured to select only nodes 204 and 208 because the RF signal transmission of node 206 may be blocked by visitor 224.

[0051] The controller 106 may use the location and / or orientation of the visitor 224 relative to the user 222 to select a second set of RF sensing parameters to avoid interference. Figure 2 In the illustrative example of FIG, user 222 may be oriented toward node 208, and accordingly visitor 224 may be oriented toward node 206. However, the difference in distances between the chests of user 222 and visitor 224 to node 208 may make it difficult to identify the user's breathing signal using node 208. Therefore, controller 106 may select node 204 whose orientation angle is second closest to user 222 for monitoring one or more physical and / or physiological parameters of user 222.

[0052] The position and / or orientation of a node can be determined manually or automatically. The position of a node can be manually inserted by a user, for example, via a user interface (such as a user input device with a touch screen or a display and a keyboard). The position and / or orientation can also be determined automatically, for example, based on tracking the position and environment of the node (for example, via a camera), or in any other manner known to those skilled in the art. Alternatively or additionally, the position and / or orientation can be node parameters stored for each node on the node, for example, during the production of the node, the arrangement of the node, or both. Alternatively or additionally, the nodes can be configured to determine their position and / or orientation. The nodes can be configured to provide / transmit their position to the controller 106. The technology for determining the position of nodes and / or people is known in the art and therefore will not be discussed in further detail.

[0053] The RF-based sensing system 202 may be configured to operate in a first operating mode, wherein when set to the first operating mode, the RF sensing parameters are adjusted for performing presence detection, e.g., presence detection of a user 222 and / or a visitor 224. RF signals 18 transmitted and received by nodes including the RF-based sensing system 202 may be received and analyzed by the presence detection unit 104 to determine the presence of the user 222 and / or the presence of the visitor 224 in the environment 220, e.g., by analyzing interference of the RF signals 18 caused by the presence of the user 220 and / or the visitor 224.

[0054] Additionally or alternatively, the presence detection unit 104 can be configured to receive input signals from a user interface. The user interface can include a simple button, dedicated software (e.g., visitor registration software), etc. For example, when the user 222 and / or the visitor 224 are present in the environment 202, the user, the visitor, or a person working around the user (e.g., a nurse, a doctor, etc.) can press a button or manually input in the dedicated software.

[0055] Additionally or alternatively, the presence detection unit 104 may receive data from one or more occupancy sensors (cameras, microphones, CO2 sensors, vibration sensors, PIR sensing systems, radar sensors, etc.) distributed in the environment 202, and determine the presence of the user 222 and / or the presence of the visitor 224 based on the received presence data. For example, a people counting algorithm may be implemented by the controller 106 to analyze the presence data received from the PIR sensing system to count the number of people in the environment 202 and thus identify the presence of the user 222 and / or the visitor 224. The presence detection unit 104 may distinguish the presence of the user 222 from the presence of the visitor 224 based on the received data. For example, a facial recognition algorithm may be implemented by the presence detection unit 104 to analyze the video data received from the camera to distinguish the user 222 and / or the visitor 224 (to distinguish between the user 222 and the visitor 224). In another example, a speech recognition algorithm may be implemented by the presence detection unit 104 to analyze the received audio data for the same purpose. In yet another example, the presence detection unit 104 can determine and distinguish the presence of the user 222 and / or visitor 224 based on data received from one or more occupancy sensors, wherein the one or more occupancy sensors detect signals emitted from personal devices (e.g., phones, smart watches, or badges equipped with BLE beacons associated with the user 222 and / or visitor 224). In another example, a breathing rate and / or heart rate signature associated with the user 222 and / or visitor 224 can be used to distinguish between the user 222 and the visitor 224. These methods are merely examples. Several techniques for determining the presence of occupants in a space and distinguishing between occupants based on occupancy sensor data are known in the prior art.

[0056] The controller 106 may be configured to determine whether the emotional state of the user 222 is an elevated emotional state (e.g., sad, nervous, angry, happy, etc.) by applying the machine learning model to one or more physical and / or physiological parameters. For example, the determined one or more physical and / or physiological parameters of the user 222 may be used as input to the machine learning model to determine whether the current emotional state of the user 222 is an elevated emotional state. The trained machine learning model may make such a determination because the machine learning model may have been trained with inputs that may include instances (time series data) of one or more physical and / or physiological parameters of the user 222 or instances (time series data) of one or more physical and / or physiological parameters from a group of similar users (e.g., users sharing a similar cultural background, age group, medical / mental condition, etc.), and output a corresponding labeled instance of an elevated emotional state for the user or a group of similar users.

[0057] Changes in certain activity patterns such as going to the toilet, going out, sleeping activities, etc. can indicate the user's elevated emotional state. The controller 106 can be configured to receive activity data indicating the user's activity within the observation period. For example, the RF-based sensing system 202, motion sensors such as PIR, single-pixel thermopile sensors, etc. can be used to detect small, large and medium movements of the user 222. The controller 106 can apply a feature learning algorithm to the time series of the received activity data within the observation period (e.g., one week, one month, etc.) to extract features related to the user's activity or indicating the user's activity. These features may include, but are not limited to, the total number of times the activity is performed, the frequency of the activity, the rolling average of daily activities within the observation period, etc. The controller 106 can be configured to determine whether the user 222's emotional state is an elevated emotional state (e.g., sadness, tension, anger, happiness, etc.) by applying a machine learning model to one or more physical and / or physiological parameters and the extracted features indicating the user's activity within the observation period. For example, the features extracted from the received activity data and the determined one or more physical and / or physiological parameters of the user can be used as input to the machine learning model to determine whether the user's current emotional state is an elevated emotional state. A trained machine learning model can make such a determination because the machine learning model may have been trained with inputs that may include instances of one or more physical and / or physiological parameters and a series of activity data instances over time, and output corresponding labeled instances of elevated emotional states for the user or the general population.

[0058] Chronic abuse or abuse-related trauma may be reflected in changes in patterns of certain activities of the user (e.g., changes in toileting, going out, and sleeping). For example, if the user does not drink enough water during the day, this will lead to dehydration and increase the number of toilet visits. The controller 106 can be configured to receive activity data for an observation period, extract features indicative of the user's activities during the observation period based on the received activity data, and determine the occurrence of an abuse event by applying a machine learning model to the extracted features. The machine learning model can be trained to determine the probability of abuse or abuse-related trauma based on the extracted features marked during the abuse event or situation.

[0059] The controller 106 may be configured to output a signal indicating the emotional state of the user 222 to the target device. The output signal may have different forms. For example, if the user's emotional state is determined to be an elevated emotional state, the output signal may be a message and / or notification to a device such as a personal smart phone or other digital interface. In another example, when the user 222 is experiencing stress, a notification may be sent to a digital interface located in a care facility. For example, in the case where the user's emotional state is an elevated emotional state, the output signal may be in the form of a visual notification, such as a flashing illuminator; an auditory signal, such as a special tone; and / or a perceptible warning, such as a vibration. Additionally or alternatively, the user may be notified that he (she) is angry by a message in a personal device (e.g., a smart phone). Additionally or alternatively, the controller 106 may be configured to output a collection of the user's emotional state in digital or graphical form, for example, on a dashboard having a key emotional state indicator and / or one or more physical and physiological parameters. Such a dashboard may be presented on a digital user interface (e.g., a web user interface). For example, the controller 106 may output the user's 222 stress, anger, and other levels within an observation period. In further examples, the controller 106 may output the determined heart rate or breathing rate of the user 222 in graphical form.

[0060] The controller 106 may be configured to receive data indicating a visitor identification of the visitor 224 and to associate the (elevated) emotional state of the user 222 with the visitor identification. For example, the data may include an RFID tag to identify the visitor 224, such as a person working in a hospital, a nursing facility, etc. In another example, the data may include a video or voice recording of the environment 220. The controller 106 may, for example, apply facial and / or voice recognition, clothing recognition of a person working in a nursing facility / hospital to identify the visitor 224. Additionally and / or alternatively, the data may include RF signals 18 transmitted and / or received by the RF-based sensing system 102. The controller 106 may, for example, determine certain physical and / or physiological parameters of the visitor 224, such as breathing rate, body mass index, etc., which allow identification of the visitor 224. For example, the visitor 224 may be identified based on his / her weight and a unique breathing rate and / or heart rate pattern.

[0061] The controller 106 may be configured to select a second set of RF sensing parameters of the RF-based sensing system 102 for monitoring one or more physical and / or physiological parameters of the user. The set of RF sensing parameters that may be adjusted / selected may include, for example, receive sensitivity, orientation of one or more nodes, number of nodes at different relative positions, connectivity of one or more nodes to one or more other nodes, one or more frequency channels, transmission power, transmit beamforming, receive beamforming, and bandwidth.

[0062] For example, if respiration rate detection of user 222 is desired when both user 222 and visitor 224 are present in environment 220, the wireless communication signal may need to be spatially confined to determine small respiration-related chest movements of user 222, while keeping electromagnetic radiation as far away from visitor 224 as possible, e.g., to avoid interference from hand movements of user 222 when talking to visitor 224. In this case, RF-based sensing system 202 may adjust the operating frequency in the second operating mode and operate at a higher channel frequency compared to the channel frequency of the first operating mode, e.g., 5 GHz / 6 GHz compared to 2.4 GHz. Preferably, RF sensing-based system 201 operates at millimeter wave or even THz RF sensing.

[0063] The connectivity of one or more nodes may refer to the data rate at which messages are exchanged between nodes of the RF-based sensing system 202. For example, for occupancy and respiration detection, RF wireless communication signals are typically sent at a data rate of 30 Hz, while for heartbeat or fall detection, a higher data rate (sampling frequency) may preferably be used, e.g., 1000 Hz may be used for fall detection.

[0064] In another example, for the task of breathing rate detection, when operating in the second operation mode, a channel bandwidth higher than 80 MHz may be required. In contrast, when operating in the first operation mode, for example for the task of occupancy detection, a channel bandwidth of 20 MHz may be sufficient for basic motion sensing.

[0065] In another example, the controller 106 may increase the data rate of the RF-based sensing system 202 to be able to perform heart rate or respiration rate monitoring, increase the transmission power of the RF-based sensing system 202 to increase coverage, increase the number of retries to increase robustness, use beamforming on the RF-based sensing system 202 to adjust the detection area of ​​the node, etc. The controller 106 may be configured to adjust / select a second set of RF sensing parameters for the RF-based sensing system 102 such that the performance of the RF-based sensing system 102 is optimized for the task of monitoring one or more physical and / or physiological parameters of the user 222.

[0066] For example, Figure 2 A situation is shown where visitor 224 is blocking the line of sight between node 206 and user 222. Controller 106 may select only nodes 204 and 206 to monitor one or more physical and / or physiological parameters of user 222 to optimize the performance of RF-based sensing system 202.

[0067] It should be understood that the above examples of RF sensing parameters for the first or second operating modes are merely examples and that a person skilled in the art will be able to conceive of alternative solutions without departing from the scope of the appended claims.

[0068] In many cases, the emotional state of user 222 may remain elevated for a period of time after the visit from visitor 224 ends. The controller 106 may also be configured to receive data indicating the end of the visit of the visitor to user 222, and switch the RF-based sensing system to the first operating mode after a period of time after the visit ends. The controller 106 may also be configured to estimate the recovery time, i.e., the time for user 222 to transition from an elevated emotional state to a baseline emotional state, based on one or more physical and / or physiological parameters of the determined user 222. For example, the controller 106 may continue to operate in the second operating mode and monitor the heart rate of user 222 for a period of time after the visit. In this case, the controller 106 may determine the recovery time of user 222 by summing the time that the heart rate of user 222 remains above a predetermined baseline heart rate threshold. Alternatively, the controller 106 may be configured to obtain the recovery time, for example, from a memory storing the recovery time of the user. The controller 106 may also be configured to switch the RF-based sensing system to the first operating mode after the recovery time.

[0069] Figure 3 A method for using a RF-based sensing system (e.g., based on Figure 1 An embodiment of a method 300 for determining an emotional state of a user using an RF-based sensing system 102) is presented in FIG.

[0070] In step 302 , a first input indicating the presence of user 222 in environment 220 is received.

[0071] In step 304 , a second input is received indicating the presence of visitor 224 in environment 220 .

[0072] In step 306, the RF-based sensing system is set to a first mode of operation.

[0073] In step 308, it is determined whether a visitor is present. If so, the RF-based sensing system is set to a second mode of operation.

[0074] When the RF-based sensing system is set to the second operating mode, steps 310-314 are performed. Step 310 includes obtaining an RF signal. Step 312 includes determining one or more physical and / or physiological parameters of the user based on the RF signal. Step 314 includes determining whether the user's emotional state is an elevated emotional state based on the one or more physical and / or physiological parameters.

[0075] The method 300 may be performed by computer program code of a computer program product when the computer program product is executed on a processing unit of a computing device, such as the controller 106 of the system 100 .

[0076] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.

[0077] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer or processing unit. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0078] Aspects of the present invention may be implemented in a computer program product, which may be a collection of computer program instructions stored on a computer-readable storage device that may be executed by a computer. Instructions of the present invention may be any interpretable or executable code mechanism, including but not limited to scripts, interpretable programs, dynamic link libraries (DLLs) or Java classes. Instructions may be provided as complete executable programs, partial executable programs, as modifications (e.g., updates) of existing programs, or as extensions (e.g., plug-ins) of existing programs. In addition, partial processing of the present invention may be distributed over multiple computers or processors or even "clouds".

[0079] Storage media suitable for storing computer program instructions include all forms of non-volatile memory, including but not limited to EPROM, EEPROM and flash memory devices, magnetic disks such as internal and external hard drives, removable disks and CD-ROM disks. The computer program product may be distributed on such storage media or may be provided for download via HTTP, FTP, email or through a server connected to a network such as the Internet.

Claims

1. A system (100) for determining an emotional state of a user (222), the system comprising: - a radio frequency (RF) based sensing system (102, 202) comprising one or more nodes (122, 124, 204, 206, 208) arranged to transmit and / or receive RF signals (18) for RF based sensing; a presence detection unit (104) configured to receive presence data, determine the presence of a user in the environment (220) based on the received presence data, determine the presence of a visitor (224) in the environment (220) based on the received presence data, and distinguish the presence of the user from the presence of the visitor based on the received data; - a controller (106) configured to: receiving a first input indicating the presence of a user (222) in the environment (220), and further receiving a second input indicating the presence of a visitor (224) in the environment (220), When a user (222) is present, the RF-based sensing system (102, 202) is set to a first operating mode, wherein when set to the first operating mode, the RF-based sensing system (102, 202) operates according to a first set of RF sensing parameters, switching the RF-based sensing system (102, 202) to a second operating mode when a user (222) and a visitor (224) are present, wherein when set to the second operating mode, the RF-based sensing system (102, 202) operates according to a second set of RF sensing parameters that is different from the first set of RF sensing parameters, wherein the second set of RF sensing parameters enables sensing of one or more physical and / or physiological parameters of the user, When the RF-based sensing system (102, 202) is set to a second operating mode, an RF signal (18) is obtained, determining one or more physical and / or physiological parameters of the user based on the RF signals, and A determination is made based on one or more physical and / or physiological parameters as to whether the user's emotional state is an elevated emotional state, wherein an elevated emotional state is an emotional state that deviates from a baseline emotional state of the user.

2. The system of claim 1, wherein the second set of RF sensing parameters enables performance of the RF-based sensing system to be optimized for sensing one or more physical and / or physiological parameters of a user.

3. A system according to any preceding claim, wherein the controller is configured to receive input indicating the position and / or orientation of the one or more nodes relative to a visitor and / or a user, and wherein the second set of RF sensing parameters is used to avoid interference from the presence of the visitor with sensing one or more physical and / or physiological parameters of the user.

4. A system according to any one of the preceding claims, wherein: - the presence data is RF signals (18) transmitted and received by the RF-based sensing system (102) when the RF-based sensing system is set to a first operating mode, wherein a first set of RF sensing parameters is used for presence detection of users and / or visitors; as well as The presence detection unit (104) is configured to determine the presence of a user (222) and / or the presence of a visitor (224) based on the RF signal.

5. A system according to any one of the preceding claims, wherein the presence data is one of: - input signals from a user interface; -Data from one or more occupancy sensors located in the environment.

6. A system according to any of the preceding claims, wherein the controller is configured to determine whether the user's emotional state is an elevated emotional state by applying a machine learning model to one or more physical and / or physiological parameters, wherein the machine learning model is trained to determine elevated emotional states based on one or more physical and / or physiological parameters.

7. A system according to claim 6, wherein the controller is configured to receive activity data indicating user activity during an observation period, and extract features indicating user activity during the observation period based on the activity data, and wherein the machine learning model is trained to determine the elevated emotional state based on one or more physical and / or physiological parameters and the extracted features.

8. A system according to any one of the preceding claims, wherein the controller is configured to: - receive activity data indicative of user activity during an observation period, - extracting features from the activity data indicating the user's activity during the observation period, and - Determining the occurrence of an abuse event by applying a machine learning model to the features, wherein the machine learning model is trained to determine the probability of an abuse event using the features over an observation period as input.

9. The system according to any one of the preceding claims, the controller being further configured to: - Outputting a signal indicative of the user's emotional state to a target device.

10. The system according to any one of the preceding claims, wherein the controller is further configured to: - receiving data indicative of a visitor identification of a visitor, -Associate the user's emotional state with the visitor identification.

11. A system according to any preceding claim, wherein the RF sensing parameter set includes one or more of the following: receive sensitivity, orientation of one or more nodes, number of nodes at different relative positions, connectivity of one or more nodes to one or more other nodes, one or more frequency channels and bandwidths, transmit beamforming, receive beamforming.

12. The system according to any one of the preceding claims, wherein the controller is configured to: - receiving data indicating the end of a visit by a visitor; - estimating a user's recovery period based on one or more physical and / or physiological parameters; - After a period of time after the visit ends, switching the RF-based sensing system to the first mode of operation.

13. A controller according to any preceding claim, for use in a system according to any preceding claim.

14. A method for determining an emotional state of a user (222) based on a radio frequency (RF) based sensing system (102, 202), the RF based sensing system comprising one or more nodes (122, 124, 204, 206, 208) arranged to transmit and / or receive RF signals (18) for RF based sensing, the method comprising the steps of: - receiving (302) a first input indicating the presence of a user in the environment; - receiving (304) a second input indicating the presence of a visitor in the environment; - based on the first input and the second input, distinguishing the presence of the user from the presence of the visitor; - setting (306) the RF-based sensing system to a first operating mode when the user is present, wherein when set to the first operating mode, the RF-based sensing system operates according to a first set of RF sensing parameters; - switching (308) the RF-based sensing system to a second operating mode when the user and the visitor are present, wherein when set to the second operating mode, the RF-based sensing system operates according to a second set of RF sensing parameters that is different from the first set of RF sensing parameters, wherein the second set of RF sensing parameters enables sensing of one or more physical and / or physiological parameters of the user; - when the RF-based sensing system is set to a second operating mode, obtaining (310) an RF signal; - determining (312) one or more physical and / or physiological parameters of the user based on the RF signal; - Determining ( 314 ) whether the user's emotional state is an elevated emotional state based on one or more physical and / or physiological parameters, wherein an elevated emotional state is an emotional state that deviates from the user's baseline emotional state.

15. A computer program product for a computing device, the computer program product comprising computer program code for performing the method according to claim 14 when the computer program product is run on a processing unit of the computing device.

Images