Method for determining a label for a fall event

Abstract

A method for determining a truthful label of a fall event is disclosed. The method includes receiving signals from one or more sensors configured to remotely measure signals indicative of user movement characteristics, analyzing the received signals using a fall detection algorithm to determine a label indicative of a fall event by the user, and initiating a first user interface interaction mode of a user interface, wherein in the first user interface interaction mode, the user interface is configured to receive a first input from the user indicating a self-label of the fall event, receiving the first input, and determining a level of mismatch between the self-label and the determined label. If the level of mismatch exceeds a threshold, the method further includes switching the user interface to a second user interface interaction mode, wherein in the second user interface interaction mode, the user interface is configured to receive a second input from the user indicating contextual information about the fall event, receiving the second input, and updating the self-label of the fall event based on the received second input.

Description

[Technical Field]

[0001] The present invention relates to a method for determining a label of a fall event.The present invention further relates to a controller for determining a label of a fall event.The present invention further relates to a system for determining a label of a fall event. [Background technology]

[0002] Falls are a serious problem in elderly care, potentially leading to morbidity and mortality among the elderly. Falls not only potentially injure the elderly, but also psychologically, often resulting in a fear of falling, which in turn leads to social isolation and depression. As our society continues to age, there is an urgent need to develop fall detection and / or prevention systems. Thanks to the rapid development of sensor networks and advances in software technology (machine learning algorithms), fall detection systems can use various sensors, such as accelerometers, radar sensors, time-of-flight (ToF) sensors, and Wi-Fi nodes, to detect signal patterns characteristic of falls and thus determine whether a fall event has occurred.

[0003] However, while current fall detection systems perform well under laboratory conditions, these systems still have problems producing reliable results when applied to real-life conditions. Fall detection algorithms are typically pre-trained on training datasets that primarily contain laboratory-simulated fall data and use only a small amount of available real-world fall data. To improve the accuracy of fall detection systems, pre-trained fall detection algorithms need to be refined (updated) for specific elderly care facilities and / or specific elderly behaviors to better detect fall detection corner cases. Retrospective verification of fall time and type (retrospective labeling of fall events) is necessary to refine fall detection algorithms to the specifics of elderly care facilities and / or elderly activity and motor movement. Due to privacy regulations and further practical reasons, labelling of fall events in real-life conditions needs to be done either by the older person (self-labelling) or by one or more caregivers (or care facility staff). Summary of the Invention [Problem to be solved by the invention]

[0004] The inventors have noticed that older adults often tend to underreport or intentionally lie about whether an event constituted a fall. This may be because, for example, the fall was caused by their own actions (e.g., getting up in the middle of the night to go to the bathroom alone without asking a caregiver for help), they fear losing their independent living status, or they are forgetful. Thus, the self-label provided by older adults to a fall detection system (e.g., via a user interface) may be inaccurate or even intentionally incorrect. Incorrect (inaccurate) labels can significantly impair the accuracy of a fall detection system. Underreporting fall events, i.e., when older adults intentionally label fall events as no-falls, can lead to an increased number of false negatives in a fall detection system that is capable of immediately reporting a fall. On the other hand, over-reporting of fall events (i.e., older adults self-labeling non-fall events as falls) can lead to an increased number of false positives, which can lead to costly and unnecessary actions by caregivers and / or nursing homes and / or hospitals associated with older adults at home.

[0005] It is therefore an object to provide a method for determining a more accurate label of a fall event. [Means for solving the problem]

[0006] According to a first aspect, the above object is achieved by a method for determining a label of a fall event. The method includes the steps of receiving signals from one or more sensors configured to measure signals indicative of user movement, analyzing the received signals using a fall detection algorithm to determine a label indicative of the fall event, initiating a first user interface interaction mode of a user interface, wherein in the first user interface interaction mode, the user interface is configured to receive a first input from the user indicating a self-label of the fall event, receiving the first input, and determining a level of mismatch between the self-label and the determined label. If the level of mismatch exceeds a threshold, the method includes switching the user interface to a second user interface interaction mode, wherein in the second user interface interaction mode, the user interface is configured to receive a second input from the user indicating contextual information about the fall event, receiving the second input, and updating the self-label of the fall event based on the received second input.

[0007] Signals from one or more (remote) sensors, such as radar sensors, time-of-flight (ToF) sensors, Wi-Fi® Doppler sensors, microphone sensors, etc., may be used to determine user movement characteristics (patterns) and / or audio patterns indicative of a fall. The patterns may include the fall event itself as well as patterns preceding and following the fall. A fall detection and / or prevention algorithm analyzes the received sensor signals to determine a label indicative of a fall event associated with the received signals. The fall detection algorithm may be trained to determine whether a fall has occurred based on the received signals. For example, the determined label may indicate whether the received signals include a fall event or not, or may indicate whether the received signals include a type of fall event, such as a fall with injury, a fall without injury, a soft fall, a stroke fall, a near-fall (an elderly person losing balance without falling to the floor), etc. To improve the accuracy of the fall detection algorithm for recognizing fall events and / or classifying specific types of fall events, the user may be asked to self-label the fall event. User feedback (self-label of the fall event) is particularly necessary for remote sensing modalities that have limited accuracy for determining fall events. In a first user interface interaction mode, the user provides a self-label of the fall event. The self-label may indicate whether a fall event occurred or did not occur, and may indicate the type of fall event, e.g., fall with injury, fall without injury, etc.

[0008] If the received signal data is labeled, it may be used to retrain (update) the fall detection algorithm in the future to better recognize motion and / or audio patterns characteristic of falls (or types of falls). However, the self-label of the fall event provided by the user may be intentionally erroneous and / or inaccurate, for example, for the reasons described above. The method includes determining a level of mismatch between the self-label provided by the user and the label of the fall event determined by the fall detection algorithm. If the level of mismatch exceeds a threshold, the method includes switching the user interface interaction mode to a second mode in which the user interface is configured to receive a second input indicating (relevant) contextual information regarding the fall event—i.e., contextual data related to the circumstances of the fall event—that can determine the veracity of the self-report of the fall event provided by the user and contextualize the fall event (provide a broader understanding of the fall event). For example, the contextual data regarding the fall event may include user behavior preceding the event labeled as a fall. In another example, the user may be triggered / requested to reconsider their self-label of the event. In a further example, the contextual data may include information by the user that corroborates their self-label. The user may have initially provided an incorrect and / or inaccurate self-label of the fall event. By being explicitly asked to provide more (contextual) information about the event, the user is triggered (requested) to reconsider and reconsider their initial self-labeling and provide an accurate (confident) label of the event. This results in improved labeling, which allows for more efficient updating (retraining) of the fall detection algorithm.

[0009] The second input may include verbal and / or non-verbal cues. The method may further include analyzing the verbal and / or non-verbal cues in the second input to determine a user intent score, the user intent score indicating the user's intent to deceive the fall detection system (for the self-label), and updating the self-label of the fall event based on the user intent score. For example, the user intent score may be a probability (likelihood) that the user provided an incorrect self-label. Various machine learning (ML) models and techniques may be used to determine whether a user's intent to deceive is present based on verbal and non-verbal cues present in the user's response as evidence of deception. For example, non-verbal parameters (cues) of multiple audible responses, such as pitch, duration patterns, energy, and linguistic parameters, including filled pauses such as "um" or "ah," may be used as inputs to a speech ML model to determine a user's deceptive or non-deceptive intent in generating the audible response. A natural language processing (NLP) model, such as a stylometry model, may be used to determine (classify) whether (a portion of) the text in a user's text response is deceptive or not based on linguistic (inconsistencies in the response in the second user input) and non-linguistic (linguistic) cues in the text response. In another example, visual features in a video response provided by a user in the second input may be used as inputs to ML models, such as support vector machines and logistic regression models, to determine a user's deceptive or non-deceptive intent in generating the video response. For example, by analyzing micro-expressions and eye movements that indicate deceptive behavior. By associating a user intent score with the self-label provided by the user, the self-label may be updated accordingly.For example, if the user intent score indicates that the user's intent is not to deceive, the self-label is updated according to the first user input. Alternatively, if the user intent score indicates that the user's intent is to deceive, the self-label is updated according to the context information. This further improves labeling, thereby enabling more efficient updating (retraining) of the fall detection algorithm. The method may further include storing the user intent score along with the updated self-label in a training set and updating the fall detection algorithm based on the training set. Updating the fall detection algorithm to account for uncertainty associated with the user self-label can improve the accuracy and robustness of the fall detection algorithm, particularly to fine-tune the fall detection / fall prevention system to the unique quirky behavior of elderly people and their specific room setup.

[0010] The method may further include receiving a further input indicating a physiological parameter of the user during the period in which the user provides the second input, and analyzing the further input to determine a user intent score based on the physiological parameter of the user. If a person lies, the physiological response during the answer may indicate a stress response that may result from lying. For example, a person who is lying may be more agitated and exhibit an increased heart rate and breathing rate, or may sweat more (which causes changes in skin conductivity). By analyzing the physiological response (parameter) of the user during the period in which the user provides the second input, a better estimation of the user's deceptive intent may be achieved.

[0011] The method may further include obtaining a historical (past) user intent score for the user and determining a (current) user intent score depending on the historical user intent score. A person who has been known to intentionally deceive about their self-label of a previous fall event may be more likely to provide a current inaccurate label of the fall event. Thus, by considering the historical user intent score, the current user intent score can be more accurately determined.

[0012] The step of receiving a second input indicating contextual information about the fall event includes receiving information about at least one of the user's behavior prior to the fall event, supporting evidence by the user regarding the fall event, location data of the fall event, time data of the fall event, the presence of an additional person (e.g., a nurse) at the time of the fall event, and light settings (intensity and light spectrum) at the time of the fall event and before the fall event. For example, although people may fall for various reasons and under various conditions, people are more likely to fall while walking or ascending / descending stairs. Most falls are caused by improper sit-to-stand transfers. Similarly, elderly people are more likely to fall upon waking up due to temporary muscle weakness and balance disorders. Thus, the user's behavior prior to the fall can be a good indicator of a fall event. The time and location of the fall contain important information that provides context to the fall event. For example, most falls among elderly people occur at night while going to the toilet. The presence of another person at the time of the fall may indicate that the elderly person likely did not attempt to walk to the toilet alone and therefore unlikely to have tripped and fallen. The lighting (brightness level) at the location of the fall may also contribute to the fall event. Similarly, the lighting (light intensity and spectrum) to which the elderly person was exposed during the day / hour before the fall may also contribute to the fall event. Research suggests that users exposed to circadian lighting (lighting settings designed to promote circadian health) experience a 40% reduction in fall rates. Requiring a user to provide supporting evidence regarding the fall may trigger the user to reconsider the self-label provided or may reveal contradictory answers indicating an inaccurate self-report by the user. Therefore, receiving contextual information (data) related to the fall event may enable a broader understanding of the fall event and trigger the elderly person to confirm or deny their initial self-label of the fall event. Furthermore, contextual information (data) may reveal contradictory answers regarding the fall sensing data.

[0013] In the second user interface interaction mode, the user interface may be configured to select a question setting from a group of default question settings and output the selected question setting to the user. For example, the group of default question settings may include questions such as "What did you do before the fall event?", "What is the location of the fall event?", "Are you injured?", etc. Outputting the selected question setting to the user may facilitate the user providing contextual information about the fall event.

[0014] Additionally and / or alternatively, the user interface may be configured to determine question formulations based on natural language processing (NLP) algorithms and output the determined question formulations to the user. A powerful new class of large-scale language models is enabling machines to generate text in natural human language. These large-scale language models can generate deductive (non-existent) follow-up questions to seniors in natural human language.

[0015] The user interface may be configured to determine a question setting based on the level of mismatch between the self-label and the determined label and output the determined question setting to the user. For example, follow-up questions (settings) may be customized based on the level of mismatch between the self-label and the label determined (by the fall detection algorithm). The level of mismatch between the self-label and the label determined by the algorithm may indicate the user's intention to deceive or not. The wording of the question style (e.g., friendly rather than confrontational) affects how people respond to the question. A harsh, confrontational question may lead to user dissatisfaction if the user's first input was truthful. However, if the user's first input was deceptive, such a question will encourage the user to provide an accurate self-label of the fall event. Thus, by optimizing the question style (selecting a question setting) based on the level of mismatch between the self-label and the determined label, a more accurate self-label may be determined.

[0016] Determining a label indicating a fall event may include determining a type of fall event, and the first user interface interaction mode may be initiated if the determined fall event (the fall itself and the activities preceding and following the fall) is of a new type not previously seen. Labeling fall events significantly improves the accuracy of fall detection. However, older adults may become frustrated when constantly asked to provide a self-label for a fall event. If the fall type is common to the user (e.g., a near fall without injury), there may be no need to probe the user to provide a self-label. However, if a previously unseen fall type is predicted by a fall detection algorithm, the fall detection algorithm may have low reliability in such predictions. Therefore, it is beneficial to initiate the first user interface interaction mode only upon the determination of a new type of fall (not previously seen by the older adult). Accurately understanding the context leading up to a fall event is also important for preventing future falls.

[0017] Alternatively, the second user interface interaction mode may be conditioned on whether the determined fall event is of a new type. If a previously unseen fall type is predicted by the fall detection algorithm, more contextual information may be needed to correctly update the fall event label. Therefore, it is beneficial to initiate the second user interface interaction mode only upon the determination of a new type of fall (unseen for this elderly person).

[0018] The method may further include receiving input indicating one or more characteristics of the user and determining a user interface input and / or output modality based on the one or more characteristics of the user. For example, the one or more characteristics of the user may include a health condition, and an audio output modality via an audio assistant device or a virtual reality device may be used for a visually impaired user. In another example, the one or more characteristics of the user may include a living condition. A voice input modality with speech recognition may be used for a user living alone, and a keyboard input modality may be used for a user living in a shared facility. Adjusting the user interface input and / or output modality based on the user's characteristics and preferences enables better user engagement with the user interface.

[0019] The method may further include receiving input indicating one or more characteristics of the user; determining a time period for switching the user interface to the second user interface interaction mode, the time period based on the one or more characteristics of the user and / or the user intent score; and switching the user interface to the second user interface interaction mode after the determined time period. The one or more characteristics of the user may include the user's psychological or physiological state. For example, a user with dementia or memory problems may be more prone to forgetting details about the fall event long after the fall event. Thus, it may be beneficial for the user to initiate the second user interface mode immediately after the method determines that the level of mismatch exceeds a threshold. On the other hand, it may be beneficial for some users to be prompted to provide contextual information about the fall event at a later time (e.g., when the user is less stressed / worried about a possible fall event or less likely to be upset by being queried about a false positive fall). Thus, it is beneficial to adapt the time period for switching the user interface to the second user interface interaction mode based on the characteristics of the user.

[0020] The method may further include obtaining data indicative of the user's psychological and / or physiological state, and determining the level of mismatch depending on the user's psychological and / or physiological state. Certain diseases (e.g., Parkinson's disease, people with a history of stroke) and injuries have been shown to be strongly associated with falls. Furthermore, people suffering from dementia and / or memory problems are more likely to inadvertently label fall events inaccurately. By obtaining data indicative of the user's psychological and / or physiological state, the level of mismatch can be more accurately determined.

[0021] The method may further include determining whether the fall detection algorithm provided a false positive and / or false negative indication if there is a discrepancy between the label determined by the fall detection algorithm and the updated self-label; storing the received signals with the false positive and / or false negative indication in a training set; and updating the fall detection algorithm based on the training set. The updated self-label provides a more accurate indication of whether a fall actually occurred and / or an updated, more accurate labeling of the type of fall. In this manner, the fall detection algorithm can be updated to reduce the incidence of false positives and false negatives. This allows the fall detection algorithm to be adapted to the characteristics of a particular user's falls or activity, thereby improving the overall accuracy of the fall detection algorithm. Retraining may be performed for a particular elderly person and / or a particular room layout. Retraining may utilize single-shot or few-shot learning.

[0022] According to a second aspect, the object is a controller for determining a label for a fall event, the controller comprising: - receiving signals from one or more sensors configured to measure signals indicative of a user's movements; - analyzing the received signals using a fall detection algorithm to determine a label indicative of a fall event by the user; - initiating a first user interface interaction mode of the user interface, wherein in the first user interface interaction mode, the user interface is configured to receive a first input from the user indicating a self-label of the fall event; - receiving a first input, - determining the level of mismatch between the self-label and the determined label; - if the level of mismatch is above a threshold, switching the user interface to a second user interface interaction mode, wherein in the second user interface interaction mode, the user interface is configured to receive a second input from the user indicating contextual information regarding the fall event; - receiving a second input; and - updating the self-label of the fall event based on the second input received; This is achieved by a controller configured to:

[0023] According to a third aspect, the object is a system for determining a label of a fall event, the system comprising: - one or more sensors configured to measure signals characteristic of a user's movements; - a controller as described above; This is achieved by a system including:

[0024] According to a fourth aspect, the above object is achieved by a computer program for a computing device, the computer program comprising computer program code for performing the method for determining a label of a fall event when the computer program is executed on a processing unit of the computing device.

[0025] It will be appreciated that the controller, system and computer program may have similar and / or identical embodiments and advantages as the methods described above. [Brief explanation of the drawings]

[0026] The above and additional objects, features, and advantages of the disclosed systems, devices, and methods will be better understood through the following illustrative and non-limiting detailed description of embodiments of the devices and methods, with reference to the accompanying drawings. All figures are schematic and not necessarily to scale, and generally show only those parts necessary to clarify the invention; other parts may be omitted or merely suggested. [Figure 1] 1 illustrates a schematic diagram of an example system for determining a label for a fall event. [Figure 2] 1 illustrates a schematic diagram of an example of a user interface on a personal device. [Figure 3] 1 illustrates schematically a method for determining a label for a fall event. DETAILED DESCRIPTION OF THE INVENTION

[0027] FIG. 1 illustrates an example of a system 100 for determining a label of a fall event. The system 100 includes one or more sensors 102, 104 configured to measure signals 41, 42 indicative of a user's movement characteristics. The one or more sensors 102, 104 may be, for example, radar sensors, Wi-Fi nodes, infrared (IR) sensors, acoustic sensors, and / or other sensors. In one example, the one or more sensors 102, 104 may be co-located with a lighting device (not shown). The signals 41, 42 from the one or more sensors 102, 104, possibly after some processing, form a feature set. Exemplary features may include magnitude, spectral content, directional distribution, mean, variance, etc. Alternatively, the signal itself, i.e., a time series of sample values, e.g., a time series of channel state information (CSI) values ​​in a Wi-Fi signal, can serve as the feature set. For example, different motions and positions introduce different multipath distortions to the Wi-Fi signal, generating different patterns in the channel state information (CSI) time series. Thus, the channel state information (CSI) time series (signals 41, 42) from the Wi-Fi nodes may be used to determine patterns characteristic of the user's movements during a fall.

[0028] The system 100 further includes at least one data processor or controller 106. The controller 106 may be configured to receive signals 41, 42 from one or more sensors 102, 104. The controller 106 may connect to and communicate with each sensor 102, 104 via a wireless connection, for example, via a radio frequency or optical communication link. For example, a Wi-Fi, ZigBee, BLE, Lo-Ra, UWB, VLC, IR, Li-Fi, etc. connection. Alternatively, the connection may be wired. Each sensor 102, 104 may include a transmitter (not shown) for transmitting at least a subset of the respective signals 41, 42, or extracted features, to the controller 106 via a wired or wireless connection. The controller 106 may include a receiver (not shown) for receiving the respective signals 41, 42, or extracted features, from each sensor 102, 104. The system 100 may further include at least one data repository or storage or memory 108 for storing computer program code instructions. The controller 106 may be communicatively coupled to a cloud 120. Alternatively, the system 100 may include a server. Each sensor 102, 104 may communicate its respective signal 41, 42 or extracted feature to the server (or cloud) so that the server may obtain the respective signal 41, 42 or extracted feature. In this case, the controller 106 may be configured to obtain (receive) the respective signal 41, 42 or extracted feature from the server.

[0029] The controller 106 may be configured to analyze the received signals 41, 42 or extracted features using a fall detection and / or prevention algorithm to determine a label indicative of a fall event. For example, the determined label may indicate whether the received signals 41, 42 or extracted features include a fall event or do not include a fall event, or may indicate that the received signals 41, 42 include a type of fall event, e.g., "fall with an injury," "fall without an injury," etc. Additional fall event types may include a “trip and fall” event (i.e., a high-velocity fall to the ground from a walking position), a “fall entering a chair” event, a “soft fall” event (i.e., a user grips furniture to slow a fall to the ground), a “brain stroke fall” event (i.e., a fall from a standing position first onto one knee and then to the ground), a “pick-from-ground fall” event (i.e., a prolonged fall that occurs when a user attempts to pick something up from the ground), etc. A trained fall detection and / or prevention algorithm may make such a determination because it has already been trained on inputs including instances or segments (time series data) (or extracted features) of signals 41, 42 received from sensors 102, 104 and may output corresponding labeled instances of fall and / or non-fall incidents (events) and / or types of incidents (fall events). In particular, the fall detection algorithm may determine whether a fall event or type of fall event has occurred by comparing the signals 41, 42, or an extracted feature set, to a set of parameters used to classify whether a fall (or type of fall) has occurred. These parameters may include or be based on a feature set from known (types of) falls, e.g., from a training set.

[0030] 2 illustrates an example of a user interface 230 on a personal device 240. The controller 106 is configured to initiate a first user interface interaction mode of the user interface 230, and in the first user interface interaction mode, the user interface 230 is configured to receive a first input 10 from the user 220 indicating a self-label of a fall event. The self-label may indicate whether a fall event occurred or did not occur and may indicate the type of fall event, e.g., a fall with injury, a fall without injury, etc. The user 220 may be prompted via the user interface 230 on the personal device 240 to provide a text and / or voice self-label of the fall event and / or activity preceding the fall event. In one example, the personal device 240 may include a voice assistant device, and the user 220 may be prompted by the user interface 230 on the personal voice assistant device 240 to provide a text and / or voice self-label of the fall event. In yet another example, the personal device 240 may include a virtual or augmented reality device, e.g., a virtual reality headset, and the user 220 may be presented with a fall event via the virtual or augmented reality device 240 and asked to provide a text and / or voice self-label of the fall event. In a further example, the user 220 may press a button, e.g., on a wearable device, to affirm / negate the label of the fall event. The button may be an alarm reset button that is associated with an alarm signal that is generated if the fall detection algorithm determines a fall. If the user presses the alarm reset button within a predetermined timeout period (which may be zero), the event is labeled as a non-fall. Otherwise, if no alarm reset signal is received within the timeout period, the label is a fall.

[0031] The controller 106 may further be configured to receive the first input 10. For example, the controller 106 may connect to and communicate with the user sensor 230 via a wireless connection, e.g., via a radio frequency or optical communication link. The connection may alternatively be wired. The controller 106 may be included in the same device 240 as the user interface 230. The device 240 may include a transmitter (not shown) for transmitting the first user input 10 to the controller 106 via a wired or wireless connection. The controller 106 may include a receiver (not shown) for receiving the first user input 10. Alternatively, the device 240 may communicate the first user input 10 to a server (or cloud 120), and the controller 106 may be configured to subsequently retrieve (receive) the first user input 10 from the server.

[0032] The controller 106 may be further configured to determine a level of mismatch between the self-label by the user 220 and the label determined by the fall detection and / or fall prevention algorithm. For example, the controller 106 may apply a weighted average algorithm to the labels (by the user and by the fall detection algorithm) to determine the level of mismatch. For example, if the fall detection algorithm predicts a 60% probability of a fall and the user self-label indicates no fall (0% probability of fall), the level of mismatch may be determined (by the controller 106) to be 30%, assuming equal weights for the fall detection algorithm and the user self-label. In another example, the determined level of mismatch may be determined to be 50%, assuming a higher weight for the label predicted by the fall detection algorithm. In a further example, the controller 106 may determine the level of mismatch by applying a confidence learning machine learning algorithm to the labels. Such confidence-based models for characterizing noisy labels and identifying mismatches between labels relating to the same event are known in the field of supervised learning and will not be discussed further in the context of this application.

[0033] If the level of mismatch exceeds a threshold, the controller 106 may be configured to switch the user interface 230 to a second user interface interaction mode, in which the user interface 230 may be configured to receive a second input 20 from the user 220 indicating contextual information about the fall event. The contextual data provided as the second input 20 about the fall event may include the user's actions preceding the event labeled as a fall. In another example, the user may be triggered / requested to reconsider their self-label of the event. This may be done, for example, by telling the user, "75% of the user's asked to clarify a trip and fall self-declaration refined their answer after receiving additional information." In a further example, the contextual data may include information by the user that corroborates their self-label. In another example, the contextual data may include location data of the fall event, e.g., GPS location data from a sensor device attached to the user 220, contextual location data from the user 230, e.g., that the location of the fall event was in the kitchen, bathroom, living room, etc. In yet another example, the contextual data may include time data, e.g., time data received by a sensor attached to the user 230 and / or time data received from the user 230. Additionally, the contextual data may include lighting settings (spectrum and / or intensity) during or before the fall. For example, the user may provide information regarding whether they had lights on (off) before the fall event.

[0034] In the second user interface interaction mode, the controller 106 may be configured to select at least one question setting from a group of default question settings and output the one or more selected question settings to the user 220, for example, via the user interface 230 on the personal device 240. For example, the group of default question settings may include questions such as "What did you do before the fall event?", "What is the location of the fall event?", "Are you injured?", "What is the date?", "Are you sure that this is the correct label?", "What problems are there with your initial answer?", etc. The controller 106 may be configured to select one or more (or all) of the default (predetermined) question settings and output the selected question settings to the user 220. The default question settings may be stored in the memory 108 or the cloud 120.

[0035] Additionally and / or alternatively, the controller 106 may be configured to determine a question set in natural human language that is customized for a particular user, for example, by using a natural language processing algorithm. For example, the question set, e.g., a follow-up question, may be customized based on a second user input (e.g., contextual information about the fall), historical data about the user's previous fall event, user specifics, etc. For example, it may be known (e.g., from a caregiver or from camera footage) that a first dementia patient likes to play with extension cords on the floor and may become dizzy if they have to bend over for a long time to reach the cord. However, elderly people already know that self-lowering is wrong and therefore often initially, even vehemently, deny it if they are found to have (re)lowered on the floor. The question set may be customized for such cases of intentionally lowering on the floor. Methods and techniques for creating text in natural human language are known in the art and will not be described in detail in the context of this application.

[0036] The controller 106 may be further configured to determine a question setting based on the level of mismatch between the self-label and the determined label and output the determined question setting to the user. The controller 106 may be configured, for example, to select a more aggressive style of question setting, such as "What problems are there with your initial answer?", when the level of mismatch between the self-label and the determined label is high (above a threshold), and a more friendly style of question setting, such as "Are you sure that this is the correct label?", when the level of mismatch between the self-label and the determined label is moderate (below a threshold). The question setting may be selected from a group of default question settings categorized according to the level of mismatch and / or based on a conditional natural language processing algorithm in which the style of the question is conditioned based on the level of mismatch.

[0037] The controller 106 may be configured to update the self-label of the fall event based on the second input 20 received via the user interface 230. For example, the controller 106 may be configured to analyze the received contextual data, e.g., using a natural language processing algorithm (NLP), to determine an updated (more accurate) self-label.

[0038] The second input may include verbal and / or non-verbal cues. The controller 106 may be further configured to analyze the verbal and / or non-verbal cues in the second input to determine a user intent score and update the self-label of the fall event based on the user intent score. Various machine learning (ML) models and techniques may be used to determine the presence or absence of a user's deceptive intent based on verbal and non-verbal cues present in the user's response as evidence of deception. For example, non-verbal parameters of a plurality of audible responses, such as pitch, duration patterns, energy, and linguistic parameters, including speech parameters such as voiced pauses, such as "um" or "ah," may be used as inputs to a speech ML model to determine the user's deceptive or non-deceptive intent in generating the audible response. A natural language processing (NLP) model, such as a stylometry model, may be used to determine (classify) whether (a portion of) the text in a user's text response is deceptive or non-deceptive based on linguistic cues in the text response (inconsistencies in the response provided as a second input) and non-linguistic features such as word count, number of words longer than six letters, etc. (liars may use more abbreviated forms of language). For example, it is known that a deceptive language style includes fewer first-person singular pronouns, fewer third-person pronouns, fewer exclusive words, more negative emotion words, and more motion verbs. In another example, visual features in a user's video response may be used as input to an ML model, such as a support vector machine and a logistic regression model, to determine the user's deceptive or non-deceptive intention in generating the video response. For example, by analyzing micro-expressions and eye movements that indicate deceptive behavior. Such ML models and techniques for analyzing text, voice, or video content to detect deception are known in the art and will not be described in detail within the context of this application. The controller 106 may be configured to update a self-label of the fall event based on the user intent score.For example, if the user intent score indicates that the user's intent is not to deceive, e.g., if the deceptive user intent score is below a threshold, e.g., 50%, the self-label is updated according to the first user input. Controller 106 may be further configured to store the user intent score along with the updated self-label in a training set and update the fall detection algorithm based on the training set. The training set may be stored in memory 108 and / or cloud 120. The stored information may be used, for example, to adapt or retrain the algorithm, e.g., adjust the algorithm's loss function to reflect the user intent score in the updated training dataset.

[0039] The controller 106 may be further configured to obtain a historical user intent score for the user (e.g., the user may have provided answers of questionable veracity in the past) and determine a current user intent score based on the historical score. For example, the controller 106 may determine the current user intent score as a weighted average of the user's historical user intent score and the current user intent score. In one example, the weights may be equal. Alternatively, the controller 106 may determine the current user intent score by assigning a higher weight to the historical user intent score.

[0040] The controller 106 may be further configured to obtain data indicative of a physiological parameter of the user and to determine a user intent score further based on the physiological parameter of the user. For example, the controller 106 may be configured to receive input from one or more sensors monitoring the user's physiological parameters during the period in which the user provides the second input 20, such as an electrocardiography (ECG) sensor, a photoplethysmography (PPG) sensor monitoring the user's heart rate, a radar sensor monitoring the user's heart rate and / or respiratory rate, etc. This input may be used as an input to an ML model to determine whether the user intends to deceive. The ML model may make such a determination by being trained to detect deception using known instances of sensor signals associated with deception in a training set.

[0041] The controller 106 may be configured to determine the type of fall event, e.g., "fall with injury," "fall into chair," "soft fall," etc., by analyzing the received signals 41, 42 or features extracted from the received signals. The controller 106 may be configured to initiate the user interface 230 according to the first user interface interaction mode upon determining that the fall event is of a new type. That is, the controller 106 may initiate the first user interface interaction mode on the condition that the fall detection algorithm determines a type of fall not previously seen for this user. In another example, the controller 106 may be configured to switch the user interface 230 to the second user interface interaction mode only if the fall detection algorithm determines a new type of fall event.

[0042] The controller 106 may further be configured to receive input indicating one or more characteristics of the user and determine a user interface input and / or output modality (unimodal or multimodal) based on the one or more characteristics of the user. The input may include the user's medical health record indicating the user's physiological and / or psychological state, the user's living conditions, signals from one or more sensors monitoring the user, etc. The one or more user interface output modalities may include visual (computer graphics via a screen), audio, vibration, etc. For example, the one or more characteristics of the user may include the user's physiological and / or psychological state. An audio output modality via an audio assistant device or a virtual reality device may be used for a visually impaired user. In another example, a visual output modality via a screen on a personal device may be used for a hearing impaired user, etc. In another example, the one or more characteristics of the user may include the user's stress state (e.g., based on heart rate monitoring). A visual output modality via a screen on a personal device may be used for a user who is under stress, compared to an audio assistant device (which may contribute to an increased stress level for the user). The one or more user interface input modalities may include keyboard input, pointing device, touch screen, and / or more complex modalities such as computer vision, speech recognition, motion, orientation, etc. For example, the one or more characteristics of the user may include a living situation. A voice input modality with speech recognition may be used for a user living alone, while a keyboard input modality may be used for a user living in a shared facility.

[0043] The controller 106 may be further configured to determine a time period for switching the user interface 230 to the second user interface interaction mode and to switch the user interface 230 to the second user interface interaction mode after the time period. The time period may be based on one or more characteristics of the user. For example, the controller 106 may determine a shorter(er) time period for a user with a medical condition related to forgetfulness. In a further example, the controller 106 may determine a slower(er) time period for a user currently experiencing psychological and / or physiological conditions related to stress.

[0044] FIG. 3 illustrates a method 300 for determining a label for a fall event, the method comprising: - receiving (302) signals from one or more sensors 102, 104 configured to measure signals 41, 42 characteristic of the movement of a user 220; - analyzing (304) the received signals using a fall detection algorithm to determine a label indicative of a fall event by the user; - initiating (306) a first user interface interaction mode of the user interface, wherein in the first user interface interaction mode the user interface is configured to receive a first input from the user indicating a self-label of the fall event; - receiving a first input (308); - determining (310) the level of mismatch between the self-label and the determined label; - if the level of mismatch is above a threshold, switching (312) the user interface to a second user interface interaction mode, in which the user interface is configured to receive a second input from the user indicating contextual information regarding the fall event; - receiving a second input (314); - updating (316) a self-label of the fall event based on the received second input; 1 illustrates an example of a method including:

[0045] The method 300 may be performed by computer program code of a computer program when the computer program is executed on a processing unit of a computing device, such as the controller 106 .

[0046] In one example, method 300 may further include optional step 318 of determining whether the fall detection algorithm provided a false positive and / or false negative indication if there is a discrepancy between the label determined by the fall detection algorithm and the updated self-label, optional step 320 of storing the received signals 41, 42 with the false positive and / or false negative indication in a training set, and optional step 322 of updating the fall detection algorithm based on the training set. During operation of the fall detection and / or prevention algorithm, instances of the received signals 41, 42 or features extracted from the received signals 41, 42 may be stored in a training set in memory 108 and / or cloud 120 to update the algorithm. The algorithm may use the (updated) training set to compare current signals / features with those in the (updated) training set to determine the current label of the event. To improve the training of the algorithm, instances of the signals / features may be stored along with a value indication of the algorithm's performance. For example, if the label determined by the fall detection algorithm indicates a fall but the updated self-label indicates a no-fall, the received signals 41, 42 or extracted features may be labeled to represent a false positive (FP) and stored in the training set with the FP indication. If the label determined by the fall detection algorithm does not indicate a fall but the updated self-label indicates a fall, the received signals 41, 42 or extracted features may be labeled to represent a false negative (FN) and stored in the training set with the FN indication. Signals 41, 42 and feature sets whose updated self-labels are the same as the labels determined by the algorithm may be stored as either TP (true positive) or TN (true negative), respectively.The stored information may be used, for example, to adapt or train algorithms to reduce the rate of false positives and false negatives.

[0047] It should be noted that the above-described embodiments are illustrative rather than limiting of the present invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.

[0048] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "to comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The singular reference of an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer or processing unit. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain means are recited in mutually different dependent claims does not indicate that a combination of these means cannot be used to advantage.

[0049] Aspects of the present invention may be embodied in a computer program product, which may be a collection of computer program instructions stored on a computer-readable storage device that can be executed by a computer. The instructions of the present invention may be any interpretable or executable code mechanism, including, but not limited to, a script, an interpretable program, a dynamic link library (DLL), or a Java class. The instructions may be provided as a complete executable program, a partial executable program, a modification (e.g., an update) to an existing program, or an extension (e.g., a plug-in) to an existing program. Furthermore, portions of the processing of the present invention may be distributed across multiple computers or processors, or a "cloud."

[0050] Suitable storage media for storing computer program instructions include all forms of non-volatile memory, including, but not limited to, EPROM, EEPROM, and flash memory devices, magnetic disks such as internal and external hard disk drives, removable disks, and CD-ROM disks. The computer program may be distributed on such storage media or may be made available for download via HTTP, FTP, email, or a server connected to a network such as the Internet.

Claims

1. 1. A method for determining a label of a fall event, the method comprising: receiving signals from one or more sensors configured to measure signals indicative of a user's movement; analyzing the received signals using a fall detection algorithm to determine a label indicative of a fall event by the user; initiating a first user interface interaction mode of a user interface, wherein in the first user interface interaction mode, the user interface is configured to receive a first input from the user indicating a self-label of the fall event; receiving the first input; determining a level of mismatch between the self-label and the determined label; if the level of mismatch exceeds a threshold, switching the user interface to a second user interface interaction mode, wherein in the second user interface interaction mode, the user interface is configured to receive a second input from the user indicating contextual information regarding the fall event; receiving the second input; updating the self-label of the fall event based on the received second input; A method comprising:

2. The second input includes verbal and / or non-verbal cues, and the method further comprises: analyzing the verbal and / or non-verbal cues to determine a user intent score, the user intent score indicating the user's intent to deceive; updating the self-label of the fall event based on the user intent score; The method of claim 1 , comprising:

3. The method comprises: receiving a further input indicative of a physiological parameter of the user; analyzing the further input to determine the user intent score based on the physiological parameter of the user; The method of claim 2 , comprising:

4. The method comprises: obtaining a historical user intent score for the user; determining the user intent score based on the historical user intent score of the user; The method of claim 2 , comprising:

5. 5. The method of claim 1, wherein receiving a second input indicating contextual information related to the fall event comprises receiving information related to one of user behavior prior to the fall event, supporting evidence related to the fall event, time data, location data, the presence of additional people at the time of the fall event, and light settings at the time of and before the fall event.

6. 6. The method of claim 1, wherein in the second user interface interaction mode, the user interface is configured to select a question setting from a group of default question settings and output the selected question setting to the user.

7. 7. The method of claim 1, wherein in the second user interface interaction mode, the user interface is configured to determine a question formulation based on a natural language processing algorithm and output the determined question formulation to the user.

8. 8. The method of claim 1, wherein in the second user interface interaction mode, the user interface is configured to determine a question configuration based on the level of mismatch between the self-label and the determined label and to output the determined question configuration to the user.

9. 9. The method of claim 1, wherein determining a label indicative of the fall event includes determining a type of the fall event, and wherein the first user interface interaction mode is initiated if the determined fall event is of a new type.

10. 10. The method of claim 1, wherein determining a label indicative of the fall event includes determining a type of the fall event, and wherein the second user interface interaction mode is conditioned on whether the determined fall event is of a new type.

11. 11. The method of claim 1, comprising receiving input indicative of one or more characteristics of the user, and determining a user interface input and / or output modality based on the one or more characteristics of the user.

12. The method comprises: receiving input indicative of one or more characteristics of the user; determining a time period for switching the user interface to the second user interface interaction mode, the time period being based on a user intent score of the one or more characteristics of the user and / or the self-label; switching the user interface to the second user interface interaction mode after the determined period of time; 3. The method of claim 1 or 2, comprising:

13. A controller for determining a label of a fall event, the controller comprising: receiving signals from one or more sensors configured to measure signals indicative of a movement characteristic of the user; analyzing the received signals using a fall detection algorithm to determine a label indicative of a fall event by the user; initiate a first user interface interaction mode of a user interface, wherein in the first user interface interaction mode, the user interface is configured to receive a first input from the user indicating a self-label of the fall event; receiving the first input; determining a level of mismatch between the self-label and the determined label; if the level of mismatch exceeds a threshold, switching the user interface to a second user interface interaction mode, wherein in the second user interface interaction mode, the user interface is configured to receive a second input from the user indicating contextual information regarding the fall event. receiving the second input; and updating the self-label of the fall event based on the received second input; The controller is configured as follows:

14. 1. A system for determining a label of a fall event, the system comprising: one or more sensors configured to measure signals characteristic of a user's movements; A controller according to claim 13; Including, the system.

15. 13. A computer program for a computing device comprising computer program code for performing the method of any one of claims 1 to 12 when the computer program is executed on a processing unit of the computing device.

Images