Timely detection and response to context-specific health events
By using multiple sensors in wearable devices to build context profiles and compare them with baseline profiles, automatically identifying and responding to adverse health events, the problem of inaccurate detection and response in the prior art is solved, and fast and timely emergency response is achieved .
Patent Information
- Application Number
- CN202380035374.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-25
- Filing Date
- 2023-03-09
- Publication Date
- 2024-11-22
AI Technical Summary
The prior art is difficult to accurately detect and respond to individual adverse health events, especially when individuals are alone or unable to communicate effectively, resulting in extended response times that affect survival and recovery prospects.
By using signals generated by multiple sensors in a wearable device, the user's current context profile is constructed and compared with the baseline profile of the same context, possible adverse health events are identified and automated remediation actions are initiated.
It improves the detection accuracy of adverse health events, reduces false alarms, ensures timely response in emergencies, and improves individual survival and recovery opportunities.
Smart Images

Figure CN119012958A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to detecting and responding to abnormal health events, and more particularly, to determining the severity of adverse health events in real time and providing responses thereto. Background Art
[0002] An adverse health event may be characterized by a sudden change in an individual's physiological condition or state. Such a change may be caused, for example, by a cardiovascular event such as cardiac arrest, a neurological event such as a stroke or epilepsy, an accident such as a fall, a respiratory event, or a variety of other health-related events. Such events, regardless of their underlying cause, often involve medical emergencies, where response time is critical to an individual's survivability and prospects for recovery following the sudden onset of such an event. Unfortunately, in many cases, an individual experiencing a medical emergency may be alone or unable to communicate effectively. In such circumstances, the individual may be unable to inform anyone of the nature of the adverse health event or request emergency assistance. Summary of the invention
Technical issues
[0003] One or more embodiments herein provide methods and systems for initiating remedial action based on a user's current contextual profile.
Technical solution
[0004] In an example implementation, an automated health event and detection method may include updating a current contextual profile of a user based on signals generated by one or more sensors of a wearable device, the signals being generated in response to one or more physically measurable phenomena associated with the user. The method may include selecting a context-aware baseline profile having the same context as the current contextual profile, the context-aware baseline profile being selected from a plurality of context-aware baseline profiles having different contexts. The method may include comparing features of the current contextual profile with corresponding features of a context-aware baseline profile of the user having the same context as the current contextual profile. The method may include initiating remedial action in response to identifying an occurrence of a possible adverse health event affecting the user based on the comparison.
[0005] In another example implementation, a system may include one or more processors configured to initiate operations. The operations may include updating a current contextual profile of a user based on signals generated by one or more sensors, the signals being generated in response to one or more physically measurable phenomena associated with the user. The operations may include selecting a context-aware baseline profile having the same context as the current contextual profile, the context-aware baseline profile being selected from a plurality of context-aware baseline profiles having different contexts. The operations may include comparing features of the current contextual profile with corresponding features of a context-aware baseline profile of the user having the same context as the current contextual profile. The operations may include initiating remedial action in response to identifying an occurrence of a possible adverse health event affecting the user based on the comparison.
[0006] In yet another example implementation, a computer program product includes one or more computer-readable storage media and program instructions stored together on the one or more computer-readable storage media. These program instructions can be executed by computer hardware to initiate operations. These operations may include: updating a user's current context profile based on signals generated by one or more sensors, which are generated in response to one or more physically measurable phenomena associated with the user. These operations may include: selecting a context-aware baseline profile having the same context as the current context profile, wherein the context-aware baseline profile is selected from a plurality of context-aware baseline profiles having different contexts. These operations may include: comparing features of the current context profile with corresponding features of a user's context-aware baseline profile having the same context as the current context profile. These operations may include: initiating remedial action in response to identifying the occurrence of a possible adverse health event affecting the user based on the comparison.
[0007] This summary is provided only to introduce particular concepts and does not identify any key or essential features of the individual claimed matters. Other features of the arrangements of the present invention will become apparent from the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The arrangement of the present invention is shown by way of example in the accompanying drawings. However, the accompanying drawings should not be interpreted as limiting the arrangement of the present invention to only the specific implementations shown. Various aspects and advantages will become clear by reading the following detailed description and referring to the accompanying drawings.
[0009] Figure 1 An example adverse health event detection and emergency response system is shown.
[0010] Figure 2 Shown by Figure 1An example current context profile generated by the system.
[0011] Figure 3 Shown by Figure 1 An example collection of context-aware baseline profiles generated by the system.
[0012] Figure 4 Schematically shows the Figure 1 Example of context-based anomaly detection performed by the system.
[0013] Figure 5 Schematically shows the Figure 1 Example of emergency scoring performed by the system.
[0014] Figure 6 Schematically shows the Figure 1 An example of a user responsiveness determination performed by a system.
[0015] Figure 7 Shown by Figure 1 An example rule set for an implementation of a context-aware privacy manager for a system.
[0016] Figure 8 Shows Figure 1 certain operational aspects of the system.
[0017] Fig. 9 Shows Figure 1 certain operational aspects of the system.
[0018] Fig.10 Shows Figure 1 certain operational aspects of the system.
[0019] Fig.11A and Fig. 11B Example methods of adverse health event detection and emergency response are shown.
[0020] Fig.12 Example methods of adverse health event detection and emergency response are shown.
[0021] Fig.13 Shown is the method for implementing Figure 1 An example electronic device of a system. DETAILED DESCRIPTION
[0022] Although the present disclosure concludes with claims defining novel features, it is believed that the various features described herein will be better understood from consideration of the specification in conjunction with the accompanying drawings. The processes, machines, manufactures, and any variations thereof described in the present disclosure are provided for illustrative purposes. Any specific structural and functional details described should not be construed as limiting, but merely as a basis for the claims and as a representative basis for teaching those skilled in the art to variously employ the features described in almost any appropriately detailed structure. Furthermore, the terms and phrases used in the present disclosure are not intended to be limiting, but rather to provide an understandable description of the features described.
[0023] The present disclosure relates to detecting and responding to abnormal health events, and more specifically, to determining the severity of adverse health events and providing responses thereto. In certain aspects, adverse health events and / or determination of their severity can be performed in real time. As already described, adverse health events often occur when the person suffering from the event is alone. In addition, adverse health events may prevent a person from communicating effectively, thereby being unable to explain the nature of the event or even to seek assistance. This may be particularly problematic because many elderly people, who are generally more likely to suffer adverse health events than others, often live alone.
[0024] One possible solution is a device that can detect signs of an adverse health event and automatically respond by initiating a call for emergency assistance. A device with one or more sensors that generate signals for measuring the vital signs of the device user (e.g., heart rate, respiratory rate, and other physiological measurements) can detect possible adverse health events. Given the unpredictable timing of such events, the device would need to be with the user at all times - either worn by the user or close enough to the user to detect when the user suffers an adverse event. Therefore, ideally, the device is portable.
[0025] In order to be a reliable indicator of an adverse health event, the device must be able to distinguish between normal vital sign values and abnormal vital sign values, thereby indicating an adverse health event. One challenge is that an abnormal measurement in one context may be normal in another context. An individual's vital signs are highly dependent on the context, namely the individual's activities, environment, and other contextual factors that can affect the individual's physiology at a given moment. For example, if an individual's heart rate exceeds 120bpm when resting, the elevated heart rate may well indicate cardiac discomfort or other emergency health events. If the same person's heart rate exceeds 120bpm when performing strenuous activity (e.g., exercising), the same heart rate may be normal. The challenge is that a vital sign measurement that is abnormal for one person may be normal for another. Individual physiological differences may arise from differences in age, gender, underlying medical history, or similar contextual parameters that have little to do with adverse health events.
[0026] According to the inventive arrangement described herein, an example method, system, and computer program product are provided that can automatically determine in real time a possible adverse health event of an individual and initiate remedial action in response thereto. One aspect of the disclosed inventive arrangement is to determine possible adverse health events with improved accuracy based on comparing a user's current context-aware health profile (e.g., a matrix) with a user-specific context-aware baseline profile having the same context. As defined herein, a "context" is a value set of multiple context parameters that affect a user's physiological condition or state. Relatedly, a "context parameter" as defined herein is a measurable or observable variable corresponding to a physical factor or phenomenon that can affect a user's physiology. Such factors may include, for example, the current activity performed by the user, the user's current environment, the user's underlying medical history, and other factors (including demographic factors such as the user's gender and age). For example, a context may include a location context (e.g., indoors or outdoors), an activity context (e.g., sitting, sleeping, exercising, climbing stairs), and an environmental context (e.g., ambient temperature, humidity, altitude).
[0027] Thus, this aspect of the arrangement of the present invention allows for more accurate detection of adverse health events by considering the user's current vital signs in the specific context of measuring the vital signs by one or more sensors. The user's current context-aware vital signs are compared to baseline measurements that reflect the same context as the user's current vital signs, thereby improving detection accuracy.
[0028] In some arrangements, the disclosed inventive arrangements are implemented in a wearable device (e.g., a pair of earbuds, a smart watch, smart glasses (including a virtual, augmented, or mixed reality head mounted display (HMD) device), a chest strap, or the like). In other arrangements, the inventive arrangements are implemented in multiple devices operably coupled to each other. One device (e.g., a wearable device with sensing capabilities) can be operably coupled to another device (e.g., a smartphone, tablet, or laptop with greater processing power and additional sensing capabilities).
[0029] Devices implementing the arrangements of the present invention and / or associated devices operably coupled thereto may include a plurality of sensors. The sensors may include physiological sensors, geographic location and environmental sensors, motion sensors, and multimedia sensors (e.g., microphones, cameras, eye movement trackers). The physiological sensors may generate signals from which various physiological data (e.g., a user's heart rate, respiratory rate, and blood oxygen saturation) may be derived.
[0030] One aspect of the inventive arrangements disclosed herein is the implementation of one or more machine learning models capable of classifying a user's motion and physical condition based on data derived from signals generated by motion sensors and signals captured by multimedia sensors. For example, machine learning classification of signals generated by an inertial measurement unit (IMU) or other micro-electromechanical (MEM) motion sensor can detect a fall of the user. For example, machine learning classification of signals generated by an eye tracking unit can detect an unconscious state of the user.
[0031] One of the advantages of the contextual information provided by such sensors and machine learning models is an enhanced ability to accurately determine the nature of an adverse health event. As described herein, such information can be used as part of an automated response provided by the disclosed arrangement that enables a health professional or other assistant to more accurately assess and properly respond to the condition of a user experiencing an adverse health event.
[0032] Another advantage is the reduction of false positives without increasing processing overhead. Continuous monitoring can be performed using multiple lightweight sensors to monitor the user's vital signs, but such sensors are often susceptible to motion artifacts. If only lightweight sensors are used in detecting adverse health events, there is a high chance of false positives. Using multimedia sensors can provide more accurate results. However, considering the overhead incurred by using multimedia sensors, continuous monitoring using multimedia sensors may not be suitable for portable devices (e.g., earbuds, smart watches, smart glasses).
[0033] The arrangement of the present invention overcomes this dilemma by using multimedia sensors to improve detection accuracy, but only in response to a preliminary determination that there is a high probability of an emergency situation due to an adverse health event. If, based on continuous monitoring, there is less certainty about the severity of the possible adverse health event, the arrangement of the present invention initiates continuous and enhanced monitoring of the user's condition. Therefore, another aspect of the arrangement of the present invention is the selective use of multimedia sensors to improve detection accuracy while avoiding excessive consumption of resources of devices and sensors used to detect and respond to adverse health events.
[0034] Further aspects of the arrangements of the present invention will be described in more detail below with reference to the accompanying drawings. For the purpose of making the description simple and clear, the units shown in the accompanying drawings are not necessarily drawn to scale. For example, for the sake of clarity, the size of some units may be exaggerated relative to other units. In addition, if deemed appropriate, reference numerals are repeated between the drawings to indicate corresponding, similar or analogous features.
[0035] First reference Figure 1, depicts an example adverse health event detection and response system (system) 100. As an example illustration, the system 100 includes a context-aware anomaly determiner 102, an emergency event and user response (EE / UR) determiner 104, an intelligent responder 106, a vital sign monitor 108, an environmental monitor 110, a motion monitor 112, and a multimedia sensor processor 114. The multimedia sensor processor 114 includes, for example, both a multimedia data request processor 116 and a multimedia data processor 118.
[0036] In various arrangements, system 100 may be implemented in hardware (e.g., dedicated hardwired circuitry), software (e.g., program code executed by one or more processors), or a combination thereof. For example, in some embodiments, system 100 is implemented in a computer such as device 1300 ( Fig.13 ) is implemented in a device such as . Thus, in some arrangements, system 100 includes electronically stored in a memory (e.g., memory 1304) and executed on one or more processors (e.g., device 1300 ( Fig.13 The device may be a wearable device such as a health monitoring device worn by a user (e.g., a chest strap) or a multi-function device such as a smart watch, smart glasses, earbuds, or other devices including virtual reality, augmented reality, or mixed reality HMD devices.
[0037] The system 100 performs various functions based on context-aware data generated in real time by processing various signals as described below. The signals can be generated by various different types of sensors communicatively coupled to the vital signs monitor 108, the environmental monitor 110, the motion monitor 112, and the multimedia sensor processor 114. Therefore, in various arrangements, different elements of the system 100 can be implemented in different devices operably coupled to each other. For example, signals captured by sensors of a wearable device (e.g., earbuds, smart glasses) can be signal processed by a companion device (e.g., a smart phone, a smart watch, a cloud-based server) that is wirelessly operably coupled to the wearable device and has powerful processing capabilities.
[0038] The signals generated by the sensors processed by the system 100 are used by the system to construct a current contextual profile of the user. The current contextual profile provides a real-time indication of the user's physiological condition or state given a corresponding set of contextual factors, including the user's current activity and environment. The system 100 compares the current contextual profile with a baseline profile corresponding to the same context to identify any anomalies in the user's physiology and respond accordingly.
[0039] The vital signs monitor 108 processes signals generated by various sensors operably coupled to the system 100. The sensors include, for example, an inductive respiration sensor (RIP) 120, a pulse oximeter 122, an electrical skin and galvanic skin response (EDA / GSR) sensor 124, a photoplethysmography (PPG) sensor 126, and a thermometer 128. Different combinations of these and / or other sensors in different arrangements may be operably coupled to the system 100. Data derived from the signals generated by the sensors and processed by the vital signs monitor 108 may include, for example, the user's respiration rate, heart rate, blood oxygen saturation (SpO 2 ) and core body temperature. In other arrangements, the same and / or other data may be processed by the vital signs monitor 108, depending on the various sensors operably coupled to the system 100.
[0040] The user's physiological state or condition is one dimension of the overall context. The context may also include the user's activities as well as the user's location and environment. The activity, location, and environment may all affect the user's current physiological condition or state. For example, if the user is engaged in vigorous activity (e.g., exercising) rather than being sedentary, the user's breathing rate and heart rate are expected to be higher. Similarly, the user's core body temperature may be affected by whether the user is indoors or outdoors, on a hot humid day or a cold snowy day. By comparing the user's current contextual profile with a context-aware baseline profile corresponding to the same context, the accuracy of the system 100 in identifying physiological indicators that indicate abnormal or poor health conditions may be improved.
[0041] Similar to the vital signs monitor 108, the environment monitor 110 and the motion monitor 112 determine contextual parameters by processing signals generated by various other sensors operably coupled to the system 100. The geographic location and environmental data processed by the environment monitor 110 can be derived from sensors including, for example, a global positioning system (GPS) sensor 130, a thermometer 132, and a hygrometer 134. In other arrangements, combinations of these and / or different sensors can be operably coupled to the system 100. Based on the data derived from the signals generated by the various sensors, the environment monitor 110 can determine various contextual parameters that affect the physiological condition of the user. The contextual parameters include, for example, geographic location data, ambient temperature, air pressure, atmospheric pressure, and humidity for determining whether the user is indoors or outdoors. In other arrangements, depending on the various sensors operably coupled to the system 100, the same and / or other data can be processed by the environment monitor 110.
[0042] The motion monitor 112 processes the user motion detected by the motion detection sensor. Exemplarily, the motion detection sensor includes one or more micro-electromechanical system (MEMS) sensors 136. In different arrangements, the MEMS sensor 136 may include an accelerometer, a magnetometer, etc. In some arrangements, the MEMS sensor 136 includes an inertial measurement unit (IMU) that combines a MEMS accelerometer and a MEMS gyroscope. In some arrangements, the MEMS sensor 136 implemented as an IMU can be embedded in a wearable device (e.g., earplugs, smart watches, smart glasses, etc.). The motion monitor 112 can determine contextual parameters based on data derived from the signals generated by the sensor, such as motion intensity, the number of irregular movements or twitching movements, the maximum and minimum accelerometer and gyroscope values, the average intensity of motion associated with vigorous activity, estimated energy consumption, motion entropy, and other motion-related parameters.
[0043] In some arrangements, the motion monitor 112 implements a machine learning model, such as a classification model (e.g., a random forest). The motion monitor 112 generates a vector or a high-order tensor based on motion data derived from a signal generated by a sensor. The elements of the vector or high-order tensor include motion features. The machine learning model can be trained to classify the input vector or high-order tensor, and the classification indicates a specific activity or posture of the user. For example, a random forest classifier implemented by the motion monitor 112 can provide a fall detection mechanism by classifying certain motion features as unexpected events (such as falls) that may indicate a user. Although the prediction accuracy of the model exceeds 95% by some estimates, there is an often unacceptable incidence of false positives or false alarms. Therefore, as described below, the system 100 incorporates additional features for verifying the model's prediction of a user's fall or other adverse health events to prevent false alarms. More generally, the machine learning model implemented by the motion monitor 112 can identify the user's current activity by classifying real-time motion detected by the motion monitoring unit. The activity classification determined by the motion monitor 112 may include, for example, vigorous activity (e.g., strenuous exercise), brisk activity (e.g., climbing stairs), moderate activity (e.g., walking), and rest (e.g., sitting, lying down). In some embodiments, the machine learning model implemented by the motion monitor 112 may be trained to classify user-specific motion based on motion features captured over time by monitoring the user using various sensors of a device worn or carried by the user.
[0044] The context-aware anomaly determiner 102 generates a current context profile of the user by aggregating data derived from and received from the vital sign monitor 108, the environmental monitor 110, and the motion monitor 112. The current context profile provides a real-time quantitative snapshot of the user's physiological state or condition, current activity, and current environment. The context-aware profile is a data structure. In some arrangements, the data structure includes an array (e.g., a matrix or a high-order tensor) whose elements correspond to a series of physiological measurements or user vital signs (e.g., breathing rate, heart rate) and user motion captured by a sensor at specific intervals (e.g., 10 minute intervals) within a predetermined time range.
[0045] Additional references Figure 2 , depicting an example current context profile 200. Exemplarily, the current context profile 200 is a matrix having a set of rows including features corresponding to the user's vital signs, including respiratory rate (RR) and heart rate (HR). These rows are concatenated with rows of motion features, including acceleration (Acc) and gyroscope orientation (Gyr) features. The columns of the current context profile 200 correspond to time T 1 , T 2 …T K . Thus, illustratively, the context-aware profile 200 is a time series matrix whose elements correspond to a series of physiological measurements or user vital signs and user movements captured by sensors within a predetermined time range. The context-aware anomaly determiner 102 determines a specific context based on the user's current activity (e.g., determined by classifying a machine learning model of the user's movements) and based on the user's location and environment (e.g., determined based on the user's geographic location determined by GPS and other environmental parameters determined by the environmental monitor 110). Exemplarily, the specific context of the current context profile 200 corresponds to the activity "resting" and the location "indoors".
[0046] The current contextual profile 200 illustrates an efficient data structure for continuously or semi-continuously passively monitoring a user's context-specific physiological state or condition. For example, the earliest element of the time series matrix (T 1 ), and shift each remaining column to the left to add as T K The most recently determined features (vital signs and motion) are inserted in the last column below.
[0047] Still reference Figure 1, context-aware anomaly determiner 102 performs context-based anomaly detection 138 by comparing features of the user's current contextual profile 200 with corresponding features of a context-aware baseline profile having the same context as the user's current contextual profile. Context-aware anomaly determiner 102 may detect the presence of anomalies at different times T in response to detecting that the user's current contextual profile 200 is different from the user's current contextual profile. k and T k+1 The comparison may be performed based on a significant change in one or more of the monitored user's vital signs (e.g., heart rate, breathing rate). Alternatively, the context-aware anomaly determiner 102 may perform the comparison at each discrete interval (e.g., 10-minute intervals) of monitoring the user's vital signs and updating the user's vital signs to generate the user's current contextual profile 200.
[0048] Now additionally refer to Figure 3 , the context-aware anomaly determiner 102 selects a context-aware baseline profile 300 having the same context as the user's current context profile 200 from a plurality of context-aware baseline profiles 302 having different contexts. Although the plurality of context-aware baseline profiles 302 only include four context-aware baseline profiles, the plurality of context-aware baseline profiles may include and typically include many more context-aware baseline profiles having different contexts. For example, if the context is characterized by J locations, K environments, and L activities, the plurality of context-aware baseline profiles may include up to JxKxL context-aware baseline profiles having different contexts.
[0049] The context-aware anomaly determiner 102 performs context-based anomaly detection 138 by performing a row-by-row, feature-by-feature comparison of the current context profile 200 and the corresponding features of the context-aware baseline profile 300 having the same context. In some arrangements, the context-aware anomaly determiner 102 constructs a null distribution for each row of the context-aware baseline profile 300 based on values calculated as cumulative sum (CuSum) statistics. The CuSum statistic combines current and previous data to provide enhanced detection of changes in the process and is therefore well suited for health data analysis. The CuSum statistics for each column of the current context profile 200 are evaluated. If the CuSum statistic for the i-th row and j-th column of the matrix representation of the current context profile 200 deviates significantly from the null distribution (i.e., the deviation is greater than a predetermined threshold (e.g., greater than 1.5 standard deviations)), the context-aware anomaly determiner 102 marks the deviation as an anomaly.
[0050] Figure 4An example of context-based anomaly detection 138 performed by a context-aware anomaly determiner 102 according to certain embodiments is schematically illustrated. The context-aware anomaly determiner 102 extracts context information from a current context profile 200. Based on the context information, the context-aware anomaly determiner 102 selects a context-aware baseline profile 300 from context-aware baseline profiles 302 electronically stored in a database 148. The context-aware anomaly determiner 102 determines an anomaly score S based on a CuSum statistic generated from a row-by-row, feature-by-feature comparison between the current context profile 200 and the context-aware baseline profile 302. a 400. The context-aware anomaly determiner 102 performs the following operations on each column T of the current context profile 200: 1 To T K Determine the row-based score W mi , where m indicates a specific context, i indicates a row of a given column, and each column corresponds to a specific time. Assuming that the current context profile 200 is an NxK matrix, the context-aware anomaly determiner 102 calculates the score W by summing the rows of each column j m·j To determine the j If the value deviates significantly from the null distribution, an anomaly is detected at the i-th row of the j-th column. For each column T of the current contextual profile 200, 1 To T K , the sum of significant deviations from the null distribution is:
[0051]
[0052] Where N is related to time T j The number of rows corresponding to the jth column, and
[0053]
[0054] Sum the anomaly counts for each column to generate an anomaly score S a 400:
[0055]
[0056] Based on the anomaly score S a 400, the context-aware anomaly determiner 102 determines an action according to the following rules:
[0057] If at decision 402, S a >T h , where T his a predetermined high threshold indicating the occurrence of a possible abnormal health event, the context-aware anomaly determiner 102 initiates an event confirmation process performed by the EE / UR determiner 104 as described below. The context-aware anomaly determiner 102 also transmits S a , contextual information and motion features; and
[0058] If at decision 404, S a <T h But S a >M, where M is a predetermined low threshold, the context-aware anomaly determiner 102 transmits instructions to the vital sign monitor 108 and the motion monitor 112 to increase the frequency at which each monitor collects and processes data derived from signals generated by sensors that monitor the user's vital signs and motion (e.g., from a 10-minute interval to a 1-minute interval). Otherwise, the context-aware anomaly determiner 102 optionally updates the user's baseline data but does not initiate further action.
[0059] Reference again Figure 1 , if the anomaly score S a 400 is greater than the high threshold T h , so that the context-aware anomaly determiner 102 calls a process performed by the EE / UR determiner 104, then the EE / UR determiner 104 responds by transmitting a recording request to a multimedia sensor processor 114 operably coupled to one or more multimedia sensors 140. The multimedia sensor 140 includes, for example, a microphone 142 for recording user voice and other sounds, a camera 144 for recording user movements, and an eye tracker 146 for tracking user eye movements. In response to the recording request transmitted by the EE / UR determiner 104, the multimedia request processor 116 invokes the operation of one or more multimedia sensors 140 to collect at least one multimedia event. The multimedia event may, for example, include the user's breathing sound or voice, the user's movement or gesture, the user's eye movement, or other such events captured by the multimedia sensor 140. The multimedia data processor 118 extracts features from the multimedia sensor to be input into one or more machine learning models implemented by the EE / UR determiner 104.
[0060] In some arrangements, one or more machine learning models implemented by the EE / UR determiner 104 are trained to classify various multimedia events based on features extracted by the multimedia data processor 118. Examples of audio-based multimedia event recordings include a user's dying breathing, labored breathing, groaning, or other pain-related sounds, unintelligible speech, etc. Examples of camera-captured multimedia events include videos showing a user unable to get up after a fall, sweating, motionless, or other abnormal movements. Each audio-based and camera-captured multimedia event detected and classified by the machine learning model generates an event input a i , which is used by the EE / UR determiner 104 to generate the emergency score S as described below EES As already described, emergency events may also be associated with abnormal body movements detected by the motion monitor 112. The motion data generated by the motion monitor 112 may also be classified by the machine learning model, as described above. The classification of the motion data may correspond, for example, to the user experiencing a convulsion, falling, exhibiting an abnormal gait or posture, or other abnormal movement. Each such event detected and classified by the machine learning model may generate an event input b i , which may also be used by the EE / UR determiner 104 to generate an urgency score.
[0061] The EE / UR determiner 104 calculates the highly abnormal event score S h and anomaly score S a and for event a i and b i Multiply by event a i and b i The urgency score S is generated by summing the products of the negative log-probabilities of EES :
[0062]
[0063] Among them, P ei It is event i The probability of occurrence, and e i =a i or e i =b i The probability P ei It may be predetermined based on empirical evidence and stored electronically in the database 148. Highly abnormal event score S h Based on predetermined values generated in response to certain highly abnormal events. These highly abnormal events may include, for example, a fall, a very high body temperature (e.g., greater than 103F), low blood oxygen saturation (e.g., less than 90%), an abnormally high resting heart rate (e.g., greater than 120 bpm), an abnormally high resting respiratory rate (e.g., greater than 30 breaths per minute), etc.
[0064] Figure 5 The EE / UR determiner 104 generates the emergency score S EES In response to a recording request transmitted by the EE / UR determiner 104, one or more multimedia events 502 are detected from signals generated by the multimedia sensor. Multimedia data 504 is derived from the detected multimedia events 502. Extracted features 506 (i.e., features extracted from the detected multimedia events 502) are input to a machine learning model classifier for use in generating one or more multimedia event classifications 508. The multimedia event classifications 508 may be combined with the motion event classifications 510 made by the motion monitor 112 to generate multimedia event inputs 504, respectively. 1 …e m and sports events m+1 …e n As mentioned above, the emergency score S EES is the negative log probability multiplied by the multimedia event input e 1 …e m and sports events m+1 …e n The sum of the values of each of the h and anomaly score S a If S EES If the EE / UR value is greater than a predetermined emergency threshold R indicating the likelihood of an emergency event, the EE / UR determiner 104 performs a user responsiveness determination process 514 and, depending on the determination, may take other actions.
[0065] Optionally, each time the user's current contextual profile is updated, the system 100 may run one or more priority checks to determine whether one or more highly abnormal events have been detected based on the user's vital signs and / or motions that immediately invoke the responsiveness determination process 514. The value generated in response to detecting such a highly abnormal event may be set such that the highly abnormal score S is h Always high enough to ensure S EES 514. In some arrangements, the system 100 is configured to enable a user to specifically designate highly abnormal events, the detection of which automatically invokes the responsiveness determination process 514 and an emergency response (if necessary). For example, an elderly user who may be severely debilitated by a fall may designate such an event as a highly abnormal event. When a motion is detected that the machine learning model classifies as corresponding to a fall, the resulting abnormal event score S h Make S EESis always greater than the emergency threshold R, and the responsiveness determination process 514 is automatically invoked. For example, an individual who is susceptible to epilepsy may designate such an event as a highly abnormal event, so that when a movement is detected that the machine learning model classifies it as corresponding to epilepsy, the abnormal event score S h So that the procedure is also called automatically.
[0066] Figure 6 Schematically illustrates a user responsiveness determination process 514 performed by the EE / UR determiner 104 according to certain arrangements ( Figure 5 ) example. The EE / UR determiner 104 can determine whether there is voice activity based on the multimedia event classification 508 at decision 600, or based on the motion event classification 510 ( Figure 5 ) to determine if there is user motion. If there is no voice activity at decision 600 or no motion is detected at decision 602, the EE / UR determiner 104 pushes a query 604 to the user asking whether the system 100 should initiate a call 608 for emergency assistance. Even if both voice activity and motion are detected, the S at decision 606 EES If the EE / UR determiner 104 is greater than the emergency threshold R, the EE / UR determiner 104 still pushes the query 604 to the user. The query 604 asks the user whether assistance is needed. If at decision 608, the user responds “No” to the query 604 or if S EES If the EE / UR determiner 104 is less than the emergency threshold R, the EE / UR determiner 104 generates a flag 610 and applies it to the event-related data, indicating that the event is not an emergency. If at decision 608, the user responds "yes" to the query 604 or does not respond after a preset number of queries (e.g., two attempts), the EE / UR determiner 104 initiates an action of the smart responder 106. Optionally, the EE / UR determiner 104 may also transmit additional data related to the possible emergency situation to the smart responder 106. The data may include, for example, a user profile, current vital signs, contextual information, location information, contact information, and / or other information relevant to and useful to the emergency responder.
[0067] If invoked by EE / UR determiner 104, smart responder 106 automatically calls the emergency contact using the communication capabilities of the device in which system 100 is implemented. For example, if system 100 is implemented in a wearable device (e.g., earbuds, smart watch, smart glasses), the device may include a wireless communication subsystem, such as device 1300 ( Fig.13) wireless communication subsystem 1324, which smart responder 106 can utilize to call the emergency contact. Alternatively, if the device in which system 100 is implemented lacks communication capabilities, the device can be communicatively coupled to another device (e.g., a smart phone) among the user's devices, which smart responder 106 can use to call the emergency contact.
[0068] Smart responder 106 can make an automatic call or send a text message to emergency medical services (EMS) and / or pre-designated emergency contacts. Based on the GPS location data generated by the system, smart responder 106 can make an automatic call or send a text message to the EMS closest to the user's current location. Similarly, if the user has electronically stored a list of the names of multiple pre-designated emergency contacts in database 148, smart responder 106 can search and select the contact closest to the user's current location to call or send a text message. Optionally, the user can specify one or more contacts, such as a doctor or family member, to be contacted by smart responder 106 (regardless of the user's location relative to the pre-designated contacts). In each case, smart responder 106 can transmit a pre-recorded voice or text message and attach information such as the user's location and an overview of the user's current vital signs.
[0069] In some arrangements, health professionals can register their information in a global database and enter their qualifications and supporting materials. After the user's emergency health event has been verified, the smart responder 106 can transmit a notification to a registered professional near the user's current location determined by the smart responder 106 based on GPS or other geographic location data. In a specific arrangement, users of multiple adverse health event detection and response systems can also agree to link to a public communication network (e.g., a central server) to be contacted in the event of proximity to another user experiencing an emergency health event. Such individuals may be able to provide at least limited assistance before the health professionals arrive.
[0070] System 100 can establish a communication link between a device implemented with system 100 and another device used by an EMS dispatcher or an emergency contact. Via this communication link, smart responder 106 can keep in touch with EMS dispatchers and / or other emergency contacts, and provide updates about the user to them. In addition to notifying EMS dispatchers, emergency contacts and / or nearby users, smart responder 106 can also transmit information useful when providing medical assistance. In some arrangements, smart responder 106 generates an overview of the user's context-based information. This can include the user's current vital signs, recent activities, and any previous or current motion indicating the user's current physiological state or condition. Via the communication link established between the device implemented in system 100 or via the device coupled to the communication ground of the supporting device and the device of the EMS dispatcher or other health professionals, the EMS dispatcher or other health professionals can request the system 100 to use the data derived from the multimedia sensor 140 and processed by the multimedia sensor processor 114 to collect and transmit the current information about the user. For example, if the dispatcher needs to assess the user's current physiological condition, the dispatcher may request real-time audio and / or video of the user in addition to the user's vital signs.
[0071] Using GPS data, smart responder 106 can estimate the time until an ambulance arrives and the distance between the user and the health professional. For example, in a remote area, where no ambulance is available nearby, smart responder 106 can transmit a request for help to a medical provider registered with the system or to a user who can provide at least limited assistance until an ambulance can reach the user.
[0072] During an emergency, the system 100 can increase the frequency of updating the user's current contextual profile so that monitoring is continuous or nearly continuous. However, usually, continuous or nearly continuous, monitoring may be inefficient. However, since a set of user-specific context-aware baseline profiles must usually be built over time through the system 100's monitoring of the user, the system 100 (at least during the initial stage of use) may not have enough user-specific data to build a complete set of snapshots of user baselines under different contexts. For example, during most of the time, a typical user may be sedentary or perform activities that do not require the user to exert too much physical effort. Therefore, at least in the early stages, the system 100 may only have a partial set of context-aware baseline profiles corresponding to a limited number of contexts for the user. Therefore, in some arrangements, the system 100 implements collaborative filtering to generate a temporary but extended set of context-aware profiles for the user based on data corresponding to other users identified as being demographically similar to the user.
[0073] The system 100 may obtain data corresponding to other users identified as being demographically similar from a global database of context-aware baseline profiles. In some arrangements, the system 100 identifies demographically similar users by computing a similarity matrix, where the similarity Sim of any two users may be computed, for example, using a Pearson correlation. (x,y,c) :
[0074]
[0075] where “x” and “y” correspond to different users within the same context “c”, and F i (i=1…k) are context-specific features (e.g., vital signs, motion). Based on the similarity matrix calculation, the system 100 uses the k-nearest neighbor algorithm to identify similar users. Then, the context-aware baseline profile of the similar users is estimated based on the weighted average of the context-specific features (e.g., vital signs, motion) of each similar user.
[0076] A large number of users can be divided into two or more user groups based on epidemiology, demographics and / or other health-related factors. Therefore, according to some arrangements, users of the system 100 can be assigned to one of a plurality of different priority groups based on user-specific factors. For example, users can be assigned to a high-risk group. This assignment can be based on factors such as age (e.g., greater than 55 years old), treatment history (e.g., number of hospitalizations), health-related events (such as cardiac arrest, lung or kidney disease, cancer), and / or other factors that are determined to put users at a higher risk based on epidemiological evidence. In some arrangements, the system 100 can utilize a device portal to enable medical professionals (e.g., the user's doctor) to assign users to specific risk groups. Users with no prior medical history of major medical events or basic abnormal health conditions can be assigned to a low-risk user group. Over time, based on updates to the user's context-aware profile, the system 100 can suggest that users be reassigned to different groups. For example, the system 100 may suggest reassigning the user to a high-risk group in response to the user suffering from one or more adverse health events or in response to one or more persistent changes in the user's vital signs relative to a given context. As described above, each time a possible adverse event is detected but no emergency event is caused, the system 100 may update the user's baseline profile.
[0077] Depending on the risk group to which the user is assigned, the system 100 may be adapted to provide a user with a risk score S based on the risk group. EES to determine whether the user may have experienced an adverse or emergency health event. If the user is classified as at risk, the system 100 may use the data used to determine the anomaly score S aand the highly abnormal event score S h For example, if the user has a history of heart attacks, the system 100 applies a higher weight to heart-related features. More generally, the system 100 may apply a higher weight to one or more values of M, T, and M. h and R to set a lower threshold for detecting adverse health events or emergency health events. In other arrangements, the system 100 can be based on user-specific risk factors r p and optionally a context-specific risk factor r c To determine S EES :
[0078]
[0079] A user-specific risk factor r can be calculated based on the user's physiological or medical condition p Context-specific risk factors r c can correspond to a specific context, but when combined with a user-specific risk factor r p When used in combination, it can also reflect the condition or status of the user. For example, the user-specific risk factor r of a user with heart disease is 0.01% compared to a user without heart disease. p The context-specific risk factor for vigorous exercise compared with sedentary activity is c When two risk factors are combined, such as a user with heart disease who exercises vigorously, S EES S EES Much higher.
[0080] The system 100 optionally includes a context-aware privacy manager that allocates sensing resources in response to activation requests from the vital sign monitor 108, the environment monitoring unit E, the motion monitor 112, and the multimedia sensor processor 114. The context-aware privacy manager can apply a rule-based approach to allocate sensing resources based on whether the request is related to monitoring a user experiencing a possible emergency event or whether it is related to continuously monitoring the user. In some arrangements, the context-aware privacy manager updates and maintains a priority queue for each context and allocates sensing resources accordingly.
[0081] Figure 7An example rule set 700 applied by the context-aware privacy manager for updating and maintaining a priority queue is shown. Exemplarily, the context-aware privacy manager responds to a possible emergency event by assigning the motion sensor 136 and the multimedia sensor 140 to continuously sense and continuously record motion and multimedia data derived from the signals generated by the sensors. Otherwise, for continuous sensing, the context-aware privacy manager assigns the motion sensor 136 to continuously sense and continuously record the generated motion data, but only generates and records multimedia data once every 10 minutes, thereby limiting the use of more resource-intensive multimedia sensors and related signal processing. The context-aware privacy manager primarily assigns the motion sensor 136 for vital sign monitoring. However, the motion sensor 136 may introduce an excessive number of motion artifacts that may produce inaccurate or unreliable motion data. Therefore, the context-based sensor scheduler may apply additional pre-established rules. First, if the user is performing a motion-intensive activity, thereby making the motion sensor 136 less reliable, the context-aware privacy manager assigns the multimedia sensor 140 for more frequent user monitoring. However, since monitoring using multimedia sensors 140 tends to be relatively more privacy-intrusive, the context-based sensor scheduler will enforce less frequent multimedia sensor-based monitoring in most cases. Second, therefore, if the user is sedentary or stationary, the context-aware privacy manager limits monitoring by the multimedia sensors 140 to a frequency sufficient to improve the reliability of the motion sensors 136 involved in vital sign monitoring. For example, the frequency may be limited to intermittent monitoring at regular intervals (e.g., at 10-minute intervals).
[0082] Figure 8 800, certain operational aspects of the system 100 are shown. Exemplarily, the system 100 is implemented in an earbud 802 worn by a user 804. The user 804 is a 75-year-old male who has had a heart attack, has high blood pressure, is overweight, and is a smoker. To classify the user 804 as high risk, the system 100 applies a low threshold M=3, a high threshold T h =10 and the emergency threshold R=20. The system 100 illustratively updates the current contextual profile 806 based on the signals generated by one or more sensors, and selects a context-aware baseline profile 808 with the same context (resting-indoor) for comparison. Although the current contextual profile 806 and the context-aware baseline profile 808 are represented by a 10x5 matrix, it is expressly noted that both can and typically will include matrices with many more elements. Initially at 1:40 PM and 1:50 PM, the first row of the current contextual profile 806 indicates a higher than baseline breathing rate (RR), and the sixth row indicates abnormal body movement. The system determines an abnormality score S aExceeds the lower threshold M but is less than the upper threshold T h , thereby prompting the context-aware anomaly determiner 102 to increase the monitoring frequency. As the monitoring frequency increases, additional abnormal motion and continuous abnormal breathing rate are detected, resulting in anomaly scores of 8 and 12 at 1:51 pm and 1:52 pm, respectively. Anomaly score S of 12 a Exceeds the threshold T h , EE / UR determiner 104 responds to possible abnormal events by initiating a confirmation process. The process generates multimedia (audio-based) events and motion classifications. The multimedia event classification corresponds to agonal breathing and lack of phonation and sweating. The motion event classification corresponds to abnormal convulsions. When the latest (last column of the matrix) feature set is combined with the previous feature set at 1:53 p.m., S EES The score is above the emergency threshold R. In this example, the system 100 confirms a possible emergency event (a possible heart attack). Once the EE / UR determiner 104 confirms the emergency event, the smart responder 106 initiates remedial action as described above.
[0083] Fig. 9 Certain operational aspects 900 of the system 100 are also shown. Exemplarily, the system 100 is implemented in a smartwatch 902 worn by a user 904. The user 904 is a 70-year-old female who is classified as a high-risk person by the system 100 due to having heart disease, diabetes, and high cholesterol. Figure 8 For example, the IMU of smartwatch 902 generates a signal that system 100 (using the above-mentioned machine learning) classifies as a fall, which is pre-assigned a highly abnormal event score S h 15, and automatically invokes the immediate user responsiveness determination of the EE / UR determiner 104. This determination generates an anomaly score S based on the latest update of the current contextual profile 906, based on a comparison with the context-aware baseline profile 908, and event classification of audio-based multimedia events and motion events (also made using the machine learning model). a The audio-based multimedia event classification corresponds to dysphonia mixed with silent intervals. The motor event classification corresponds to abnormal twitching. h and S a The sum of the scores and the product of the event times the negative log probability of the event, S determined by the EE / UR determiner 104 EES The score (indicating a possible stroke) exceeds the emergency threshold R. The EE / UR determiner 104 responds by invoking a remedial action of the smart responder 106 .
[0084] Fig.10Certain operational aspects 1000 of the system 100 are also shown. Exemplarily, the system 100 is implemented in smart glasses 1002 worn by a user 1004. The user 1004 is a user who is classified as low risk by the system 100 and has a threshold value M, T, and h and R is assigned values of 10, 20, and 30 for a 30-year-old healthy male. For example, the thermometer 128 of the vital sign monitor 108 embedded in the smart glasses 1002 detects a highly abnormal core body temperature of the user 1004, which is pre-assigned as a highly abnormal event score S h A highly abnormal event with a value of 15. A highly abnormal core body temperature of user 1004 is detected (which generates an abnormal event score S of 15). h ) automatically invokes the immediate user responsiveness determination of the EE / UR determiner 104. This determination generates an anomaly score S based on the latest update to the current contextual profile 1006, based on a comparison with the context-aware baseline profile 1008, and the event classification (also made using the machine learning model) a The video-based multimedia event classification corresponds to the unconscious state and sweating of the user 1004. The audio-based multimedia event classification corresponds to shortness of breath and vomiting. The eye tracking classification corresponds to dizziness, and the motion event classification corresponds to a motionless state. The S determined by the EE / UR determiner 104 EES Score (indicating possible hypothermia) (equal to S h and S a The sum of the scores plus the sum of the products of each abnormal event multiplied by the negative log probability of the event) exceeds the emergency threshold R. The EE / UR determiner 104 responds by invoking a remedial action of the smart responder 106.
[0085] Fig.11A and Fig. 11B An example method 1100 of adverse health detection and emergency response is shown. The method 1100 may be performed by a device (or a device operably coupled to a companion device) including an adverse health detection and emergency response system (collectively referred to as a "system") as described herein.
[0086] At box 1102, the system can build a database of context-aware baseline profiles specific to the user. If at box 1104, the system determines that the database is not yet complete, then at box 1106, the system can use data of similar users to provide missing data as needed. In some arrangements, the system implements collaborative filtering to generate a temporary but extended set of context-aware profiles for the user based on data corresponding to other users identified as being demographically similar to the user. In some arrangements, the system identifies demographically similar users by calculating a similarity matrix and determining a statistical relationship (such as a Pearson correlation) between the user and other users to identify similar users. The construction process can continue over time as the system modifies the context-aware baseline profile in response to ongoing changes in the user's vital signs.
[0087] At box 1108, the system can update the current contextual profile of the user. The update can be based on signals generated by one or more sensors, which are generated in response to one or more physical measurable phenomena associated with the user. The current contextual profile can include features corresponding to the user's vital signs (i.e., measurable variables indicating the user's current physiological condition or state) and data about the user's current activity, environment, and location. In some arrangements, the system intermittently updates the current contextual profile at a frequency determined based on the risk category to which the user is assigned. The risk category can be determined based on user demographic data.
[0088] At block 1110, the system determines a context and, at block 1112, selects a context-aware baseline profile having the same context as the current context profile. The context-aware baseline profile is selected from a database of multiple context-aware baseline profiles. Different context-aware baseline profiles correspond to different contexts.
[0089] At block 1114, the system compares features (matrix elements) of the current contextual profile to corresponding features of the selected context-aware baseline profile. One or more deviations of features of the current contextual profile from one or more corresponding features of the selected context-aware baseline profile may indicate that the user is experiencing an adverse health event, possibly one that rises to the level of a medical emergency.
[0090] In some arrangements, the system determines the likely occurrence of an abnormal health event based on an anomaly score generated by the system at block 1116. In one aspect, the anomaly score can be determined based on a CuSum statistic calculated based on the deviation of each current contextual profile feature from a null distribution determined based on a context-aware baseline profile.
[0091] At block 1118, if the anomaly score is less than the high threshold, then at block 1120, a determination is made whether the anomaly score is greater than the low threshold. If so, because the anomaly score is less than the high threshold but greater than the low threshold, the system adjusts sensor-based signal collection (e.g., increases the frequency of signal collection) to enhance monitoring of the user at block 1122. By adjusting the frequency of intermittently updating the current contextual profile in response to a deviation between the current contextual profile and one or more corresponding features of the context-aware baseline profile, the system collects information that can resolve uncertainty regarding the user's current physical condition.
[0092] Otherwise, if a determination is made at block 1120 that the anomaly score is less than the low threshold, then the system discards the anomaly score at block 1124. Alternatively, the system can update the context-aware baseline profile, assuming that significant differences between the current contextual profile and the context-aware baseline profile indicate ongoing changes in certain vital signs of the user, but do not indicate an anomaly, for example (e.g., average blood pressure increases without issue as the user ages).
[0093] If at box 1118, the anomaly score is greater than the high threshold, the score indicates a possible serious health event, possibly a serious health event that rises to the level of a medical emergency. More information about the user's actual physical condition is obtained, and the possibility of false positives is reduced. At box 1126, the system initiates a confirmation process. One aspect of the confirmation process is to capture multimedia events using multimedia sensors and motion events using motion sensors at box 1128.
[0094] The system classifies multimedia events and sports events using one or more machine learning classification models (e.g., random forests) at blocks 1130 and 1132, respectively. These classifications identify one or more abnormal or highly abnormal events.
[0095] At block 1134, the system determines an emergency event score. In some arrangements, the emergency event score is the sum of the highly abnormal event score and the anomaly score plus the sum of the product of the abnormal event multiplied by the negative log probability of the event.
[0096] The confirmation process determines whether any speech can be detected at box 1136, and whether any movement of the user can be detected at box 1138. If both speech activity and movement activity are detected, and if at box 1140, the emergency score is less than the emergency threshold, the system discards the emergency score at box 1146 and marks the event as a "false positive" at box 1148.
[0097] At box 1142, the system responds to the inability to detect voice activity or motion activity or to an emergency score greater than the emergency threshold. Any of these situations invokes the action of the system, whereby the system pushes a query to the user. Therefore, the system attempts to determine the responsiveness of the user. In order to determine the responsiveness of the user, in some arrangements, the system pushes a query to the user at box 1144. The query asks the user whether assistance is needed. If the user responds "no", at box 1146, the system discards the emergency score and, at box 1148, marks the event as a "false alarm". However, if the user responds "yes" or there is still no response after the system makes a predetermined number of (e.g., two) query attempts, the system initiates a remedial response.
[0098] In some arrangements, the system responds by identifying one or more potential responders at box 1150. In some arrangements, the system identifies an emergency responder that is located closest to the user's current location based on geolocation determination. Optionally, the system can also search a list of emergency contacts pre-specified by the user to initiate contact with one or more of the listed contacts. At box 1152, the system notifies one or more of the potential responders, and notifies each potential responder of the emergency event affecting the user.
[0099] Fig.12 An example method 1200 of poor health detection and emergency response is shown. The method 1200 may be performed by a device (or a device operably coupled to a companion device) including a poor health detection and emergency response system (collectively referred to as a "system") as described herein.
[0100] At block 1202, the system updates a current contextual profile of a user based on signals generated by one or more sensors of a device (e.g., a wearable device), the signals being generated in response to one or more physically measurable phenomena associated with the user. At block 1204, the system selects a context-aware baseline profile having the same context as the current contextual profile. The context-aware baseline profile is selected from a plurality of context-aware baseline profiles having different contexts.
[0101] At block 1206, the system compares features of the current contextual profile with corresponding features of a context-aware baseline profile of the user having the same context as the current contextual profile. At block 1208, the system initiates remedial action in response to identifying an occurrence of a possible adverse health event affecting the user based on the comparison.
[0102] Fig.13An example device 1300 configured to operate as a system is shown. Device 1300 (e.g., a wearable device or device and a companion device operably coupled thereto) can implement an adverse health event detection and emergency response system such as system 100. Device 1300 illustratively includes one or more processors 1302, which are coupled to memory 1304 via interface circuit 1306. Device 1300 stores computer-readable instructions (also referred to as "program code") in memory 1304, which is an example of a computer-readable storage medium. Processor 1302 executes program code accessed from memory 1304 via interface circuit 1306.
[0103] For example, memory 1304 may include one or more physical memory devices such as local memory 1308 and mass storage device 1310. Local memory 1308 is implemented as one or more non-persistent memory devices that are typically used during the actual execution of program code. Local memory 1308 is an example of runtime memory. Examples of local memory 1308 include any of various types of RAM suitable for being used by a processor to execute program code. Mass storage device 1310 is implemented as a persistent data storage device. Examples of mass storage device 1310 include a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other suitable memory. Device 1300 may also include one or more cache memories (not shown) that provide temporary storage of at least some program code in order to reduce the number of times program code must be retrieved from a mass storage device during execution.
[0104] Examples of interface circuit 1306 include, but are not limited to, input / output (I / O) subsystems, I / O interfaces, bus systems, and memory interfaces. For example, interface circuit 1306 can be implemented as any of a variety of bus structures and / or a combination of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus.
[0105] In one or more example implementations, the processor 1302, the memory 1304, and / or the interface circuit 1306 are implemented as separate components. The processor 1302, the memory 1304, and / or the interface circuit 1306 may be integrated into one or more integrated circuits. For example, the various components in the device 1300 may be coupled via one or more communication buses or signal lines (e.g., interconnects and / or lines). The memory 1304 may be coupled to the interface circuit 1306 via a memory interface (e.g., a memory controller or other memory interface (not shown)).
[0106] Device 1300 may include one or more displays. Exemplarily, for example, device 1300 includes display 1312 (e.g., screen). Display 1312 may be implemented as a touch-sensitive or touch screen display capable of receiving touch input from a user. The touch-sensitive display and / or touch-sensitive pad may detect contact, motion, gestures, and contact interruption using any of a variety of available touch-sensitive technologies. Examples of touch-sensitive technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more contact points with a touch-sensitive display and / or device.
[0107] The device 1300 may include a multimedia sensor such as a camera subsystem 1314. The camera subsystem 1314 may be coupled to the interface circuit 1306 directly or through a suitable input / output (I / O) controller. The camera subsystem 1314 may be coupled to an optical sensor 1316. The optical sensor 1316 may be implemented using any of a variety of technologies. Examples of the optical sensor 1316 may include, but are not limited to, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) optical sensor. The optical sensor 1316 may be, for example, a depth sensor. The camera subsystem 1314 and the optical sensor 1316 are capable of performing camera functions such as recording or capturing images and / or recording video.
[0108] Device 1300 may include other multimedia sensors such as an audio subsystem 1318. Audio subsystem 1318 may be coupled to interface circuitry 1306, either directly or through a suitable input / output (I / O) controller. Audio subsystem 1318 may be coupled to speaker 1320 and microphone 1322 to facilitate voice-enabled functions such as voice recognition, voice replication, digital recording, and telephony functions.
[0109] The device 1300 may include one or more wireless communication subsystems 1324. Each wireless communication subsystem 1324 may be coupled to the interface circuit 1306 directly or through a suitable I / O controller (not shown). Each wireless communication subsystem 1324 is capable of facilitating communication functions. Examples of wireless communication subsystems 1324 may include, but are not limited to, radio frequency receivers and transmitters and optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the wireless communication subsystems 1324 may depend on the specific type of device 1300 implemented and / or the communication network over which the device 1300 is intended to operate.
[0110] As an illustrative and non-limiting example, the wireless communication subsystem 1324 can be designed to operate through one or more mobile networks, WiFi networks, short-range wireless networks (e.g., Bluetooth), and / or any combination of the foregoing networks. The wireless communication subsystem 1324 can implement a hosting protocol so that the device 1300 can be configured as a base station for other wireless devices.
[0111] The device 1300 may include one or more sensors 1326, each of which may be coupled to the interface circuit 1306 directly or through a suitable I / O controller (not shown). The sensors 1326 may include sensors for measuring the physiological condition or state of the user, such as a RIP sensor, a pulse oximeter, an EDA / GSR sensor, a PPG sensor, and a core thermometer. The sensors 1326 may also include motion detection sensors including one or more MEMS sensors. In different arrangements, the MEMS sensors may include accelerometers, magnetometers, and the like. In certain arrangements, the MEMS sensors may include one or more IMUs that combine a MEMS accelerometer and a MEMS gyroscope. Other examples of the sensors 1326 may include, but are not limited to, position sensors (e.g., GPS receivers and / or processors) that can provide geolocation sensor data, electronic magnetometers (e.g., integrated circuit chips) that can provide sensor data that can be used to determine the direction of the magnetic north pole for directional navigation, accelerometers that can provide data indicating changes in the speed and direction of the movement of the device 1300, and altimeters (e.g., integrated circuits) that can provide data indicating the height.
[0112] The device 1300 may also include one or more input / output (I / O) devices 1328 coupled to the interface circuit 1306. The I / O devices 1328 may be coupled to the interface circuit 1306 directly or through an intermediate I / O controller (not shown). Examples of I / O devices 1328 include, but are not limited to, a track pad, a keyboard, a display device, a pointing device, one or more communication ports (e.g., a universal serial bus (USB) port), a network adapter, and buttons or other physical controls. A network adapter is a circuit that enables the device 1300 to be coupled to other systems, computer systems, remote printers, and / or remote storage devices through an intermediate private or public network. A modem, a cable modem, an Ethernet interface, and a wireless transceiver that is not part of the wireless communication subsystem 1324 are examples of different types of network adapters that may be used with the device 1300. One or more I / O devices 1328 may be adapted to control the functionality of one or more or all of the sensors 1326 and / or one or more wireless communication subsystems 1324.
[0113] Memory 1304 stores program code. Examples of program code include, but are not limited to, routines, programs, objects, components, logic, and other data structures. For purposes of illustration, memory 1304 stores operating system 1330 and applications 1332. In addition, memory 1304 may store code for implementing an adverse health event detection and emergency response (AHEDER) system (e.g., system 100 ( Figure 1 ))'s AHEDER system program code 1334.
[0114] Device 1300 is provided for purposes of illustration and not limitation. Devices and / or systems configured to perform the operations described herein may have Fig.13 The architecture shown is different from the architecture. This architecture can be a combination of Fig.13 A simplified version of the architecture described in the embodiment of the present invention includes a memory capable of storing instructions and a processor capable of executing instructions. In this regard, the device 1300 may include a processor capable of executing instructions. Fig.13 Fewer components than shown or in Fig.13 Additional components not shown in the example may be included, depending on the specific type of device implemented. In addition, the specific operating system and / or application included may vary depending on the type of device (such as the type of I / O devices included). In addition, one or more of the example illustrative components may be incorporated into another component or otherwise form a part of another component. For example, a processor may include at least some memory.
[0115] Device 1300 may be implemented as a data processing system, a communication device, or other suitable system suitable for storing and / or executing program code. Device 1300 may be implemented as an edge device. Example implementations of device 1300 may include, but are not limited to, earplugs, smart watches, smart glasses, HMD devices or other wearable devices, smart phones or other portable devices, laptops, tablet computers, or other computing devices.
[0116] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. Nevertheless, several definitions that apply throughout this document are expressly defined as follows.
[0117] As defined herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
[0118] The term "substantially" means almost correct or accurate, a value or amount close but not exact. For example, the term "substantially" may mean that a feature, parameter or value described is within a predetermined amount of an exact feature, parameter or value.
[0119] As defined herein, the terms "at least one," "one or more," and "and / or" are open-ended expressions that are both conjunctive and disjunctive in operation unless expressly indicated otherwise. For example, each of the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," and "A, B, and / or C" means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together.
[0120] As defined herein, the term "automatically" means without human intervention.
[0121] As defined herein, the term "computer-readable storage medium" means a storage medium containing or storing program code used by or associated with an instruction execution system, apparatus, or device. As defined herein, a "computer-readable storage medium" is not itself a transient propagation signal. A computer-readable storage medium may be, but is not limited to: an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. As described herein, different types of memory are examples of computer-readable storage media. A non-exhaustive list of more specific examples of computer-readable storage media may include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, etc.
[0122] As defined herein, a "data processing system" refers to one or more hardware systems configured to process data, each hardware system including at least one processor programmed to initiate operations and memory.
[0123] As defined herein, "execution" and "running" include a series of actions or events performed by a processor according to one or more machine-readable instructions. As defined herein, "running" and "execution" refer to the active execution of actions or events by a processor. The terms run, running, executing, and executing are used synonymously herein.
[0124] As defined herein, the term “if” means “when” or “at” or “in response to” or “in response to”, depending on the context. Thus, the phrase “if it is determined” or “if [the condition or event] is detected” may be interpreted to mean “upon determining” or “in response to determining” or “upon detecting [the condition or event]” or “in response to detecting [the condition or event]” or “in response to detecting [the condition or event]”, depending on the context.
[0125] As defined herein, the terms "individual," "person," and "user" refer to a human being.
[0126] As defined herein, the term "processor" refers to at least one hardware circuit. The hardware circuit may be configured to execute instructions contained in a program code. The hardware circuit may be an integrated circuit. Examples of processors include, but are not limited to, a central processing unit (CPU), an array processor, a vector processor, a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic array (PLA), an application specific integrated circuit (ASIC), a programmable logic circuit, and a controller.
[0127] As defined herein, the term "in response to" and similar language as described above (e.g., "if," "when," or "at") indicates a prompt response or reaction to an action or event. The response or reaction is performed automatically. Thus, if a second action is performed "in response to" a first action, there is a causal relationship between the occurrence of the first action and the occurrence of the second action. The term "in response to" indicates a causal relationship.
[0128] As defined herein, "real time" means that a user or system senses sufficiently immediately that a particular process or determination is to be performed, or that the processor is able to keep up with the processing responsiveness of some external process. Relatedly, "real-time processing" refers to processing activities that are performed in real time.
[0129] As defined herein, a "server" refers to a data processing system that is configured to share services with one or more other data processing systems. Relatedly, a "client device" refers to a data processing system that requests shared services from a server and with which a user interacts directly. Examples of client devices include, but are not limited to, workstations, desktop computers, computer terminals, mobile computers, laptop computers, notebook computers, tablet computers, smart phones, personal digital assistants, smart watches, smart glasses, gaming devices, set-top boxes, smart TVs, etc. In one or more embodiments, the various user devices described herein may be client devices. Network infrastructure (e.g., routers, firewalls, switches, access points, etc.) is not a client device as defined herein.
[0130] As defined herein, the term "substantially" means that the recited characteristic, parameter or value need not be strictly achieved, but deviations or changes may occur in an amount that does not impair the effect that the characteristic is intended to provide, including, for example, tolerances, measurement errors, measurement precision limitations and other factors known to those skilled in the art.
[0131] The terms first, second, etc. may be used herein to describe various elements. Unless otherwise stated or the context clearly indicates otherwise, these elements should not be limited by these terms, as these terms are only used to distinguish one element from another.
[0132] The computer program product may include one or more computer-readable storage media having computer-readable program instructions thereon that cause a processor to perform aspects of the present invention. In the present disclosure, the term "program code" is used interchangeably with the term "computer-readable program instructions". The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a LAN, a WAN, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge devices including edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network, and forwards the computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device.
[0133] The computer-readable program instructions for performing the operation of the inventive arrangement described herein may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages and / or procedural programming languages. The computer-readable program instructions may specify state setting data. The computer-readable program instructions may be executed completely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer as an independent software package, or completely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including LAN or WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some cases, to perform various aspects of the inventive arrangement described herein, electronic circuits including, for example, programmable logic circuits, FPGAs, or PLAs may be executed by utilizing the state information of the computer-readable program instructions to personalize the electronic circuits.
[0134] Certain aspects of the inventive arrangement are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products. It should be understood that each block of the flowcharts and / or block diagrams and combinations of blocks in the flowcharts and / or block diagrams can be implemented by computer-readable program instructions (e.g., program codes).
[0135] These computer-readable program instructions can be provided to a processor of a computer, a special-purpose computer, or other programmable data processing device for producing a machine, so that the instructions (executed via the processor of the computer or other programmable data processing device) create a device for implementing the functions / operations specified in the flowchart and / or block diagram. In this way, by operably coupling the processor to the program code instructions, the processor's machine is converted into a special-purpose machine for executing the instructions of the program code. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide the computer, programmable data processing device, and / or other device to work in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for performing various aspects of the operations specified in one or more boxes of the flowchart and / or block diagram.
[0136] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other device to cause a series of operations to be performed on the computer, other programmable device, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable device, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.
[0137] Flowcharts and block diagrams in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product of various aspects of the arrangement according to the present invention.In this respect, each frame in the flow chart or block diagram can represent a part of a module, segment or instruction, which includes one or more executable instructions for realizing a specified operation.In some alternative implementations, the operation recorded in the frame may occur in an order outside the order shown in the accompanying drawings.For example, the two frames shown in succession may actually be substantially performed simultaneously, or the frame may be performed in reverse order at some time, depending on the function involved.It will also be noted that each frame of the block diagram and / or flow chart illustration and the combination of frames in the block diagram and / or flow chart illustration can be realized by a system based on special-purpose hardware, which performs specified functions or actions, or performs a combination of special-purpose hardware and computer instructions.
[0138] The corresponding structures, materials, acts, and equivalents of all means or step plus function elements that may be found in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed.
[0139] The description of the embodiments provided herein is for illustrative purposes and is not intended to be exhaustive or limited to the disclosed forms and examples. The terms used herein are selected to explain the principles of the arrangements of the present invention, the practical application or technical improvement of the technology found in the market, and / or to enable a person of ordinary skill in the art to understand the embodiments disclosed herein. Modifications and variations may be apparent to a person of ordinary skill in the art without departing from the scope and spirit of the described inventive arrangements. Therefore, reference should be made to the following claims rather than to the foregoing disclosure to indicate the scope of these features and implementations.
Claims
1. A method, include: updating a current contextual profile of a user based on signals generated by one or more sensors of a wearable device, the signals being generated in response to one or more physically measurable phenomena associated with the user; selecting a context-aware baseline profile having the same context as the current context profile, the context-aware baseline profile being selected from a plurality of context-aware baseline profiles having different contexts; comparing features of the current contextual profile to corresponding features of a context-aware baseline profile of the user having the same context as the current contextual profile; as well as Remedial action is initiated in response to identifying an occurrence of a possible adverse health event affecting the user based on the comparison.
2. The method according to claim 1, in, Characteristics of the current contextual profile and the context-aware baseline profile include one or more of physiological data, activity data, and environmental data corresponding to the user.
3. The method according to claim 1, further comprising: include: The current contextual profile is intermittently updated, wherein a frequency of the updating is determined based on identifying a risk category for the user based on user demographic data.
4. The method according to claim 1, further comprising: include: In response to a deviation between the current contextual profile and one or more corresponding features of the context-aware baseline profile, a frequency of intermittently updating the current contextual profile is adjusted.
5. The method according to claim 1, in, One or more of the plurality of context-aware baseline profiles are generated based at least in part on data corresponding to other users identified as being demographically similar to the user.
6. The method according to claim 1, in, The remedial actions include: determining a responsiveness of the user; and In response to a response request from the user or in response to a determination that the user is unresponsive, one or more emergency notifications are transmitted to at least one of an emergency responder or another entity.
7. The method according to claim 6, in, The other entity is a pre-designated contact of the user, and the method comprises: determining the location of a pre-designated contact of the user; determining a distance from the user to a pre-designated contact of the user; and In response to determining that the distance is within a predetermined distance threshold, providing the notification to the pre-designated user.
8. The method according to claim 1, in, The remedial actions include: generating an adverse health event score based on a sum of weighted deviations between elements of the current contextual profile and corresponding elements of a corresponding context-aware baseline profile; and In response to the adverse health event score exceeding a predetermined threshold, an event confirmation process is activated.
9. The method according to claim 8, in, The event confirmation process includes: activating at least one multimedia sensor; classifying at least one multimedia event captured by the at least one multimedia sensor; generating an urgency score based at least in part on classifying the at least one multimedia event; In response to the urgency score exceeding a predetermined threshold, determining a responsiveness of the user; and In response to a response request from the user or in response to a determination that the user is unresponsive, one or more emergency notifications are transmitted.
10. A system, include: Multiple sensors; as well as One or more processors operably coupled to the plurality of sensors, the one or more processors configured to initiate operations comprising: updating a current contextual profile of a user based on signals generated by one or more of the plurality of sensors, the signals being generated in response to one or more physically measurable phenomena associated with the user; selecting a context-aware baseline profile having the same context as the current context profile, the context-aware baseline profile being selected from a plurality of context-aware baseline profiles having different contexts; comparing features of the current contextual profile to corresponding features of a context-aware baseline profile of the user having the same context as the current contextual profile; and Remedial action is initiated in response to identifying an occurrence of a possible adverse health event affecting the user based on the comparison.
11. The system according to claim 10, in, Characteristics of the current contextual profile and the context-aware baseline profile include one or more of physiological data, activity data, and environmental data corresponding to the user.
12. The system according to claim 10, in, The processor is configured to initiate operations, the operations further comprising: The current contextual profile is intermittently updated, wherein a frequency of the updating is determined based on identifying a risk category for the user based on user demographic data.
13. The system according to claim 10, in, The processor is configured to initiate operations, the operations further comprising: In response to a deviation between the current contextual profile and one or more corresponding features of the context-aware baseline profile, a frequency of intermittently updating the current contextual profile is adjusted.
14. The system according to claim 10, in, One or more of the plurality of context-aware baseline profiles are generated based at least in part on data corresponding to other users identified as being demographically similar to the user.
15. The system according to claim 10, in, The remedial actions include: determining a responsiveness of the user; and In response to a response request from the user or in response to a determination that the user is unresponsive, one or more emergency notifications are transmitted to at least one of an emergency responder or another entity.
