Signal Processing for Care Delivery

Abstract

A system, device, and method for monitoring at least one person in at least one environment. The care processing system receives data from at least one sensor in the environment for monitoring the care recipient. The received data is matched with a pattern of such data configured by a signal processing system. Upon such data matching, at least a first sensor generates an event that matches, in whole or in part, with such pattern. The signal processing configures the first sensor to collect data created by such event, or directs a second set of sensors and subsequent sets of sensors to collect data related to the event, whereby the signal processing may determine the accuracy of such data for matching the pattern. A situation is represented to the monitored person in the environment for the purpose of providing care to the monitored person.

Description

[Technical field]

[0001]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63 / 328,083, entitled "Signal Processing for Care Provision," filed April 6, 2022.

[0002] Aspects of the present disclosure generally relate to a system for monitoring a care recipient. [Background technology]

[0003]

[0003] In a traditional infrastructure technology environment, Personal Emergency Response Systems (PERS), also known as Medical Emergency Response Systems, allow a person to call for help in an emergency by pressing a button.

[0004]

[0004] One exemplary system is a two-way voice communication pendant that allows a person to call for assistance anywhere around the home. Personal emergency response devices offer the possibility of aging in place and independent living for care recipients. With personal emergency response devices, a person can stay connected to family and emergency services through their existing landline phone. Summary of the Invention

[0005]

[0005] A system for monitoring a care recipient by an interested party, comprising a plurality of environmental sensors and a care processing system. The plurality of environmental sensors are configured to monitor the care recipient and provide a detected data set representative of the care recipient's behavior in the environment. Each behavior is represented by a multi-dimensional feature set forming a part of the care recipient's health care profile. The care processing system comprises a transceiver, a non-transitory computer-readable storage medium, and at least one hardware processing unit. The transceiver is configured to receive the detected data set. The non-transitory computer-readable storage medium is configured to store a stationary data set. The stationary data set represents a previous stationary behavior of the care recipient in the environment. The at least one hardware processing unit determines a health or care event of the care recipient by comparing the detected data set with the stationary data set. When a health or care event occurs, the care processing system is configured to change a state of the plurality of environmental sensors or notify an interested party.

[0006]

[0006] In an alternative embodiment, a system develops a pattern representative of a health state of a care recipient by an interested party. The system comprises a plurality of environmental sensors and a care processing system. The plurality of environmental sensors are configured to monitor the care recipient and provide a detected data set representative of a behavior of the care recipient in the environment. Each behavior is represented by a multi-dimensional feature set forming a part of a healthcare profile of the care recipient. The care processing system comprises a transceiver and at least one hardware processing unit. The transceiver is configured to receive the detected data set. The at least one hardware processing unit determines a variation in the detected data set indicative of a transition state between a first pattern and a second pattern in a health state representative of the health and care state of the care recipient. The care processing system is configured to change a sensor configuration of the plurality of environmental sensors to adjust to the transition state.

[0007]

[0007] In yet another alternative embodiment, a system monitors a care recipient by a participant. The system includes a plurality of environmental sensors and a care processing system. The plurality of environmental sensors are configured to monitor the care recipient and provide a detected data set representative of a care state of the care recipient in an environment. The care processing system includes a transceiver and at least one hardware processing unit. The transceiver is configured to receive the detected data set. The at least one hardware processing unit identifies and determines a care signal representative of a care state of the care recipient. The care signal includes a multi-dimensional feature set. The care processing system is configured to respond to the care signal involving a participant. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an exemplary set of modules that combine to provide, at least in part, a system for monitoring a care recipient. [Diagram 2] FIG. 1 is a diagram showing a health care profile (HCP) representing a care process of a person under monitoring (PUM) from an initial care state through a series of care states. [Diagram 3] FIG. 1 shows further HCPs. [Figure 4] FIG. 1 illustrates a set of modules for monitoring PUM in an environment. [Diagram 5] FIG. 1 illustrates the integration of a set of response systems with a care processing system. [Figure 6] FIG. 2 illustrates how a surveillance focus module interoperates with the monitored environment. [Figure 7] 1 illustrates a transition state between two behavioral patterns, each of which comprises a set of pattern elements in the HCP. [Figure 8] FIG. 13 illustrates the transition state between behavioral pattern changes and PUM care status changes. [Figure 9] FIG. 1 illustrates the use of a prediction and matching system. [Figure 10] FIG. 1 illustrates one or more digital twins operating in collaboration with an HCP. [Figure 11] A diagram showing how a PERS device generates data that is combined with data generated by general smart devices, dedicated sensors, external data sources, etc. [Figure 12] FIG. 1 illustrates a PUM in an environment. [Figure 13] FIG. 1 illustrates a PUM in an environment for generating a dataset. [Figure 14] FIG. 1 illustrates one embodiment of a Care Hub. [Figure 15] FIG. 1 illustrates a PUM in an environment. [Figure 16] FIG. 1 illustrates potential states of PUMs and the environments in which they are monitored. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009]

[0008] An aspect includes a care processing system that receives data from at least one sensor in an environment for monitoring a care recipient. The received data is matched with a pattern of such data configured by the care processing system. Upon matching of such data, at least a first sensor generates an event that matches, in whole or in part, such pattern. The care processing configures the first sensor to collect data created by such event, or directs a second set of sensors and subsequent sets of sensors to collect data related to the event, whereby the care processing may determine the accuracy of such data for matching the pattern. A situation is represented to the monitored person in the environment for the purpose of providing care to the monitored person.

[0010] Care processing requires two basic elements: a recognizable care signal that can be separated from the background conditions in which it resides.

[0011]

[0010] When one or more people reside primarily in an environment, e.g., an enclosed space such as a room, apartment, house, etc., this environment can be considered to provide a background against which signals representative of events that may affect the care and health of the person, known as the person under monitoring (PUM), may be detected. Since entropy is constantly increasing, no environment can be considered to be in a state of quiescence. Rather, the environment will contain a certain background set of characteristics that creates a background against which sensors over time may measure the variation of one or more of those characteristics. These variations may represent changes in the state of the environment and can be identified and / or classified as events. For example, temperature, pressure, light, radio frequency (RF) and other electromagnetic waves, humidity, and other characteristics may all change and such changes can be represented as events.

[0012]

[0011] In an environment where one or more sensors are deployed, each may have a limit with respect to the sensitivity of such sensor to detect changes in the measured properties that the sensor may measure. In this way, an individual sensor may detect, for example, motion that exceeds the sensitivity threshold of the device. Thus, each device has a detection threshold characteristic, typically defined by the sensor's specifications, and a field of perception or capture that defines the capabilities of the range of the sensor's sensing capabilities. Furthermore, each sensor has a minimum operating characteristic, which can be represented as the sensor's resting state, in that it does not detect any variations in the environment it is sensing.

[0013] Although a collection of sensors may each individually measure a possible characteristic, there is no combined context that can be created other than the collection of sensor data with no discernible event for each sensor.

[0014]

[0013] To effectively monitor an environment, it is necessary for each of the sensors to be aligned to a common model that represents the environment without activity, so that a baseline of the state of the environment is created. This involves the collection by the sensors in the environment of data captured over any period of time, such that any measurements include variations caused by time of day and / or other environmental factors. This data set can be augmented by external environmental data sets, such as weather monitoring systems.

[0015]

[0014] The system may include a set of baseline measurements that are typical for various environments, for example, these may be based in part on patterns, predictions, calculations, and, if available, measurements of actual or similar environments in one or more dimensions. For example, if an environment is carpeted, the acoustic profile will be different from one that has a hard surface floor.

[0016]

[0015] One simple approach can involve the installation of a device equipped with multiple sensors that can measure the environment such as temperature, humidity, pressure, time of day, ambient sound levels, etc. This baseline data can then be combined with the environmental specifications to create a model of the environment in which the PUM's behavior can be monitored. There are only a finite number of environmental spaces that a PUM will be in in a home and / or care setting. These can be created as digital twins in a modeling environment such as, for example, Unreal Engine, Unity, Autodesk, or other 3D modeling systems.

[0017]

[0016] A baseline of care treatments can be created based on the configuration of these environments, for example, whether the environments incorporate climate control, the ratio of soft to hard surfaces, purpose (kitchen / bathroom, etc.), and any other characteristics.

[0018]

[0017] Figure 1 shows an exemplary set of modules that combine to provide, at least in part, a system for monitoring a care recipient as described herein. Each of the elements of the figure is described herein. A person to be monitored (PUM-105) in an environment (104) including one or more sensors comprising a set of sensors (106) that are at least in part monitored by a monitoring system (107) combining a care process (108) and a motion HCP (101), a motion pattern (102) and a pattern element (103), in any arrangement, constitutes the care and health monitoring of the PUM. This may include machine learning (110) in any arrangement, and a digital twin (111). Care signals generated in such a system may be used by one or more response systems (109) to alert, communicate, and / or instruct one or more actors (112) to perform actions that are supportive of the care and health of the PUM.

[0019] Care Processing in Context

[0020]

[0019] Current signal processing is primarily data-centric, in that data generated by sensors is evaluated by processes designed to extract signals from the data. In general, signal processing is not contextual, and even if it may include feature extraction, it focuses on the data generated, not the context in which that data is generated. Many systems use weights and other forms of data measurements to evaluate input data streams, often to identify specific features of the data set, usually described as feature extraction.

[0021]

[0020] Some sensors may be configured to perform feature extraction, utilizing specific feature sets, such as those used for image processing and other similar functions. This can include motion detection, etc. These feature sets are often not compatible across sensors, as each sensor has its own implementation, and the sensor results and output may not include the data originally captured.

[0022]

[0021] Currently, many signal processing techniques involving multiple sensors often use data normalization to establish a common data set that can then be evaluated by further processing. One aspect that is often used is to use time as a baseline for many signal processing techniques, where the input data set is evaluated on a time basis, usually with linear elapsed time expressed in appropriate units.

[0023]

[0022] The approach described herein employs a different strategy, whereby a pattern or pattern framework specifically configured to represent a situation matching a person being monitored (PUM), its environment, and its health care profile (HCP) representing the person being monitored's current care state is used by the care processing monitoring system as a context for the evaluation and / or processing of data generated by one or more sensors monitoring the environment and / or the PUM. In this way, the pattern or pattern framework may incorporate a diverse range of sensors whose data outputs do not have a common normalization.

[0024] In some embodiments, the recognition of the pattern generated by one or more sensors may include a sequence of events and / or signals occurring over a period of time that may not be sequential in time. Such events and event sequences may include data from one or more sensors, a first sensor generating data that the care processing identifies as a change in the care and health status of the monitored person, and communicating the configuration change to one or more other sensors, either directly or in conjunction with the first sensor, to verify, confirm, and / or augment the data from the first sensor, thereby increasing the efficiency and accuracy of the determination of the event and / or event sequence in pursuit of identifying and determining one or more care signals indicative of a change in the care and health status of the PUM.

[0025]

[0024] The patterns or pattern frameworks developed herein may have non-linear, non-sequential, asynchronous, quantized and / or other time bases, and rather than capturing all data emitted by any set of sensors on a linear or sequential time base, the system may use an established quiescent state of at least one sensor set relative to the environment, incorporate one or more patterns of that environment, which may include the presence of a person being monitored for care (PUM), and evaluate any differences from that state captured by one or more sensors. In this way, the sensor data set may be evaluated in the context of at least one pattern operating in a care processing system including the monitoring process.

[0026]

[0025] This approach may include the use of nested, hierarchical, windowed, ordered, or other arrangements of patterns, whereby the care processing system deploys at least one pattern as a primary care processing monitoring pattern or pattern framework, and other patterns or pattern frameworks may be alternatives. These alternatives may be combined with one or more machine learning techniques to operate on the digital twin of the PUM and / or their environment. These patterns and / or frameworks may be dynamically swapped, whereby when the conditions of the environment change and the changes are consistent with two or more patterns or pattern frameworks, the monitoring system may use probabilistic analysis to determine which pattern or pattern framework is primary and which other patterns or pattern frameworks are secondary and / or alternative.

[0027]

[0026] Contextualization of data generated by one or more sensors in the environment includes a care signal processing system that assists in the contextualization. This is accomplished through the use of a comprehensive care framework, described herein as a Health Care Profile (HCP), that initially instantiates the HCP and includes a set of patterns that become populated using the sensor generated dataset to represent the behavioral patterns of the PUM in the environment. This approach provides the context for the care process to evaluate datasets of any type and complexity that assist in the delivery of care and health for the PUM.

[0028]

[0027] This can include the exchange of patterns and pattern frameworks within a comprehensive HCP that can dynamically respond to changes in the monitoring environment, and in some embodiments, a set of such patterns or pattern frameworks may have an associated weighting that represents the accuracy of that pattern or pattern framework to predict likely outcomes in the monitoring environment.

[0029]

[0028] Pattern specification may range from simple, monitoring the occurrence of a PUM condition as expressed by the HCP, in this case, e.g., coughing, which indicates difficulty in breathing, to more complex, where multiple sensor data are aggregated, e.g., where a PUM has multiple health conditions and / or has memory impairment. These patterns may be created from sensor data sets as the behavior of the PUM is observed and potentially replicated from other PUMs with similar health conditions and / or behaviors, and / or may be specified by one or more care village systems and / or authorized parties.

[0030] In some embodiments, multiple patterns may be operating simultaneously, with the same or different sensors providing data for each of these patterns.

[0031]

[0030] One aspect of the system is the way data from one or more sensors is interpreted. A single event, such a motion detection, can be evaluated in the context of a pattern that is actionable at the time. For example, if the pattern is "night sleep" and represents a person occupying a bedroom at night for the purpose of sleeping, the motion detection may be cached and when a water flow usage is detected and a second motion detection is generated, an event may be created and stored that may be represented as a token representing the bathroom being used at night.

[0032]

[0031] However, if motion detection does not generate other sensor data, at least one further pattern may be invoked, for example a night-time awakening pattern may be invoked, which may include configuring other sensors, such as smart light bulbs, to provide data indicating that a person is active.

[0033]

[0032] One aspect of care processing is the detection, identification and / or validation of care signals that at least partially represent a state of the PUM that may require action or response, including further sensing. These care signals may represent events and / or event sequences that represent, in whole or in part, the behavior of the PUM, corresponding to the care and health state of the PUM. The use of the PUM's care and health quiescent states can provide care processing in context for the detection and identification of such care signals by the care processing system.

[0034]

[0033] For any one or more sensors, there is a quiescent state from the perspective of a system monitoring the environment, for example, a care processing system, where the sensor is not providing data to the system or there is no change in its data. The sensor can have a state of operation and at least one of collecting, measuring, processing, storing, and / or transmitting data to a system that configures the sensor and establishes command and control of the sensor operation, such as a care processing system. The data generated by these sensors provide a representation of the sensed behavior of the PUM, and therefore these behaviors can be represented as patterns or pattern elements, which have a state in that the data provided by the sensor, for example in the form of a multidimensional feature set, can represent a state including a quiescent state of the behavior.

[0035]

[0034] The care processing system may evaluate the datasets represented by the multidimensional feature set to determine whether one or more of the datasets represented by the dimensions have a variance that exceeds one or more thresholds or other specifications utilized for the evaluation. This may include, for example, multiple sensor datasets that provide validation and / or confirmation of another sensor dataset to reduce false positives. In some embodiments, the care processing may configure one or more of the sensors that contribute data to the multidimensional feature set being evaluated to perform validation and / or confirmation, increase the granularity of the dataset, and / or invoke one or more other characteristics of the sensor.

[0036]

[0035] This can be evidenced by the variation of one or more behaviors exhibited by the PUM and so represented in relation to one or more pattern elements and / or movement patterns. This can include a unique, specific data set from one or more sensors that may not provide sufficient data alone to generate a care signal, but taken together with the multiple sensors, the configuration of which is adjusted based on the pattern elements representing the behavior of the PUM, so that the care signal is detected and identified. These care signals may be represented in the form of a multi-dimensional feature set, and the combination of these dimensions and the sensor data represented as the relationship between these dimensions forms the specification of the care signal.

[0037]

[0036] The care process monitoring system may configure one or more sensors in such an environment to increase the granularity, sensitivity and / or other configuration attributes of the capabilities of that sensor and / or other proximity sensors, activate other sensors from a passive state to an active state, and / or perform actions that require a response from the monitored environment and / or the PUM and / or other participants therein. This may include providing the sensor with one or more configurations that change the sensor's operating state and / or sensing capabilities. In this way, the focus of the monitoring may be adjusted to establish which of the patterns or pattern frameworks most accurately represents the current and / or possible situation in the environment.

[0038]

[0037] This can call for a further change in monitoring focus, such as other sensors, e.g. a smart speaker, being activated to determine a person's activities, such as reading, consuming a glass of water or food, etc., through monitoring, e.g., acoustic data of those activities, and / or through asking the PUM whether they are OK and what activities they are performing.

[0039]

[0038] Pattern identification and determination may be made from one or more sensor set data sets, which may include a complex set of signals, events, and / or data sets representative thereof. Pattern identification may include one or more machine learning systems, such as multi-layer neural networks, that may be invoked. These networks may be used to assist potential pattern placements that may be evaluated in one or more digital twins of the monitored environment and / or PUM to refine alignment of sensor data sets and behavioral pattern data.

[0040]

[0039] One aspect of the care village care processing system is the use of possible patterns for the PUM's behavior that may affect care and health as a framework for the sensor data to be evaluated by the care processing system. For example, if the PUM exhibits a behavior of constantly bumping into furniture, this may indicate that the PUM has a vision problem in addition to the situation being monitored. In this example, the care processing system may manipulate two patterns, the original situation pattern and the vision impairment pattern, to align the monitoring with the PUM's behavior. Once a match between the pattern and the PUM's behavior is established, the care processing may generate an alert to one or more stakeholders indicating that the PUM may need vision correction and / or assistance, for example, new glasses with a strong prescription. However, the data and patterns may also indicate that their current medical prescription regime is causing problems.

[0041] In some embodiments, a dataset of physical attributes of the environment and / or PUMs may be used to establish baseline data for one or more patterns. This may include establishing the state of the environment and / or PUMs, particularly with respect to the quiescent state of the environment and / or PUMs. Such a dataset may include relationships between the environment and participants, including one or more PUMs.

[0042]

[0041] A determination of an optimal data set to be collected from a set of sensors, each sensor having multiple features such that only certain features are selected, and attributes of these features, such as time / duration / signal resolution / data type / data size, can be configured in some embodiments to conform to one or more pattern specifications. This can include, for example, the selection of a specific sensor in a multi-sensor device, such as a smartphone, whose configuration may be changed by the care processing system, such as when the monitoring focus is changed, for example, to verify and / or confirm an event detected by one or more other sensors that are providing data to one or more motion patterns. For example, the focus and zoom of a camera in a smartphone can be changed to verify an event provided to the care processing by another sensor, such as an acoustic sensor.

[0043]

[0042] In some embodiments, the care signal processing system module can operate as part of a set of pattern frameworks to configure a set of available sensors. Data from these sensors can be retained in a repository, such as an elastic repository, for a period of time, as determined by the pattern framework specifications, which can form a set of reference data. This data can be used, for example, to establish a quiescent state of the environment, which can include the presence of a PUM.

[0044] Each of these data sets may be randomly sampled to determine whether the data is within the quiescent specifications of the pattern specification currently invoked for that environment. The rate of sampling, sample size, and evaluation process may be varied depending on the specifications of the quiescent pattern specification. In some embodiments, a set of criteria may be used to establish thresholds and sample rates appropriate for the situation being monitored.

[0045]

[0044] In some embodiments, at least one sensor may be configured to be an edge sensor, and the data set is processed and / or evaluated in the sensor device or in a connected device physically close to the sensor on a real-time, near real-time or event-driven basis. In some embodiments, this processing may be performed remotely in the cloud, provided that suitable communications are available. In some embodiments, this processing may be performed on the device containing the sensor, which includes one or more communication capabilities, e.g., wireless cell coverage such as 5G. In some embodiments, the edge sensor may be connected to a care hub or to other similar hub or router devices incorporating one or more communication capabilities, e.g., cell coverage such as 5G, PSTN using copper, cable, fiber, or other wired connections. In some cases, such devices may have multiple communication capabilities with a failover system to support the multiple communication capabilities. This edge sensor may provide state-of-the-art detection that may be supplemented, verified, and / or confirmed by other sensors that have an established and / or predetermined relationship with the sensor. For example, a microelectromechanical system (MEMS) microphone may be configured to listen for low frequency signals that are processed and evaluated at the edge sensor to detect events such as a step, and thus when such low frequency signals are detected, may communicate with other sensors, such as smart light bulbs, for example, at night when a nocturnal sleep pattern is in operation, or with other sensors having active sensing, such as frequency modulated continuous wave (FMCW) radar capabilities to determine a person's position, breathing, or other aspects.

[0046]

[0045] Depending on the capabilities of the sensor, the edge sensor may be configured to detect events and event sequences that may indicate changes in the environment and / or state of the PUM. This may include, for example, measurements of movement, e.g., stepping, walking, jerking movements, sudden movements, etc., as indicators of changes in the mobility of the PUM, clear changes in timing, e.g., time spent in the kitchen, bathroom or other locations indicating an activity taking more time than usual, changes in behavior including overuse or underuse of activities that are part of the quiescent state, consumption or other variations.

[0047]

[0046] The care processing system operates one or more patterns, each of which includes one designated edge sensor that generates data which is processed to compare with resting state pattern data of the environment and / or PUM, including any part thereof.

[0048]

[0047] In this approach to pattern determination, the complete environment and PUM are thereby considered as a set of states created from both the environment and PUM frameworks, based at least in part on quiescent conditions, and represented as a set of patterns including the combined behavior of the environment and PUM, forming a dataset of the care processing system. Although the care processing system may collect data generated by individual sensors, the use of one or more patterns significantly reduces the amount of data processing required to identify signals indicative of potential or actual care events. This approach allows edge devices such as sensors, hubs, wearables, etc. to perform the processing of such datasets at the edge. Such patterns may run on devices or sensors embedded and / or located in the environment, on dedicated and / or standard off-the-shelf devices, and / or on other hardware in close proximity to the environment being monitored. In some embodiments, such sensors, devices, and / or hardware may act as an aggregator of data and patterns located in the environment and / or remotely, such as in the cloud and / or remote hosting systems, cloud services, or other network systems, in any location.

[0049]

[0048] In some embodiments, each sensor may include access to a repository where any data from the sensor is stored. Such a repository may be an elastic repository that allows for storage of data sets over a period of time determined in part by the pattern being operated on. These repositories are described as elastic repositories. This data may be made available to the care processing system after an event or event sequence is detected and processed to identify characteristics of the pre-empted data related to the identified event. This process may be performed across several sensors, for example using machine learning techniques, and then incorporated into existing or new patterns for future deployment.

[0050]

[0049] The care process for the care system is configured to use a set of patterns that represent the behavior of the PUM in context, rather than simply collecting all data from all sensors. This approach involves separating the steady-state background sensor data, which represents the quiescent state, from those behavioral elements that are the context of the PUM as they pass through their respective HCPs.

[0051]

[0050] One aspect of the system is at least one device configured to provide event and / or event sequence data sets to one or more care signal processing systems. The sensors may be configured as edge sensors in a dynamic manner, for example, FMCW sensors may be configured as such in living areas and acoustic sensors may be configured as such in bathrooms. This dynamic transfer of edge capabilities may incorporate additional sensors having their configurations, including activation, deactivation, fidelity and / or other operational characteristics, that change as part of a movement pattern and / or change in response to data processed by the care processing system from at least one edge sensor. The configuration of each device may be determined, in whole or in part, by the care processing system, which may include devices including a set of devices, having pre-arranged and / or dynamic relationships with each other, and configured to transmit events and / or event sequences, some of which may be in the form of alerts, to other system elements, devices and / or interested parties, including PUMs. This may include configurations for transmitting aggregated and / or combined signals to larger or other configurations of local / edge / remote devices. Such configurations may change dynamically in response to observed conditions, patterns, events, and data sets.

[0052]

[0051] In this configuration, one or more sensors can be configured to optimize the output of such sensors, e.g., to increase the fidelity of the sensors, for detection or verification, including confirmation and / or validation, of an event and / or an event sequence. For example, this can include optimizing a MEMS microphone or other acoustic sensor and / or an active emitting sensor, such as an FMCW device, to detect whether an immobile PUM is breathing and how regular the breathing may be. This can indicate whether the PUM is exhibiting, for example, sleep apnea or other breathing-related problems.

[0053]

[0052] In some embodiments, data from individual sensors and / or sets of sensors may be verified and / or confirmed by data from other sensors included in the monitoring of the PUM and their environment. This may be the situation where, for example, a multi-sensor device such as a smartphone, smartwatch, etc. provides a set of data that may represent an event. This data from a single device may indicate a fall or other care or health event and thus trigger emergency or other responders, for example. However, the care processing system may receive further data sets from other sensors in the environment, such as acoustic, camera, tactile, FMCW or other active emission sensors, and thus verify and / or confirm the data set provided by the single device. This data verification and / or confirmation may be done within the pattern currently being monitored and may indicate that a transition to another pattern is taking place depending on the event and the verification and / or confirmation. This approach reduces the tendency of a single device and / or a single sensor data set to indicate an event resulting in a false positive. This may result in unnecessary escalation of an event resulting in EMT or other resources being deployed when not actually needed. Verification and / or validation may be performed on a sensor by sensor basis by care processing, and in some embodiments the results of this processing may be stored and used in different PUM and environmental conditions as well as provide training and / or comparison data for machine learning systems.

[0054]

[0053] In some embodiments, the care processing system may be distributed across multiple sensors, devices, hubs, and / or other hardware. This may include, for example, the use of feature recognition and other technologies that reside and operate on the sensors, devices, and / or hubs, such that data generated by the sensors may undergo processing to extract one or more features from the data captured by the sensors. For example, if a camera sensor is configured to capture edge features of the image being monitored, this data may be communicated to the care processing system if and when an edge that matches the PUM moves from vertical to horizontal. In some embodiments, the raw data feed may be stored in an elastic repository, for example, for a period of time that represents the human behavior being monitored, such as 5 minutes, 30 minutes, an hour or more, while the sensor processes the incoming data to extract edge features, which are then communicated to the care processing system. For example, the care processing system may then instruct the elastic repository, upon receiving the sensor processed data set, that the data indicates a change in orientation of the PUM, then trigger other sensors to verify this change, and mark the data retained from the original sensor for some degree of persistence so that the event under consideration can be investigated. For example, if the change in orientation was due to a fall, the camera data may be made available to one or more other parties and / or further care processing systems.

[0055]

[0054] The use of distributed care processing across multiple sensors, devices, and / or systems, including care hubs, supports the privacy of PUMs within their environments while providing effective monitoring of their care and health. Each sensor may have the ability to process the data received by that sensor, for example using feature extraction, so that the sensor may communicate only extracted features to the monitoring system while simultaneously storing the raw data in a repository. In some embodiments, this communication may be in the form of a token. This allows the monitoring system to determine whether a feature set matches or meets criteria for a care and / or health event compared to a behavioral pattern, while maintaining the privacy of the PUMs through non-disclosure of the raw data. In some embodiments, such raw data may be made available to authorized and authenticated parties, such as medical professionals, EMTs, emergency responders, etc.

[0056]

[0055] One aspect of processing data generated by one or more sensors, devices, and / or systems is the use of distributed processing across multiple processing capabilities. For example, this can include processing on the sensors and / or devices, which may include, for example, feature identification, classification, and / or extraction, etc. In some embodiments, such sensors and / or devices may have access to additional processing capabilities, such as a local care hub and / or other co-located and / or remote, e.g., cloud-based systems.

[0057]

[0056] A further aspect is the deployment of a distributed decision system where the configuration of one or more sensors is determined by one or more other sensors, and the configuration may be part of a decision process initiated by one or more modules, devices, and / or systems, such as a care hub. For example, when monitoring focus increases in response to changes in motion patterns and / or pattern elements, the configuration of the sensor, for example by the care hub, may include utilizing feature extraction, identification, and / or recognition to enhance the fidelity, accuracy, and / or timing of the sensor operation, and the change in configuration may trigger further configuration changes of other collocated, logical or physical sensors, whereby the data set of the first sensor is enhanced to include being verified, confirmed, or verified by the other sensors that aid in the aggregated data set responsive to the initial decision process of the triggering module, device, and / or system, such as the care hub.

[0058] 14 illustrates one embodiment of a Care Hub (1001) utilized as part of monitoring the PUM (105) in the environment (104), comprising a monitoring module (1403), a processing module (1405), a prediction module (1404), a pattern module (1406), a decision processing module (1402), and a response system module (1401), all of which may include one or more sub-modules that may be local or remote, by reference or by embedding. For example, the sub-modules may be included in the Care Hub as hardware instantiations, including protected processing, secure encrypted storage and enhanced identity, processing, key management, and other security features, for example, to ensure that sensitive information, including communications, is protected.

[0059] Such an approach may include a distributed decision process that identifies one or more sensors as operating in an erroneous or failed mode and may be reset and reconfigured accordingly, and / or the generated data set may be ignored or assigned one or more attributes that evidence a fault condition, which may then be reported to one or more systems for fault management.

[0060] In some embodiments, Byzantine and / or consensus algorithms including similar approaches may be utilized to both identify and / or configure such sensors in any deployment.

[0061] In some embodiments, tokens may be exchanged between sensors and / or devices operating in a quiescent or other operating state, each token via a reference or embedding that includes the token itself as an instance of such operating pattern and / or pattern element state information, and through this exchange of tokens between such sensors and / or devices, this state may be maintained across multiple sensors and / or devices in the environment. For example, a token may include configuration specifications for one or more sensors and / or devices, such that specifications distributed by one or more decision processing processes include those that include the sensors and / or devices themselves.

[0062] This use of tokens may support privacy and confidentiality of information communicated between sensors, devices, systems, and modules that comprise a monitoring system within an environment.

[0063] In some embodiments, patterns may be determined in context with identification and transmission through the use of tokenized instantiations of such patterns.

[0064]

[0063] Figure 2 shows an HCP representing the caregiving journey of a PUM from an initial caregiving state, which represents an initial caregiving state for monitoring, to a series of caregiving states leading to a palliative hospice caregiving state and finally to an end-of-life caregiving state. The caregiving pathways shown herein may not be linear, may not be of the same duration as the HCP states shown herein, may not have the same transition states, or may not have the same severity of caregiving state decline, but rather this caregiving journey is likely to be person-specific for each individual PUM. However, commonalities of HCPs across many PUMs with the same status can be evaluated for patterns and behaviors that are at least partially evident in those HCP statuses.

[0065]

[0064] Figure 3 shows a further HCP where the PUM recovers to at least the initial condition that caused it to be placed under surveillance. Such a condition may be part of the HCP journey of the PUM, for example as shown in Figure 2.

[0066]

[0065] Behavioral patterns

[0067]

[0066] In almost all environments, such as home environments or care facilities in which PUMs reside, the behavior of people within the environment exhibits certain patterns. For example, the use of kitchen and bathroom facilities may have specific timings for use, with dwell times for each falling within certain parameters. Further examples include bedrooms, kitchens, and living areas, where there may be consistent dwell times for activities such as sleeping, cooking, watching television, reading, searching the Internet, etc.

[0068]

[0067] One aspect of behavior is the monitoring of activity and dwell times to establish patterns of people's movement within the environment. This can include monitoring entry and exit information to specific areas, e.g. bathroom, kitchen, etc., as well as movements between these different functional areas.

[0069]

[0068] One challenge in any care process is the recognition of a change in state of a monitored input that is an indication of an event that has occurred or may occur. For example, in an HCP environment, this may include the recognition that a user has tripped over an existing edge on their floor, or a piece of furniture such as a couch, and storing this data in one or more sensors, for example sensors embedded in devices such as accelerometers and / or acoustic monitoring devices, cameras, etc.

[0070]

[0069] In this example, these devices may store data and poll that data by edge devices or other system monitoring processes. Evaluation of this data may be performed within a pattern framework, where a set of known precursors to an event such as a fall is an increase in the number and rate of incorrect steps a person may take within the environment.

[0071]

[0070] In such an example, the system may be configured to alert a caregiver, family member, neighbour or other concerned person to this occurrence so that corrective action may be taken to avoid a possible fall. The system may calculate the probability of a fall from this dataset and advise the person in the environment to stop or reduce movement until help can arrive. This advice may be communicated via a portable or wearable device, a smart speaker, a smart TV or any other suitable device in the environment.

[0072]

[0071] Such exemplary circumstances may include the activation of one or more other sensors, which are configured to increase the monitoring focus and take a closer look at the person and the environment. This may also include the configuration and activation of devices that provide medical or other health monitoring of the person and the environment, such as blood pressure monitoring, temperature and climate control, etc.

[0073]

[0072] This may include configuration of devices to monitor the environment in a manner consistent with the events being monitored. This may include using different configurations of devices and different arrangements of sensors to create a data set suitable for performing an evaluation of the system and / or for transmission in an appropriate format to one or more interested parties. Depending on the environment and the relationship of this data set to the person being monitored (PUM), the system may trigger different patterns and pattern frameworks, such as responses that may be required prior to or upon the arrival of a caregiver, medical and / or response team and / or the like.

[0074]

[0073] If the data set is sufficiently aligned with the preformatted event sequence response configuration and / or is within a median deviation or other threshold for the predictive model, this may cause the system to invoke one or more patterns in response to the data set and the situation it represents. This may include matching the detected patterns of behavior to the preconfigured response configuration.

[0075] Encoding Behavior as a Pattern

[0076]

[0075] Certain behavioral characteristics forming at least one pattern may be monitored and any arrangement of behaviors, sequences of behaviors, events or sequences of events may be matched in whole or in part to patterns of such behaviors and / or events stored by the system. These arrangements may include hierarchical, sequential, subordinate, etc.

[0077]

[0076] These stored patterns may have response configurations that respond, in whole or in part, to these identified and monitored behavior and event patterns. This may include a set of configurations that are deployed to sensors in the environment in response to data from one or more sensors. In some embodiments, specifications may be stored and recalled when a particular behavior is exhibited and / or matches one or more stored patterns. This may include events and alerts to one or more interested parties and / or other systems that may provide one or more responses.

[0078]

[0077] Matching the monitored patterns with the stored patterns may result in varying degrees of certainty regarding the matching of these patterns. For example, a pattern may match 6 out of 7 behaviors and 4 out of 5 events in a time period common to both patterns. This may generate a pattern matching matrix, and the system may invoke further care processes and / or configure further sensors to verify and / or confirm such pattern matching.

[0079] In some embodiments, a determination of repetitive behaviors, such as where a person regularly sits, when a person prepares meals, when a person uses the bathroom, when a person walks, etc., may be identified as pattern elements that at least partially represent the daily behavior of the PUM. Such elements may represent part of the stationary state of the pattern, e.g., repetitive occurrences form a sequence of PUM behaviors. In some embodiments, these repetitive behaviors may be designated as pattern elements. For example, behavioral changes in one or more of these repetitive behaviors and / or sequences of such behaviors may represent an indication of a transition state such that the PUM is transitioning from any pattern to another pattern. This may include situations in which the pattern the PUM is transitioning to may be one of several potential other patterns. In this example, the digital twin may be used to generate additional instances of each of the possible patterns the PUM may transition to. The care process may then deploy and / or create configurations of sensors that may be used to verify, confirm and / or notify the most likely pattern candidates so as to optimize monitoring and / or detection of the patterns and the behaviors represented thereby. This approach allows for the determination of which pattern best matches the situation.

[0080] In instances where there are two patterns with equal likelihood, the care process may configure the sensors to provide sufficient data for both patterns to detect as soon as possible which pattern best represents the actual event unfolding. This may include creating alerts, messages and / or other data sets that are sent to appropriate parties and / or other systems, and may include certain pre-configuration, such as determining the location of specific parties relative to the PUM, and calculating timing and other metrics to assist in the care of the PUM, for example.

[0081] In some embodiments, edge devices and / or other sensors may have their data output directed to match a set of pattern elements. Each pattern fragment is part of a pattern framework and / or a set of patterns, all of which are part of an HCP or a pattern indicating a high probability of a transition between one HCP and another of two HCPs. The deployment and operation of these fragments may be managed by a care processing module and may be operated on sensors and devices embedded in the environment and / or a digital twin of such an arrangement.

[0082]

[0081] Figure 4 shows a set of modules for monitoring a PUM (105) in an environment, including an HCP (101), a behavioral pattern (402), and elements thereof, such as a combination of a pattern framework (401) and pattern elements (102), can be combined to form a care signal processing system that may include configuration and associated command and control functions to support effective monitoring of a PUM (105) in the context of an HCP (101). This may include an environment (104) in which the PUM is being monitored, in which one or more events, including a sequence (403), may occur. In some embodiments, a behavioral pattern (402) may be represented, in whole or in part, by one or more pattern elements (102) that may form a behavior pattern (103). In some embodiments, a behavioral pattern or other patterns may form pattern elements of further behavioral patterns, for example, hierarchically.

[0083] One aspect of the system is the use of multi-dimensional feature sets as a full or partial representation of the behavioral patterns of the PUM, expressed as patterns or pattern elements. These feature sets comprise multiple sensor data sets including relationships between the data sets from one or more sensors embedded in the environment.

[0084]

[0083] These data sets can be represented, for example, using a manifold, a Hilbert space, or other representation that can store an individual data set from a sensor and its relationship to data from another sensor. This relationship can include data sets from multiple sensors, for example, temperature sensors, acoustic sensors, and motion detectors, for example, when the PUM is sleeping, the resting state is represented, or the resting state is represented by a movement pattern, for example, a nighttime sleep pattern.

[0085]

[0084] As features are defined as relationships between multiple sensor data sets, these relationships can form feature sets that are representations of aggregated data of one or more sensors, e.g., represented as multidimensional feature sets. Such relationships may be represented, e.g., as ratios, functions and / or algorithms, spatial and / or graph-based representations, etc. In this manner, feature sets that represent relationships between two or more sensors can be used to determine the state of the PUM in the environment. Such relationships may include one or more thresholds, variances, or other data sets to accommodate variations in the sensor data. For example, relationships between acceleration in three axes and the position, height, and attitude of the PUM can be evaluated to determine whether a fall or minor trip has occurred. These sensor data sets may have further relationships with sensors to detect audio, vision, respiration, heart rate, or other sensory data sets. The combined evaluation of these data sets in the form of multidimensional feature sets can include both sequential and / or parallel, e.g., event sequences represented by the sensor data sets.

[0086]

[0085] Because the set of relationships represented by the multidimensional feature sets can provide a framework within which variations in individual sensor data sets can be evaluated, at least in part, by their relationships to one another and by their correlation to the monitored behavior of the PUM that such feature sets represent, such feature set evaluation can be used to detect transitions from any pattern or pattern element to another pattern or pattern element.

[0087]

[0086] Because each sensor data set may vary, even though a sensor may include and / or have access to an identification and / or processing of feature sets, the usefulness of these changes as metrics for evaluating an alert, event, or response is limited, including by the capabilities of the sensor. On the other hand, the variations exhibited by the combined feature sets of multiple sensors, particularly in the relationship of one or more sensors to one or more other sensors, may provide an accurate and comprehensive representation of the deployment state of the PUM. These relationship changes may form an indication that the PUM is transitioning from one operating pattern to another.

[0088]

[0087] In some embodiments, each pattern or pattern element includes a combination of data from one or more sensors representative of the behavior of a PUM in an environment.

[0089]

[0088] A feature set may include multiple dimensions, and each of the sensor data may form, in whole or in part, a dimension of the feature set. These dimensions may be represented in one or more multi-dimensional feature sets.

[0090]

[0089] One aspect of PUM behavior is the routines of daily life including, for example, sleeping, eating, using the bathroom, exercising, entertainment, social life, etc. One aspect of cultural behavior is broadcast such as TV and radio, and the Internet including streaming and interactive and other content, etc. As with many other human behaviors, the timing, selection, duration, and other media or cultural behaviors may contribute, in whole or in part, to forming patterns and / or pattern elements that are representative of PUM and / or other participant behavior.

[0091]

[0090] For example, the digital patterns of PUMs, such as their viewing of Netflix or other streaming services and / or their internet searches, may indicate and / or reveal changes in health and wellness. However, such information may be highly revealing about the PUMs and / or other parties, and thus this PII, if made widely available, may represent a significant privacy risk.

[0092] In some embodiments, the care hub may act as an aggregator of one or more sensors involved in monitoring the digital interactions of the PUM to monitor patterns and / or pattern elements indicative of its behavior. In this manner, the data may be evaluated to determine care and health impacts while protecting the privacy of the PUM and / or other parties through encryption of the data and limiting distribution of the data. This may include removal of source data after patterns are extracted and / or identified, and may include the use of tokens to represent such data, patterns, and / or pattern elements in any arrangement.

[0093]

[0092] Figure 5 illustrates the integration of a set of response systems (502) with a care processing system (507), which in turn is integrated with appropriate participants (508) of the PUM (105). This may include participants (508) such as emergency responders and systems, caregivers, family, neighbors, friends, medical professionals, etc., in any arrangement. The response may be derived in part from one or more specifications of the care situation state represented by the care processing system, and may include, for example, a configuration of sensors with different monitoring foci for different participants in any arrangement. The monitoring system (501) is integrated with environmental monitoring sensors (104) and a response system (502). The monitoring system may interact with one or more predictive systems (503), machine learning modules (504), and digital twins (505), which may in part determine potential patterns (506) and / or pattern elements that may be instantiated in whole or in part as behavior patterns (103).

[0094]

[0093] Reference Data Set

[0095]

[0094] To establish a baseline for effective processing of multiple sensor data sets in an environment, reference data sets are created for these sensors individually and in combination. One important aspect of the system is to establish a "rest" state of the sensors in the environment. This involves configuring the sensors to have a rest or quiescent state that incorporates the sensors measuring the environment when there is no activity. Thus, each sensor generates a data set that can become part of the reference data set for the environment.

[0096]

[0095] The reference data set may include states in which the person under monitoring (PUM) is present, activities such as sleeping, watching television, eating, self-care, etc. being performed, and / or other predefined or measured data sets. These data sets may form pattern elements and / or represent repetitive behaviors of the PUM.

[0097]

[0096] One aspect is the integration of data sets from different devices, for example a sensor measuring temperature and another sensor capturing an acoustic signal. Integration of these data sets is typically performed by normalization, but when the metrics used for each sensor are sufficiently different to not have a valid equivalence, the integration is performed in the context of patterns and / or pattern elements that represent the exhibited behavior. These are described herein as behavioral patterns.

[0098]

[0097] In this way, completely different data sets may be integrated by the care signal processing system to provide a consistent and reliable measure of the state of the environment in relation to one or more patterns that integrate the individual sensor data sets. However, the determination of all possible combinations of data sets into integrated sets and patterns always has an N+1 problem in that the load is likely to be high and there is the possibility of any pattern that has not yet been identified. To solve this, the system uses a reference data set in combination with an environmental specification, for example in the form of a digital twin, in combination with machine learning for the identification, classification and / or storage of these additional patterns.

[0099]

[0098] The integration may be represented using a number of techniques, including, for example, graph databases, Hilbert spaces, Riemannian or other manifolds, where individual data sets from one or more sensors are represented as relationships to another sensor data set. For example, if a reference data set represents a PUM performing a repetitive behavior, and the date of that behavior is within any threshold, e.g., pattern element, of the resting state of that repetitive behavior, then the individual sensor data represented by the metric of that sensor may have a relationship to another sensor data set that simultaneously monitors the same repetitive behavior, such that the relationship between the potentially different metrics of the sensor includes the metric of the resting state. This may include a multi-dimensional representation of the environment and the behavior of the PUM within that environment, such that the pattern or pattern element provides a sufficient metric, whereby the alert, event, and / or response system may determine that one or more relationships between the sensor data sets have or may violate one or more thresholds that the processing system is configured to represent the state of the environment and the PUM.

[0100]

[0099] These threshold conditions and associated configuration representations can include using digital twins and machine learning in combination separately to determine the probability that the state of the pattern or pattern elements will change. This can include dynamic adjustment of the threshold configurations and of the response system response placement specifications for the prevailing situation. For example, if the outside temperature is too hot or cold, the configuration may modify one or more thresholds in light of changes in the PUM's behavior, such as adding or removing clothing, shifting position, changing HVAC settings, etc.

[0101]

[0100] Figure 15 shows a PUM (105) within an environment (104) in which one or more behavioral patterns (103) are deployed and, in conjunction with a monitoring system (107) and a monitoring focus module (601), a health care condition is identified that calls a response system (1301), which may have predefined and / or dynamically created and / or modified response deployment specifications (1501) utilized, such that an appropriate response is executed by one or more participants (112).

[0102] In some embodiments, this reference data set may be a snapshot of the state of the environment and the PUM therein, and may include data generated by each sensor, individually and / or aggregated in any arrangement. It may also include a data set accumulated over one or more time periods. It may include establishing a quiescent state of one or more sensors in the environment. Such a snapshot may be persisted and used, in whole or in part, as a corpus for one or more machine learning systems.

[0103]

[0102] This may also include specific relationships and configurations of particular sensors with other sensors, such that the combination of sensors and configurations provides a collective capability focused on specific PUM behaviors, including, for example, patterns and / or pattern elements. These configurations may allow these sensor sets to operate at different granularities and resolutions, such as when the PUM and environmental conditions are stationary, to protect the privacy of the PUM.

[0104]

[0103] There are several data relationships with parameters that are well understood, such as parameters that represent physical laws and other measurable results. These relationships may be defined as algorithms and used in typical configuration and data normalization processes.

[0105]

[0104] The care processing system operates a set of patterns that integrates data sets generated by one or more sensors in the environment. This can include one or more patterns that are determined to be operational at the time. However, further sets of patterns can integrate the same data sets with other patterns and then be evaluated to determine the most likely representation of the situation occurring in the environment at the time. This processing can be performed through the use of a digital twin in combination with a machine learning system.

[0106]

[0105] The use of a set of criteria representing the state of the environment, the PUM, and the actions currently operating can provide the framework within which this processing and evaluation takes place.

[0107]

[0106] The reference data set may need to be updated in response to changes in the environment, changes in the state of the PUM (recovery, degradation, aging, learning, etc.), and / or addition, removal, updating, and / or replacement of sensors.

[0108]

[0107] In some embodiments, the reference patterns and pattern framework may incorporate medical diagnostic information, such as that commonly used by medical professionals to identify specific health and care diagnoses. This may include certain thresholds, metrics, behaviors, or other indicated characteristics of the care recipient being monitored in the environment. In this way, medical professionals may be able to monitor the person for certain behaviors and characteristics and receive alerts or data sets when such behaviors and characteristics are identified by the system. In some cases, this may include alerting other parties involved in the person's care and potentially invoking action and response by those parties.

[0109]

[0108] In some embodiments, the care processing system may use sampling techniques for data generated by sensors when the system state is stationary. Such an approach may increase efficiency and privacy.

[0110]

[0109] Surveillance focus

[0111]

[0110] The care processing system can operate as part of a monitoring and control system that can provide and / or support resources such as devices, sensors, computers, storage, machine learning, algorithms, etc. to configure and control each of the sensors and / or devices in the environment and enable this functionality.

[0112]

[0111] The monitoring focal system, in some embodiments, forms part of the care processing system and provides a dynamic capability to change the overall environmental sensing capability within the capabilities of individual sensors and / or their collections to focus on one or more aspects of the environment and PUM. This can include aggregation and accumulation of data from multiple sensors to form a unified pattern that can provide a detailed data set of the environment and PUM that can include a multi-dimensional feature set.

[0113]

[0112] A further aspect is the delegation of configuration of the monitoring focus module to authorized and authenticated parties, such as medical professionals, emergency response teams, caregivers, etc.

[0114]

[0113] One aspect of the system is the distributed nature of the configuration of sensors in that with a sensor having at least one relationship with another sensor, a first sensor may configure a second sensor to perform detailed, focused, fine-grained, high resolution or other configured operations to generate a dataset that, in combination with an initial dataset from the first sensor, comprises a complete, accurate, and / or information-rich dataset. This may include each of the sensors increasing the amount of data they are generating, the quality of other attributes, etc., which may be expressed as dimensions of their contribution to the multi-dimensional feature set.

[0115]

[0114] In FIG. 6, a monitoring focus module (601) interacts with the monitored environment (104), which may include pattern elements (102) and behavioral patterns (103), as well as predictive systems (503), machine learning (504), latent patterns (506), and digital twins (505) of any configuration.

[0116]

[0115] In some embodiments, one or more digital twins may be operating with one or more patterns in operation, which are at least partially composed of sensor data from the currently deployed operational patterns of the monitoring system. In combination with the application of one or more machine learning techniques, this may result in the detection of one or more features not previously identified and / or classified. For example, this may include data from one or more sensors over an extended period of time, e.g., a period that exceeds the capabilities of that sensor, including an accessible repository for storing that data. For example, this may be weeks or months, and the digital twin may provide. This may include data and / or configuration information from multiple sensors that have not yet been combined, aggregated, and / or evaluated as a set by the care processing system. These new identified features may be represented by additional dimensions that may be integrated into the existing multidimensional feature set and / or may be represented as dimensions of a new feature set. In some embodiments, the system may predict and assume that data from one or more sensors may be represented as features of that data, e.g., events, actions, or other identifiable attributes. In this manner, such features may be passed to the motion pattern and evaluated for accuracy of the motion of the sensor using one or more sensors, including assistance data from other co-located sensors with which the first sensor has or establishes a relationship. These features may also be created through extrapolation and / or interpolation of data sets from situations deemed to have sufficient equivalence, as determined in part by one or more machine learning and / or statistical techniques. In this manner, the experience of a PUM in any environment may be correlated to other PUMs in other environments, and the feature set, including the multi-dimensional feature set and their respective representations, may provide further information data about the situation.

[0117]

[0116] Figure 13 shows a PUM (105) in an environment (104) where sensors (S3 to S7) generate a dataset (1301) that may be processed by the care processing (1201) to at least partially form a multidimensional feature set (1202), which may then be used by the digital twin (505) and predictive system (503) to create, enhance, modify, and / or confirm stored patterns (1302) that may be utilized by the HCP (101).

[0118]

[0117] Such features may also be used by the care processing system to verify and / or validate existing data, for example, to establish the trajectory of the care situation of the PUM in the environment and / or to modify the configuration of the sensor set and / or the care processing system for the purpose of care management, including the placement of care processes, sensors and / or participants to mitigate predicted or anticipated care situations.

[0119] In some embodiments, devices in the environment may have enforcement capabilities, such as overrides, that allow the device to be configured to be active using at least one sensor of the device, which may be calibrated to provide a data set within the capabilities of the sensor and device. This configuration and control may be performed in an emergency situation where both the device and the system have pre-agreed specifications, instantiated, for example, through a common API, regarding the declaration of an emergency. In some embodiments, this may include the system and the device exchanging tokens that include embedded and / or referenced information. Such an exchange may include one or more credentials, authentication, authorization, and / or access control specifications, and / or enforcement mechanisms.

[0120]

[0119] A combination of sensors communicating with other sensors to integrate data sets, where a first sensor matches a data set to at least a portion of a pattern, and then communicates with a second sensor in the same environment, providing data already collected and / or configuring the sensor to begin collecting data so that a care processing module may integrate both data sets with the intent of verifying or validating the first sensor data set as presenting an event that includes a portion of an event sequence that matches with high accuracy to an element of the pattern, providing a rigorous and accurate determination of the conditions occurring in the environment and the PUM residing therein. Such an approach may include the use of a multi-dimensional feature set that is used in whole or in part to evaluate the input data sets and establish any changes in the behavior pattern and / or pattern elements.

[0121]

[0120] This combined information set may be provided to one or more operational digital twins of the environment and the PUM therein to identify further data that may be expected from other sensors in the environment. This may include the care processing module and / or monitoring system modifying the state and / or configuration of such other sensors, which may accurately determine the deployment event. This may result in further focusing of the monitoring capabilities of the environment and / or initiation of at least one response arrangement, where the event indicates a care incident of sufficient severity to warrant such a response. This may include a predictive response, such as alerting neighbors or other parties to assist the PUM.

[0122]

[0121] The sensor care processing system and / or device can be used to receive, process, store, and aggregate signals from one or more groups of sensors. One or more signals from a single sensor or a group of sensors can be used to determine that a change has occurred in the current operating pattern state of the PUM. This pattern change can then trigger the activation of additional sensors and / or a change in the configuration of one or more existing sensors to update the current monitoring focus to the new PUM state. This trigger can occur in one or more sensors, or other components of the sensor care processing system, individually and / or in combination.

[0123]

[0122] For example, if a PUM gets up and starts walking after sitting or lying down for some time, a signal from an active accelerometer in a wearable device such as a PERS or a smartwatch can detect the change in the PUM's movement, which can trigger the activation of a barometric altimeter sensor to accurately detect a possible fall. The change in the PUM's state pattern can also trigger a change in the configuration of one or more of the same sensors that detected such a change. For example, the same accelerometer that detected the change in the PUM's movement pattern can be automatically reconfigured to increase its accuracy and / or its signal update frequency to detect possible falls, not just general movements. For example, this can include monitoring the PUM's gait as it moves, in order to assess, in whole or in part, the likelihood that the PUM will fall or have another health or wellness event. In some embodiments, the change in monitoring focus can be determined in whole or in part by the invocation of one or more machine learning techniques.

[0124]

[0123] Configuring one or more sensors to increase the fidelity and / or granularity of their sensing capabilities enables event sequence detection that may result in accurate and / or early detection of adverse and / or other care-related conditions.

[0125]

[0124] This approach can also be used with one or more redundant sensors to detect sensor failures and avoid resulting false positives or false negatives. The use of multi-dimensional feature sets aids in identifying sensor aberrations or failures in that the relationships between feature sets diverge sufficiently to generate events representative of such divergence. For example, if a fly, spider or other insect is present on the sensor, covering it, this can result in significant differences in the data set forming the dimensions and therefore in the relationships with other co-located sensors.

[0126] In some embodiments, once an event is determined to have occurred, one or more processing systems may be invoked in real-time to evaluate relationships of a set of dimensions represented in a multidimensional feature set. This may include the use of different processing systems utilizing one or more algorithms for different data sets, dimensions and / or dimensional relationships in assisting in the monitoring focus or evaluation of one or more event types and / or patterns, pattern elements, and / or behaviors of the person and / or environment being monitored.

[0127] In some embodiments, for example, when the data contributions of the sensors are mediated by the care process in a dynamic manner, data may be shared across multiple sensors based on contextual instances. This may include passing a dataset from any sensor to another to improve the performance of the sensor and / or to clean its dataset and / or to act as a feedback mechanism to reduce noise in the dataset.

[0128]

[0127] Pattern processing

[0129]

[0128] Pattern processing can be performed using a configuration and / or data set of a set of sensor devices embedded in the environment. Distribution of processing power and functionality can be included, for example, among the sensors and / or devices in which such sensors are embedded, hubs or other devices located in the environment, such as care hubs, and networks connected to and / or accessible to the sensors, providing access to the cloud and / or other accessible processing power, including specialized processing systems in any location.

[0130]

[0129] Many sensors incorporate feature extraction techniques that identify certain characteristics of the data and in some embodiments communicate only these features. Such sensors may be incorporated into the care process, but these features may be verified and / or confirmed by other sensors in the same environment to minimize false positives. Some sensors may be configured with additional feature sets, such as features generated in a digital twin using predictive techniques and / or machine learning capabilities.

[0131]

[0130] In some embodiments, the system monitoring and / or care processing may be self-learning in that the environment is first sensed to establish a baseline, which may be represented as a pattern framework. Then, a set of patterns, such as the patterns included in the HCP representing PUM care monitoring, may be loaded into the care processing system. This may include, for example, each pattern having one or more event detection criteria, represented as a multidimensional feature set, and the sensing system may monitor each of these criteria independently and collectively. In some embodiments, when an independent event is detected, a probabilistic method is used so that the care processing is performed to predict the probability that other sensors will generate results matching the corresponding event criteria, which may be a consensus algorithm used to determine whether the event sequence is sufficient to prompt segmented monitoring, for example, whether it is sufficient to increase monitoring focus and / or whether it is sufficient to trigger or alert the issue for further escalation.

[0132]

[0131] The care signal processing configuration may be based on and / or derived from an ML model developed from at least one digital twin representing the environment, the PUM, and / or participants of any arrangement.

[0133]

[0132] The selection and configuration of edge devices is responsive to the initial deployment of the pattern, whereby the edge devices may be located in a portion of the environment occupied by the PUM and / or are selected based on sensors in the device and / or are retained by the PUM and / or by other criteria specified in the pattern. In many situations, edge devices may comprise a set, such as one device and / or sensor for each portion of the environment, such as each room in a multi-room environment. In this example, as the PUM moves around the environment, the edge devices of the pattern currently operating are selected based on position changes, such that at least one edge device actively monitors at least one event, event sequence, or other data that matches the operating pattern. This situation is similar for at least one digital twin operating in the same pattern, as well as other instances of digital twins that may operate in other patterns that are deemed more likely to be invoked in the short-term time frame.

[0134]

[0133] Some embodiments may have an edge device that incorporates sufficient configuration, processing and storage to retain sensor data related to motion patterns such that only data matching a motion pattern specification is retained and / or stored in an appropriate repository. In some embodiments, such data may be used in whole or in part for the generation of tokens.

[0135]

[0134] A pattern specification can include pattern elements, considered as pattern elements, which can be combined to form new and / or derived patterns. These can include pre-formatted event and event sequence elements, such that if a data set from a set of sensors exceeds a threshold for a particular amount of time, and, for example, a second set of sensors provides a data set that validates this occurrence, then a pattern matching algorithm is invoked.

[0136]

[0135] In some embodiments, the patterns and / or pattern elements can be mapped to the devices such that the devices evaluate the data set and communicate the results of such evaluation, for example as a token, to the care processing module. Such evaluation and communication can be performed even if the device is not currently acting as an "edge" sensor in an array of sensors controlled by the care processing system. In this example, the care processing may dynamically integrate communications from this sensor, change the state of this or other sensors, and configure further sensors to verify, confirm, and / or provide additional sensory data that fits the behavior pattern. This can include defining a typical pattern for a given HCP, pattern, pattern elements, and at least one device is dynamically configured to provide at least one "edge" of the signal of the event.

[0137]

[0136] In some embodiments, at least one edge device can trigger other sensors, devices, and / or systems for verification or other data sourcing.

[0138]

[0137] In some embodiments, such care signal processing across multiple devices for "edges," which may include, for example, multiple data sets from multiple devices over a limited period of time as a multidimensional feature set, can improve accuracy with respect to event sequence identification and reduce the likelihood of false positives.

[0139]

[0138] This approach can distribute the computational load across multiple local and remote processing resources to improve efficiency for processing large numbers of signals in complex systems.

[0140]

[0139] At least one algorithm may trigger the selection of data from at least one sensor such that each data set is then written to the distributed ledger in a sequence representing an event, providing verification of the occurrence of the event, which in some embodiments may obscure the identity of the actual event.

[0141]

[0140] In some embodiments, active scanning sensors and / or transmitters and passive sensors and / or receiver sensors can be integrated to create an integrated source data set for applying one or more algorithms for detection and identification of care-related patterns and / or pattern elements of one or more persons.

[0142]

[0141] Care processing classifier

[0143]

[0142] One of the many challenges of accurate, timely and rapid care processing and / or assessment is the classification and typing of processed signals into appropriate data structures that can inform, direct, configure, arrange and / or command other system elements. In many current situations, care processing is associated with a robust and static rule-based system where if signal A is received by the system, then rule set B is activated. This robust approach is often less useful for real deployment situations where the rule-based response is inappropriate and may impose constraints on the current state. Often these rule sets invoke responses that are too many or too few resources and / or measures depending on the state.

[0144]

[0143] The approach described herein includes the use of a set of classifiers for different signal types and the support of configuring sensors related to those classified care processing data sets. This can include a classifier for a multidimensional feature set, for example, the classifier may have a classification schema that matches a dominant set of data represented by the multidimensional feature set. In some embodiments, the use of digital twins and / or machine learning techniques may be utilized to determine the classification of such data sets, including multidimensional feature sets that may include incomplete and / or partial feature sets, and / or dimensions and / or their dimensional relationships. Many more common care-related conditions and situations, including those represented as patterns and / or pattern elements, can be predicted and specified as part of a pattern framework for care processing, such that each sensor can contribute to one or more pattern frameworks that can be used to accurately identify the situation and optimize one or more responses.

[0145]

[0144] In some embodiments, the classifiers may be dynamic, and classification operations are typically performed prior to their use, but may be responsive to pattern frameworks, including multi-dimensional feature sets that populate these frameworks to form patterns, and / or pattern elements and / or sensor data. For example, a set of sensor data may be determined by the care processing system to be part of a pattern such as movement detected in a bedroom during sleep, followed by use of the bathroom, followed by a return to sleep-related breathing. This set of data may also be considered part of a further pattern, the occurrence of this sequence of events being associated, for example, with a change in temperature of the environment, an external factor such as noise, a breathing abnormality such as sleep apnea, etc.

[0146]

[0145] In some embodiments, the patterns may be stored, for example, in a graph database, and further candidate predictive patterns, for example from predictive systems and / or digital twins, may also be stored.

[0147]

[0146] For example, an environment, being an enclosed space, may be modeled as a Hilbert space, etc., using the inner product of a set of vectors (x,y). In this manner, a multi-dimensional model of an environment, including at least one person, may be created and dynamically adapted in response to contextual changes in the environment. This may be accomplished through sparse sampling of a set of such sensors, and the use of at least one sensor as an edge sensor for the environment, without the need to monitor all inputs from multiple sensor arrangements in real time.

[0148]

[0147] This approach may also be used to evaluate multi-dimensional feature sets, evaluating dimensions and dimensional relationships to determine actual or potential transitions from any pattern or pattern element to another pattern or pattern element.

[0149]

[0148] The sampling used by the system may be based on patterns, pattern elements and / or pattern frameworks, using the HCP of the person being monitored to determine, in whole or in part, which sensors provide information to the system. This may include designation of data type, frequency and duration of such data, granularity of data, etc., since some devices may contain multiple sensors.

[0150]

[0149] Edge sensors may be dynamically configured and have such configurations deployed in response to patterns stored and managed by the system and / or to sequences of events occurring in the environment.

[0151]

[0150] In the context of sparse sampling, a random model within a particular distribution may be utilized where the entire environment is stationary. This may follow a pattern specification in the form of a pattern, pattern element, and / or pattern framework, such that sparse sampling is configured for different rates and data sets at particular times and / or for particular durations. These samplings may be responsive to events detected by at least one sensor, and the rate, type, and data exchange of sampling may change according to these events. For example, if a person sleeps at night with a sparse pattern while moving and the acoustic monitoring device detects that the person is experiencing sleep apnea, the pattern utilized for monitoring may change accordingly.

[0152]

[0151] Healthcare Pattern (HCP)

[0153]

[0152] A health care pattern is the overarching context of a PUM and generally incorporates the initial care status of the PUM for which the care system was invoked. A particular HCP is a subset of the overall health journey of a PUM, and once the status reaches a stage requiring monitoring, the health status of the PUM is likely to follow a series of statuses that ultimately lead to recovery from the status or to home care, hospice, hospitalization, palliative or end-of-life care status. The length of time that a person in a monitored status remains with any particular HCP depends on factors including the person's own health status, the assistance provided to the person, the available health care, etc. However, once a status such as Alzheimer's is determined to be of concern such that care monitoring is required, a decline in the health of the PUM is inevitable unless or until a way is found to reverse the status. An HCP is a quantized specification of the states of change and / or decline that such a PUM may undergo. In some embodiments, these states are represented by one or more behavioral patterns such as those shown in Figures 2 and 3.

[0154] In some embodiments, there is an active HCP, which may include a set of pattern elements and / or patterns that represent important indicators of the form of a data set that may be monitored for PUM in the environment. For example, for PUM with emphysema, acoustic monitoring of their breathing patterns may be mandatory.

[0155]

[0154] The HCP can be managed by the system and provide an overall framework for monitoring and care, involving each of the stakeholders including PUMs, friends and family, and medical professionals involved in the diagnosis, monitoring and care of the condition being monitored.

[0156] In some embodiments, the HCP may include one or more patterns that may be considered stable "stagnation" in the health care journey of the PUM, and the amount of time the PUM has been in such a state may vary from person to person. The HCP, and the patterns and pattern elements contained therein, may include events and event sequences that are behavioral indicators that are states of the monitored state. This may include monitoring changes in specified states and / or identifying new states. Such changes may be gradual or abrupt, and may be with varying degrees of warning.

[0157]

[0156] One or more HCPs may represent the course of PUM from an initial diagnosis of a condition requiring monitoring, through the stages of the health process, to their recovery or eventual terminal decline.

[0158]

[0157] Transition behavior The PUM exhibits one or more behaviors that are indicators of a change in a currently operating pattern that represents the behavior of the PUM. For example, if there is an increase in coughing, a change in breathing pattern, an increase in the use of sprays or breathing assistance, this may indicate that the PUM is becoming more and more distressed and is therefore transitioning from a given pattern, e.g., "stable breathing pattern," to another pattern, e.g., "disturbed breathing," which may form part of the HCP of the PUM being monitored, e.g., for emphysema. These behaviors may be represented as feature sets that include sensor and / or device data that may be designated as transition feature sets that are indicators of a change from a given pattern to another pattern.

[0159]

[0158] Detection of these behavioral changes may be direct, for example, through the use of other active or passive sensors that detect FMCW or PUM breathing patterns, which may be configured, for example, as edge sensors and may be verified or confirmed by other sensors, such as acoustic sensors, e.g., MEMS microphones, that detect changes in breathing patterns.

[0160] In some embodiments, a digital twin may be used as part of transition detection where current patterns of motion are developed and data from sensors are incorporated. These data sets may then be compared to data sets from expected patterns forming the HCP, e.g., if a "stable breathing pattern" is in motion and the expected pattern is "disordered breathing," one or more matching and / or comparison algorithms may be utilized to assess the likelihood that this transition is occurring. The digital twin, or a set thereof, may compare multiple possible patterns to assess the most likely transition. Such assessment may include using one or more machine learning techniques to identify likely trends and potential transitions.

[0161] 10 illustrates one or more digital twins (505) operating in collaboration with an HCP (701) including motion patterns (703) and potential motion patterns (1001), the digital twin including one or more motion pattern variations representing potential variations based on different simulated and / or projected sensor data, which can be matched, for example, using a prediction system (503) and / or a matching system (904) to ascertain the most likely appropriate motion pattern (1001) that most accurately represents the care and health status of the PUM, based at least in part on the behavioral transition pattern elements (705). The digital twin may then continue to project and / or predict such variations as the PUM's care journey unfolds.

[0162] In some embodiments, a transition from any behavior pattern to another may generate an alert or event and be communicated to an appropriate set of interested parties, such as a doctor, pharmacy, caregiver, relatives, etc. If the pattern is known as part of the HCP and the transition is part of a well understood health and wellness process, the behavior pattern may change and sensors, devices and / or systems may be configured for that pattern. If the transition represents an immediate and / or potential significant risk to the health and wellness of the PUM, the event and / or alert may be such that emergency and / or other interested parties are notified. For example, not detecting breathing of the PUM indicates a potentially life-threatening situation.

[0163]

[0162] In some embodiments, the relationships between the dimensions of the multidimensional feature set can provide an indication of a change in the behavior of the PUM through the evaluation of these relationships. For example, if dimension A, representing data from one or more sensors detecting the breathing of the PUM, and dimension B, representing data from one or more sensors detecting coughs, for example through an acoustic sensor such as a MEMS microphone, have N relationships, where N represents, for example, the number of coughs per breath over any period of time, the relationship increases, which can be an indication of a behavior change. In this example, another dimension may include the position of the body of the PUM relative to a vertical or horizontal axis. For example, the relationship of this dimension to the other dimension, whether the PUM is lying down more than sitting or standing.

[0164] In some embodiments, a dimension may include different combinations of sensor data. For example, a dimension representing an action such as coughing may include respiratory monitoring sensor data, wearable device sensor data, and / or acoustic sensor data. Each of these data sets may have a combined weighting or ranking that affects the overall value of the dimension.

[0165] In some embodiments, the detection and / or identification of transitional behavioral patterns may incorporate one or more machine learning techniques, including regression learning, neural networks, etc., whereby a data set representing one or more behaviors, including one or more feature sets that the sensor and / or device is configured to recognize, may represent a transition between any pattern or pattern element and another pattern and / or pattern element. This may include multiple such patterns and / or pattern elements having different rankings based at least in part on their relative probability and / or likelihood based on similar circumstances that may occur.

[0166]

[0165] Figure 7 shows a transition state between two behavior patterns (703 and 704, respectively), each comprising a set of pattern elements in an HCP (701), and a behavior pattern representing a transition (705) between the two behavior patterns in the HCP.

[0167] 8, behavioral pattern changes are identified (706) that represent precursors to transitional behavioral patterns (705) and provide advance notice, in whole or in part, of that future change in the PUM (105) care situation. This data can be identified through monitoring of a single PUM (105) and / or through monitoring of multiple PUMs with the same or similar HCPs. For example, this can be done through the use of digital twins and / or ML / AI techniques.

[0168]

[0167] Figure 9 illustrates the use of a prediction (503) and matching (904) system, which when implemented may include, for example, one or more digital twins (505) and machine learning modules (504), where a set of candidate patterns (905, 906, 907) are evaluated by the matching system (904) as most likely to match the transition behavior pattern (705). In this example, there are two pattern elements that represent precursors (901, 902) of behavioral changes represented by transition behavior pattern elements (705), for example, these pattern elements include behavioral attributes exhibited by the PUM, which are common to the behavioral pattern and its pattern elements, but may further be emphasized or have other changes that indicate a change, and in some embodiments, the monitoring focus may be changed to further identify and / or confirm such behavioral changes. In some embodiments, one or more of the candidates (907) may be part of a different HCP. In some embodiments, these HCPs may have some degree of correlation, for example, all may be associated with PUMs that have respiratory issues, and / or each of the HCPs may have a different care focus.

[0169]

[0168] As shown in FIG. 9, this can include the use of a digital twin (505) in combination with prediction (503) and matching (904) modules to evaluate pattern variations, such as those of precursors (901, 902), to identify and / or confirm transition behavior pattern elements (903) and transitions to one or more candidate behavior patterns.

[0170]

[0169] Pattern Framework

[0171]

[0170] A pattern framework is a specification based in part on a person's behavioral patterns in an environment, which may be represented by pattern elements. This framework is coupled with the person's HCP, such that such a framework may represent a set of potentially overlapping behavioral patterns that represent a person typically moving through the HCP. For example, if a PUM has, say, emphysema, a set of pattern frameworks would include typical behaviors and time frames for that condition, mitigation of the condition based on various medications, treatments or other supports provided, typical behavioral aspects of a PUM with such a condition, events, event sequences, triggers, and other data sets indicative of future or actual changes in those conditions, etc.

[0172]

[0171] The framework may include predictive indicators represented as event sequences and / or pattern elements that represent a person changing from one behavioral pattern to another, for example, a decline or increase in a health condition being monitored under care. In some embodiments, this may be represented by a multi-dimensional feature set that includes one or more dimensions.

[0173]

[0172] A pattern framework may include and / or be partially created by a set of pattern elements, which can be generally defined as sensor data including a multi-dimensional feature set that in a sequence forms a set of events. These sensor data sets can indicate various changes in the state of the sensors and the environment they are monitoring. However, the behavioral aspects of the PUM are an integral part of the pattern framework in that these specifications describe the activity of the PUM and provide a context for the sensor data sets, thereby providing effective monitoring of the care recipient.

[0174]

[0173] One advantage of this approach is that the sensor set is arranged and configured to focus on the PUM and the current activity of the PUM using behavioral specifications represented in a pattern by the care processing system, for example as pattern elements, including verifying specific activity and identifying behavioral sets that indicate a change in the care state of the PUM. This ability to identify possible precursors of a care event that requires or requires intervention is essential to the health of the PUM. This approach removes the reliance on the PUM to self-identify potentially critical care events and incorporates the necessary event and alert management systems to communicate awareness of the situation to other parties involved in the PUM's care.

[0175]

[0174] The pattern framework is first instantiated, at least in part, with the care situation being diagnosed and forming an initial specification of the pattern framework as part of the HCP. The set of potential situations that may be monitored is broad, but most of these are related to the age of the PUM and can therefore be grouped into age-specific HCPs. These groupings may also be based on the type of care monitoring, e.g., respiratory related, memory impairment related, degenerative disease related, etc. In such a framework, the pattern elements that may comprise such a framework may be sensor data centric and / or PUM behavior centric. These aspects may be configured to create a pattern framework appropriate for the care situation being monitored.

[0176]

[0175] Over time, the initially instantiated pattern framework will be further populated with data sets from sensors that match the sensor and / or behavioral data sets, allowing the initial pattern framework to evolve into active, individual patterns that are specific to and oriented around the PUM and the participants and environments that have a particular relationship to the PUM.

[0177]

[0176] Figure 4 illustrates a pattern framework (401) that may represent one or more HCPs (101), with each pattern framework populated with one or more pattern elements (102) to form one or more motion patterns (103). An HCP (101) may include multiple pattern frameworks (401), and / or each pattern framework may include multiple pattern elements (102) in any arrangement to form one or more motion patterns (103).

[0178] A further aspect is the instantiation of a digital twin incorporating the initial pattern framework. This digital twin and its multiple instantiations may then be populated with data sets from sensors at any level of granularity and used in combination with machine learning techniques to predict behaviors and initiate transitions to care processing systems, novel patterns of sensors, placements and / or configurations, and different behavioral patterns of any placement and / or HCPs.

[0179]

[0178] FIG. 16 shows one or more digital twins (1606) comprising a pattern framework (1602), sensor data (1603), pattern elements (1604), and behavioral patterns (1605), which can represent potential states of the PUMs (105) and the environments (104) in which they are monitored.

[0180]

[0179] This approach provides contextualization of sensor data sets that represent behavioral characteristics and their metrics essential for effective, efficient, and responsive care management.

[0181]

[0180] Pattern input

[0182] In some embodiments, an HCP may have several patterns that may be developed that represent the likely behavior of a PUM in an environment. This may include patterns that are predicted and / or are the same or similar to the patterns of other HCPs with common monitoring specifications. For example, if a PUM has situation A, and the HCP in that situation includes patterns A, B, C, D, etc., and the PUM being monitored has a high correlation with those patterns, then such an HCP may be used for another PUM with the same situation. In this example, patterns A, B, C, D, etc. are represented as a pattern framework that includes pattern elements that represent PUM behavior without the sensor data set. In this way, these pattern elements may be populated by the PUM sensor data set as they traverse the pattern elements and patterns of that HCP. A set of patterns that represent the behavior of a PUM in an environment may be applied across multiple HCPs. For example, if a PUM has multiple care situations, there may be significant overlap of patterns, but one may be dominant and therefore it becomes the focus of monitoring.

[0183]

[0182] Some behavioral patterns may be classified with respect to behavioral routines performed by the PUM, such as sleep, exercise, visits to or by the participant, travel, drug intake, treatment, procedure, etc. Such classification schemes may be arranged, for example, as an ontology, taxonomy, hierarchical or any other arrangement. In some embodiments, pattern executions, as well as PUMs that have executed such patterns and / or sets of patterns, may be recorded in a suitable repository and / or a suitable distributed ledger. This may typically be the case when medication or specific behaviors that are highly relevant to the health of the PUM are involved. Such recording may include a tokenization of these patterns.

[0184]

[0183] In some embodiments, this may include monitoring compliance with treatment plans, including regular intake of prescribed medicines or other medicinal compounds, and / or regular performance of prescribed activities, such as treatment-related physical exercise, sleep patterns, and eating patterns. This may include other compliances, such as insurance provider compliance, whereby insurance coverage is determined in part by behavioral compliance of the PUM and / or the actors and environment of the PUM. Further compliances may be determined by contracts and / or other specifications that are part of the overall care oversight arrangement, some of which may be legally binding and / or translated into commercial and / or business obligations. In some embodiments, this may include court-ordered behavioral and activity regimes. In some embodiments, such obligations and compliances may form, in whole or in part, smart contracts that are recorded in one or more distributed ledgers.

[0185]

[0184] The patterns and / or pattern elements utilized include a set of behavioral specifications that represent a set of events, and sensor data that represents those behaviors. These specifications can encompass multiple sensors, environments, and / or participants. The specifications may be dynamically changed in response to changes in the environment and the PUM's conditions. This may include, for example, using a monitoring focus module to increase or decrease the fidelity of sensors through variable configuration. In some circumstances, this may include replacing any pattern with another pattern.

[0186]

[0185] There may be a pattern operating in an environment that may include, for example, the configuration of a set of sensors that may be a subset of all available sensors in the environment. This pattern may have further patterns pre-deployed, such as in an elastic repository that may be local and / or remote to the sensors and / or environment, such that when the care process detects behaviors matching certain criteria, such as excessive breathing, elevated heart rate, acceleration in one or more axes, the current motion pattern may be replaced, in whole or in part, by the cached pattern in a continuous manner. This may include the activation, configuration and / or reception of data from sensors operating under the previous pattern, and / or may include the activation, configuration and reception of data from additional sensors, such as acoustic sensors, video sensors, radar sensors, mobile sensors, worn sensors and / or ingestive sensors. Similarly, the number and type of sensors may be increased or decreased as determined by the motion pattern. In some embodiments, such changes in the pattern and / or pattern elements may be initiated by a transitional behavioral pattern represented by a change in one or more dimensions of one or more multidimensional feature sets.

[0187] In some embodiments, a pattern or pattern element may include specifications for assigning priorities to one or more sensors, changing the state and / or configuration of the sensors, e.g., to conserve battery capacity, etc. For example, a GPS may be put to sleep, e.g., when the home location is known, and woken up when an exit trigger is detected.

[0188] In some embodiments, the care processing system may invoke different patterns using the same and / or segmented sets of sensors for monitoring. These patterns may be driven by one or more digital twins, and data from the sensors may include historical, estimated, predicted, and / or actual real-time or near real-time data sets for any configuration.

[0189]

[0188] The patterns and / or pattern elements may be categorized and typed according to another ontology, taxonomy, or other organizational principle. The care processing system may create new patterns and / or pattern elements based on existing patterns, for example using machine learning techniques.

[0190]

[0189] In some cases, there may be what appears to be a data set that is applicable to multiple patterns, such as motion detection, but given the nature of care processing systems, such data sets representing an event may be considered in the context of data sets representing other events potentially being monitored, as well as events that occur and / or are likely to occur in conjunction with that initial event.

[0191]

[0190] Exemplary embodiments

[0192]

[0191] Morpho-based personal emergency response system (PERS) devices are intended for elderly or disabled people to request rescue or emergency services by pressing an emergency button in the PERS device. These devices typically include an emergency button, a speaker, a microphone, and wireless communication capabilities, including limited wireless telephone capabilities used to connect the person with emergency personnel or caregivers using voice. In some cases, PERS devices also include sensors and software that use the sensor's signals to detect events, such as a fall, and automatically trigger an emergency call when such an event occurs and / or report the event to a central server. They may also include location detection sensors, such as GPS, radio frequency triangulation, beacon readers, etc., which allow the PERS device, or the system to which it connects, to trigger an emergency or other event, for example, when a person leaves a predefined area (geofence), stops moving for a long enough period of time, or under other location and / or movement-related circumstances.

[0193]

[0192] One problem with a PERS device configured with these and other sensors is that keeping all sensors active most of the time and processing their signals to effectively detect relevant events can quickly drain the device's battery, thereby reducing its usefulness in real-world situations. By applying the concepts described herein, this problem can be solved and the performance and accuracy of the PERS device's functions can be improved. For example, in normal situations (stationary state), most sensors in the device, except for the accelerometer, can be configured to remain dormant, and the software in the PERS device can listen only to signals from the accelerometer that indicate movement above a certain threshold, indicating that the person has changed state from passive to active. At this point, other sensors such as the altimeter and microphone in the device can be activated, the configuration of the accelerometer and the threshold of its signal can be changed, and the software can switch to a different set of detection logic to create a different configuration suitable for detecting the most likely event under the person's new state. Additionally, the location detection sensor and geofence logic can be activated. The geofence can also trigger a new change in configuration, for example, when a "going outside" situation is detected, and the accelerometer and altimeter operating parameters and event detection logic can be changed to detect falls under walking outside motion conditions, or stopped when a "moving into the car" situation is detected, for example, based on a combination of position change and accelerometer signals. In this approach, the sensor, processing, and communication functions are used only when the detected patterns indicate that they are needed, resulting in reduced power consumption. Furthermore, by dynamically changing the sensor configuration and detection logic, event detection accuracy can be improved.

[0194]

[0193] An implication of using a PERS device as the sole method of detecting risk-related events such as falls is the limited accuracy that results from a set of co-located sensors in a small, portable device. This makes it difficult to avoid false positive and false negative event situations. This can be improved by combining the sensor configuration, data processing, and detection logic of the PERS device with that of a device located in the same environment but external to the PERS device.

[0195]

[0194] For example, the home of a PERS user may be equipped with additional sensors such as cameras, smoke detectors, microphones, etc. Signals from these sensors may be combined with the signals of the sensors of the PERS device and other devices such as voice recognition enabled speakers (smart speakers) as a way to make the PERS system effective. This combination of PERS device data and other sensor data may provide an accurate, complete, and actionable data set regarding the state of the PUM. For example, there may be dedicated and / or general purpose devices that may interact with the PERS device directly and / or through dedicated devices such as care hubs. This may include one or more communications with such other devices in the user's home connected to the PERS device using wireless communication mechanisms such as Wi-Fi or Bluetooth, and may include one or more local and / or remote servers. This combination may be used to accurately determine the state of the user based on known user behavior patterns such as typical locations and activities in the home and the signal patterns that those activities generate at the sensor placement of the home sensors and the sensors of the PERS device.

[0196] For example, as shown in FIG. 11, PERS devices (1108) worn by PUMs (105) in an environment (104) generate data that is combined with additional data generated by general smart devices (1105) co-located in the environment, dedicated sensors (1103), external data sources (1109) such as weather, traffic, emergency, and / or other sensors, including those designated as edge devices (1102). These data sets may be processed by a care data processing module (1106) and / or a care processing module (1107) that may form multi-dimensional feature sets in conjunction with a pattern identification system (1104) that may represent, in whole or in part, pattern elements and / or behavior patterns of the HCPs (701). The care hub (1101) may operate to facilitate such aggregation, integration, processing, and communication.

[0197]

[0196] Some of the devices and / or servers in this type of configuration may include machine learning or statistical mechanisms as an adaptive method to identify patterns indicative of changes in user or environmental conditions, accurately select sensor configurations, and / or trigger events or alarms. The signals transmitted by the PERS devices and sensors in the user's home can be stored and used in the server to train machine learning systems and / or to provide a digital twin of the user and environment, allowing adaptation to changes in the user's behavior and environment, accurately predicting possible outcomes, and preparing emergency and support resources for them.

[0198]

[0197] In some embodiments, the device and its associated sensors may ascertain one or more sets of care-related biometric information that alone provide some information for monitoring, but may also be combined with HCP and other pattern management associated with the system to inform as to the overall status of the care recipient.

[0199] In some embodiments, an edge device may be configured as a hub to aggregate one or more data sets from sensors and / or coordinate one or more configurations of such sensors. This may include providing processing to such sensors depending on the capabilities of the edge device. In some embodiments, a care hub may be designed as an edge device.

[0200]

[0199] The use of distributed processing capabilities across multiple sensors, devices, modules, and / or systems can include any arrangement of monitoring environments and / or systems deployed in the cloud or other remote locations. In some embodiments, the monitoring system may be configured with multiple levels of redundancy to account for loss of communications, power, or other critical functionality. In some embodiments, the configuration may utilize standard redundancy and resiliency techniques to ensure that a minimum of monitoring functionality operates for a sufficient period of time to make additional external assistance, such as human intervention, available to the PUM.

[0201]

[0200] This may include the use of back-up power systems, multiple redundant communication systems, and local physical support such as neighbors or friends.

[0202] In some embodiments, ingestible and / or implantable sensors may be incorporated to provide the sensor data set. These sensors may form part of a set of sensors worn by the PUM, e.g., a PERS, smart clothing, smart watch, etc., which may receive data from the implanted and / or ingested sensors. In some embodiments, the PERS may provide a secondary power source for these ingested and / or implanted devices. The PERS or other devices may poll the implanted and / or ingested sensors to conserve power for these devices, e.g., using various techniques including RF, near field, inductive charging, etc.

[0203]

[0202] One aspect is the relationship between a device such as a PERS and ingested and / or implanted sensors, where data sets from individual sensors may be directed to one or more other devices such as medical monitors, and the PERS or other worn or carried device may provide a communication path to the other devices, and thus the sensor may use low power, low range communication technology, and the PERS or other worn or carried device may provide high power and / or long range communication capabilities. In some embodiments, the nature of the sensor data may be such that the PERS or other worn or carried device cannot access the sensor data, and may encrypt such data for onward transmission to other devices that are specifically identified, authorized, and / or authenticated. In some embodiments, such sensor data may be communicated to other devices in a manner that protects personally identifiable information (PII) or HIPAA data, such as protected health information (PHI) of PUM. In some embodiments, the PERS may communicate the sensor data set to a care hub, which may operate to further anonymize the data set from the sensor, for example using TOR or other Internet routing technology, to reduce potential identification through the location and / or routing of packets representing that sensor data and / or the PERS of the PUM.

[0204]

[0203] Figure 12 shows a PUM (105) in an environment (104) including a set of sensors S1-S7 and an ingestible / implantable sensor (IS1), in this example a care hub (1001) providing communication capabilities for the sensors and care processing capabilities (1201) that may be local and / or remote to the care hub. The care hub and care processing integrate and communicate with a digital twin (505), machine learning (504) and / or matching system (1203) including a multi-dimensional feature set (1202) that aids in health and care monitoring of the PUM. The state of the PUM may be represented by one or more patterns, for example, a motion pattern (103) including a resting state (1204) of the PUM and a pattern representing the environment within the HCP (101). Such changes in state may be identified by the care hub and / or care processing, which may then invoke a monitoring focus module (601) to change the configuration of one or more sensors and / or processing systems to accurately determine the state of the PUM.

[0205]

[0204] The foregoing description of the embodiments is provided to enable any person skilled in the art to practice the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and features disclosed herein.

Claims

1. A system for monitoring a care recipient by a related person, comprising: a plurality of environmental sensors configured to monitor the care recipient and provide a detected dataset representative of a behavior of the care recipient in an environment, a plurality of environmental sensors, each of said behaviors being represented by a multi-dimensional feature set forming part of a healthcare profile of said care recipient; A care processing system, comprising: a transceiver configured to receive the detected data set; a non-transitory computer-readable storage medium configured to store a stationary data set, the stationary data set representing a previous stationary behavior of the care recipient in the environment; at least one hardware processing unit for determining a health or care event of the care recipient by comparing the detected data set with the stationary data set; A care processing system comprising: Equipped with When the health or care event occurs, the care processing system is configured to change a state of the plurality of environmental sensors or notify the relevant person. system.

2. The system of claim 1 , wherein the multi-dimensional feature set comprises a manifold, a Hilbert space, or other representation capable of storing the detected data set from the multiple environmental sensors.

3. The system of claim 2 , wherein the detected data set is from a temperature sensor, an acoustic sensor, or a motion detector.

4. The system of claim 2 , wherein the detected data set represents aggregated data from the plurality of environmental sensors.

5. The system of claim 4 , wherein the aggregate data is represented as a ratio, a function, an algorithm, or a spatial representation.

6. The system of claim 2 , wherein the detected data set is from a respiration sensor or a heart rate sensor.

7. The system of claim 6 , wherein the stationary data set represents sleep, eating, bathroom use, or exercise.

8. The system of claim 2 , wherein the change in the state of the plurality of environmental sensors changes a monitoring focus of the environmental sensors.

9. The system of claim 8 , wherein the monitoring focus increases the fidelity or granularity of the environmental sensors.

10. 1. A system for developing a pattern representative of a health status of a care recipient by a related party, comprising: a plurality of environmental sensors configured to monitor the care recipient and provide a detected dataset representative of a behavior of the care recipient in an environment, a plurality of environmental sensors, each of said behaviors being represented by a multi-dimensional feature set forming part of a healthcare profile of said care recipient; A care processing system, comprising: a transceiver configured to receive the detected data set; at least one hardware processing unit for determining a variance in the detected data set indicative of a transition state between a first pattern and a second pattern in the health state representative of the health and care state of the care recipient; A care processing system comprising: Equipped with the care processing system is configured to change a sensor configuration of the plurality of environmental sensors to adjust for the transition condition. system.

11. The system of claim 10 , wherein the multi-dimensional feature set comprises a manifold, a Hilbert space, or other representation capable of storing the detected data set from the multiple environmental sensors.

12. The system of claim 11 , wherein the detected data set is from a temperature sensor, an acoustic sensor, or a motion detector.

13. The system of claim 11 , wherein the detected data set represents aggregated data from the plurality of environmental sensors.

14. The system of claim 13 , wherein the aggregate data is expressed as a ratio, a function, an algorithm, or a spatial representation.

15. The system of claim 11 , wherein the detected data set is from a respiration sensor or a heart rate sensor.

16. The system of claim 15 , wherein the stationary data set represents sleep, eating, bathroom use, or exercise.

17. The system of claim 11 , wherein the change in the state of the plurality of environmental sensors changes a monitoring focus of the environmental sensors.

18. The system of claim 17 , wherein the monitoring focus increases the fidelity or granularity of the environmental sensors.

19. A system for monitoring a care recipient by a related person, comprising: a plurality of environmental sensors configured to monitor the care recipient and provide a detected dataset representative of a care state of the care recipient in an environment; A care processing system, comprising: a transceiver configured to receive the detected data set; At least one hardware processing unit for identifying and determining a care signal representative of the care state of the care recipient, the care signal having a multi-dimensional feature set; A care processing system comprising: Equipped with the care processing system is configured to respond to the care signal by engaging the participant; system.

20. 20. The system of claim 19, wherein the multi-dimensional feature set comprises a manifold, a Hilbert space, or other representation capable of storing the detected data sets from the multiple environmental sensors.

Images