System and method for personalized closed-loop opioid addiction management

Abstract

A method comprises acquiring a plurality of multimodal physiological signals, dividing the plurality of multimodal physiological signals into a plurality of sliding windows containing a respective multimodal physiological signal, selecting a subset of the plurality of sliding windows that contain the respective multimodal physiological signal, modifying the respective multimodal physiological signal contained in the subset of the plurality of sliding windows; providing, as one or more inputs to a machine learning model, the plurality of sliding windows that contain the respective multimodal physiological signal and the subset of the plurality of sliding windows that contain the respective modified multimodal physiological signal; and identifying, using one or more outputs of the machine learning model, at least one moment of opioid administration.

Description

SYSTEM AND METHOD FOR PERSONALIZED CLOSED-LOOP OPIOID ADDICTION MANAGEMENTSTATEMENT OF GOVERNMENT SPONSORED SUPPORT

[0001] This invention was made with government support under 2320678 awarded by the National Science Foundation and K23DA045242 awarded by the National Institutes of Health. The government has certain rights in the invention.CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority under 35 U.S.C. § 119(e) to U.S. Patent Application No. 63 / 554,843 titled “System for Detecting Opioid Administrations Using Wearables and Pharmacokinetics” and filed February 16, 2024, the contents of which are hereby incorporated by reference in their entirety.BACKGROUND

[0003] Opioid use and overdoses continue to fuel the opioid epidemic in the United States. In 2019 for example, the National Survey on Drug Use and Health (NSDUH) showed that approximately 10.1 million United States residents aged 12 years or older misused opioids, with the majority (9.3 million) citing prescription opioid misuse. This misuse or abuse continues to increase, highlighting a need for action. While prescribed opioids can provide temporary pain relief, frequent and repetitive opioid over use can be considered an early indicator of opioid misuse and developing opioid use disorder.SUMMARY

[0002] In some example embodiments, there may be provided a method comprising acquiring a plurality of multimodal physiological signals; dividing the plurality of multimodal physiological signals into a plurality of sliding windows containing a respective multimodal physiological signal; selecting a subset of the plurality of sliding windows that contain the respective multimodal physiological signal, modifying the respective multimodal physiological signal contained in the subset of the plurality of sliding windows; providing, as one or more inputs to a machine learning model, the plurality of sliding windows that contain the respective multimodal physiological signaland the subset of the plurality of sliding windows that contain the respective modified multimodal physiological signal; and identifying, using one or more outputs of the machine learning model, at least one moment of opioid administration.

[0003] In some implementations, the method further comprises adjusting the machine learning model to detect drug plasma concentration in the plurality of multimodal physiological signals by teaching the machine learning model to learn pharmacokinetics of drug plasma concentrations. In further implementations, the plurality of multimodal physiological signals are acquired by a wearable device. In certain implementations, the first sliding window of the plurality of sliding windows and a second sliding window of the plurality of sliding windows comprise overlapping multimodal physiological signals. In some implementations, modifying the multimodal physiological signal comprises changing the signal to mimic an opioid administration. In further implementations, between about forty five percent of the plurality of sliding windows and about fifty five percent of the plurality of sliding windows are selected for modification. In some implementations, the machine learning model is trained to identify the moment of opioid administration on the plurality of sliding windows and the subset of the plurality of sliding windows. In certain implementations, changing the signal to mimic the opioid administration comprises multiplying the signal by an amplification factor and by a deterministic function. In some implementations, identifying the moment of opioid administration comprises identifying a signal of the plurality of multimodal physiological signals that mimics an opioid administration.

[0004] In some example embodiments, there may be provided a method comprising acquiring a plurality of multimodal physiological signals; encoding the plurality of multimodal physiological signals to generate a plurality of encoded multimodal physiological signals; generating a plurality of embeddings representing the plurality of encoded multimodal physiological signals; combining the plurality of embeddings with covariate information; providing the plurality of embeddings and the covariate information to a machine learning model; identifying, using the machine learning model, similar groups of physiological signals; and determining an amount of user distress represented by the plurality of multimodal physiological signals.

[0005] In some implementations, the amount of distress is determined as a score, the score comprising at least one of a stress score, a pain score, and / or a craving score. In certainimplementations, the method further comprises predicting at least one of a stress trajectory, a pain trajectory, and a craving trajectory. In further implementations, the method further comprises performing a non-linear topological analysis on at least one of the stress trajectory, the pain trajectory, and the craving trajectory, and, when the non-linear topological analysis fails to detect a presence of chaos in the at least one of the stress trajectory, the pain trajectory, and the craving trajectory, determining a probability of opioid misuse. In some implementations, identifying similar groups of physiological signals comprises, by the machine learning model, dynamically clustering users having similar physiological signals and demographic information into groups.

[0006] In certain implementations, the method further comprises acquiring a plurality of cognitive signals. In further implementations, the method further comprises identifying an intervention trigger based on at least one of the stress trajectory, the pain trajectory, the craving trajectory, and the probability of opioid misuse. In some implementations, the method further comprises selecting an intervention based on the intervention trigger. In certain implementations, the covariate information comprises at least one of prescription history, drug history, and demographic information.

[0007] In some example embodiments, there may be provided a system comprising at least one programmable processor; and a non-transitory machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform operations comprising: acquiring a plurality of multimodal physiological signals; dividing the plurality of multimodal physiological signals into a plurality of sliding windows containing a respective multimodal physiological signal; selecting a subset of the plurality of sliding windows that contain the respective multimodal physiological signal, modifying the respective multimodal physiological signal contained in the subset of the plurality of sliding windows; providing, as one or more inputs to a machine learning model, the plurality of sliding windows that contain the respective multimodal physiological signal and the subset of the plurality of sliding windows that contain the respective modified multimodal physiological signal; and identifying, using one or more outputs of the machine learning model, at least one moment of opioid administration.

[0008] In some example embodiments, there may be provided a system comprising at least one programmable processor; and a non-transitory machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform operations comprising: acquiring a plurality of multimodal physiological signals; encoding the plurality of multimodal physiological signals to generate a plurality of encoded multimodal physiological signals; generating a plurality of embeddings representing the plurality of encoded multimodal physiological signals; combining the plurality of embeddings with covariate information; providing the plurality of embeddings and the covariate information to a machine learning model; identifying, using the machine learning model, similar groups of physiological signals; and determining an amount of user distress represented by the plurality of multimodal physiological signals.

[0009] In some example embodiments, there may be provided a computer program product comprising a non-transitory machine readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising: acquiring a plurality of multimodal physiological signals; dividing the plurality of multimodal physiological signals into a plurality of sliding windows containing a respective multimodal physiological signal; selecting a subset of the plurality of sliding windows that contain the respective multimodal physiological signal, modifying the respective multimodal physiological signal contained in the subset of the plurality of sliding windows; providing, as one or more inputs to a machine learning model, the plurality of sliding windows that contain the respective multimodal physiological signal and the subset of the plurality of sliding windows that contain the respective modified multimodal physiological signal; and identifying, using one or more outputs of the machine learning model, at least one moment of opioid administration.

[0010] In some example embodiments, there may be provided a computer program product comprising a non-transitory machine readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising: acquiring a plurality of multimodal physiological signals; encoding the plurality of multimodal physiological signals to generate a plurality of encoded multimodal physiological signals; generating a plurality of embeddings representing the plurality of encoded multimodal physiological signals; combining the plurality of embeddings with covariate information; providing the plurality of embeddings andthe covariate information to a machine learning model; identifying, using the machine learning model, similar groups of physiological signals; and determining an amount of user distress represented by the plurality of multimodal physiological signals.

[0011] The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be apparent from the description and drawings, and from the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, show certain aspects of the subject matter disclosed herein and, together with the description, help explain some of the principles associated with the disclosed implementations. In the drawings,

[0013] FIG. 1 A illustrates a process for the training of a machine learning model in accordance with some embodiments described herein;

[0014] FIG. IB illustrates a the training of a machine learning model in accordance with some embodiments described herein;

[0015] FIG. 2A illustrates a process for predicting at least one of a stress score, a pain score, and a craving score based on multimodal physiological signals in accordance with some embodiments illustrated herein;

[0016] FIG. 2B illustrates a process for predicting an opioid misuse risk score based on a stress trajectory, a pain trajectory, and a craving trajectory, in accordance with some embodiments described herein;

[0017] FIG. 3 illustrates a system for using multimodal physiological data captured during the performance of a cognitive task to predict an opioid misuse risk score in accordance with some embodiments described herein;

[0018] FIG. 4 illustrates a system 400 for identifying an intervention trigger based on at least one of a detected opioid administration, a predicted stress score, a predicted pain score, a predicted craving score, and a predicted opioid misuse risk score in accordance with some embodiments described herein;

[0019] FIG. 5 illustrates a system 500 for forecasting a change in a multimodal physiological signal after an intervention action has been taken; and

[0020] FIG. 6 depicts an example of a system in accordance with some embodiments illustrated herein.DETAILED DESCRIPTION

[0006] Long-term and high-dose prescription opioid use places individuals at risk for opioid misuse, opioid use disorder (OUD), and overdose. Existing methods for monitoring opioid use and detecting misuse rely on self-reports (which are prone to reporting and recall bias, intentional concealment, and short detection periods) and toxicology testing (which may be infeasible in outpatient settings). Although wearable technologies for monitoring day-to-day health metrics have gained significant traction in recent years due to their ease of use, flexibility, and advancements in sensor technology, the application of wearable technologies within the opioid use space remains underexplored.

[0007] Diagnostic methods that offer objective and real-time measures of opioid use may collect data automatically and passively. Opioid use tracking systems can also help to detect treatment adherence and opioid relapse. However, models built using a purely data-driven approach (without learning any underlying pharmacological mechanisms of opioids) may limit the generalizability of the model to different types of opioids. A candidate group of properties intrinsic to opioids that may be very useful to inform models are the drugs’ pharmacokinetic (PK) parameters.

[0008] In accordance with some embodiments described herein, there is provided a pharmacokinetics-informed machine learning framework that jointly learns the pharmacokineticsof opioids while estimating opioid administration moments using multimodal physiological signals collected from a wearable wrist sensor in inpatient and outpatient settings.

[0009] In some embodiments, there is provided a way to detect oral opioid administrations using physiological signals collected from a wrist sensor. Moreover, there are disclosed herein one or more models (e.g., machine learning model, such as neural networks and the like that are informed by opioid pharmacokinetics) that increase reliability in predicting the timing of opioid administrations. By teaching a machine learning model about the timing and intensity of expected effects upon the administration of an opioid, this data can be harmonized with real-time physiologic data and can lead to improved precision of our opioid administration predictions and smarter diagnostic tools. Pharmacokinetics (PK) refers to how the body affects a specific substance after administration.

[0010] While training a machine learning (ML) model such as a neural network, using supervised learning to learn the pharmacodynamics (e.g., how opioid affects different physiological signals), the relative plasma drug concentration equation (which is derived from opioid pharmacokinetics) may be used as a loss function for the ML model, such as the neural network. The pharmacokinetics-informed machine learning model may thus be used to predict opioid use moments with high fidelity, which may outperform traditional purely data-driven supervised models that do not have any pharmacokinetics knowledge.

[0011] FIG. 1A illustrates an example of a process 100 for the training of a machine learning model, such as a neural network or other type of ML model, in accordance with some embodiments described herein. In some implementations, 102-112 of the process 100 may comprise pre-training and / or auxiliary training for the machine learning model. For example, the pre-training may comprise 102-110 and the auxiliary training may comprise 112. The process 100 may be implemented by at least one processor, such as the processor 6ofFIG. 6 or other types of processors as well.

[0012] At 102, a plurality of multimodal physiological data (which may include signals) are acquired. For example, the plurality of multimodal physiological data may be acquired from a wearable device, which is worn by a user being observed or monitored. The wearable device used to acquire the plurality of multimodal physiological data may be, for example, the input / outputdevice 640 of FIG. 6, as discussed below. Other examples of wearable devices include an Empatica, Fitbit, and Apple Watch, etc..

[0013] The plurality of multimodal physiological data may include labeled and / or unlabeled data. Unlabeled data refers to data that does not include a label (or indication) regarding whether a user of the wearable device was administered an opioid. The plurality of multimodal physiological data may comprise at least a first signal, such as a heart rate variability (HRV) signal, an electrocardiogram (ECG), or a skin temperature signal. Alternatively, or additionally, the first signal (e.g., HRV signal, ECG, and / or the like) may be converted into data (e.g., using an analog to digital converter or some other form of pre-processing).

[0014] At 104, the plurality of multimodal physiological data may be divided into a plurality of sliding windows of data to segment the multimodal physiological data. The plurality of sliding windows may contain respective segments of the multimodal physiological data. For example, if ten hours’ worth of multimodal physiological data is acquired at 102, the plurality of sliding windows may comprise hour long segments of the multimodal physiological data (although other sized windows may be used as well). In some implementations, the plurality of sliding windows are a plurality of sliding windows that capture overlapping segments of the multimodal data. For example, if ten hours’ worth of multimodal data is acquired in the pre-training step, a first sliding window of the plurality of sliding windows may comprise a first segment of the multimodal data that was collected from the start of the data acquisition through the first hour of the data acquisition. A second sliding window of the plurality of sliding windows may comprise a second segment of the multimodal data that was collected between the thirtieth minute of data acquisition and the ninetieth minute of data acquisition. In this example, the first sliding window and the second sliding window comprise thirty minutes (e.g., fifty percent) of overlapping data (the data from the thirtieth minute of the data acquisition to the sixtieth minute of data acquisition). Any number of sliding windows may be used to segment the acquired multimodal data. For example, if ten hours’ worth of multimodal data is acquired, then between about twenty sliding windows and about fifty sliding windows may be used to segment the acquired multimodal data. Any two consecutive sliding windows of the plurality of sliding windows may contain between about forty percent of overlapping data and about sixty percent of overlapping data. In some implementations, two consecutive sliding windows of the plurality of sliding windows may contain fifty percent ofoverlapping data. In some implementations, the increment by which the sliding windows are separated is between about ten percent and about twenty percent of the length of the sliding windows.

[0015] In some implementations, the plurality of multimodal physiological signals may be pre- processed prior to being processed by the ML model. For example, the plurality of multimodal physiological signals may be down sampled to a desired sampling frequency. In some implementations, the sampling frequency may be 1 Hertz. Further, the plurality of multimodal physiological signals may be filtered or cleaned before use by the ML model. For example, the cleaning may include removing portions of data where the input / output device used to capture the plurality of multimodal physiological signals was not worn by a user. In some implementations, incompatible skin temperature (e.g., less than or equal to 20 degrees Celsius), lack of electrodermal activity (e.g., an electrical conductivity value of 0), and erroneously high heart rate (e.g., about 200 beats per minute or more) may be used as indicators to identify time periods (or portions) of the multimodal physiological signals or data during which the user was likely not wearing the wearable device. Cleaning the multimodal physiological signals may further include removing motion and noise artifacts and / or other types of fdtering (e.g., fdtering the multimodal physiological signals or corresponding data through a fdter, such as a fifth order Butterworth low- pass filter with a 1 Hz cut-off frequency.

[0016] At 106, a subset of the plurality of sliding windows are selected for modification of the multimodal physiological signals they contain. The multimodal physiological signals contained in the windows of the selected subset (after the windowing at 104) are modified so that a portion of the multimodal physiological signals therein mimic a physiological signal expected after or corresponding to the administration of an opioid. In some implementations, between about forty five percent and between about fifty five percent of the plurality of sliding windows are selected for modification of the multimodal physiological signals they contain.

[0017] The modification of the signal may be applied to the data in the portion of a window in accordance with an artificial task. For example, following opioid administration, a user’s heart rate may decrease, and a user’s skin temperature may increase. These heart rate and skin temperature changes (in the multimodal physiological data) may be modeled by one or moredeterministic functions. The artificial task may apply, for example, a deterministic function representing each change in multimodal physiological data to the data in the portion multimodal physiological signals in the window.

[0018] The deterministic function representing a change in the multimodal physiological data may be a pharmacokinetic (PK) equation. For example, Dtoc Dox (e-ket— e_kat) where D is the drug amount, k_a and k_e are the rates of absorption and elimination respectively, and t is the time step. Pharmacokinetics (PK) provides simple compartment models and canonical equations to describe the relative plasma drug concentration (amount of drug in blood) over time. In certain implementations, modifying a multimodal physiological signal according to an artificial task may include multiplying a portion of the multimodal physiological signal (e.g., a portion of a multimodal physiological signal within a sliding window) by an amplification factor, such as a scalar value. The portion of the multimodal physiological signal of a sliding window (which is modified by the artificial task) may be randomly selected. In other words, the temporal coordinate of the portion of the multimodal physiological signal that is modified may be randomly selected. Further, the length (e.g., temporal range, or the size of the portion) of the multimodal physiological signal that is modified may be randomly selected.

[0019] In implementations in which the multimodal physiological signal comprises more than a single signal, at least two signals of the multimodal physiological signal may be modified according to the artificial task.

[0020] In some implementations, fifty percent of the plurality of sliding windows are selected for modification of the multimodal physiological signals contained herein. In certain implementations, between about thirty percent of the plurality of sliding windows and about seventy percent of the plurality of sliding windows are selected for modification of the multimodal physiological signals contained therein. The plurality of sliding windows in which the multimodal physiological data is modified according to the artificial task may be randomly selected from among the plurality of sliding windows.

[0021] At 108, the plurality of sliding windows of multimodal physiological data and the subset of the plurality of sliding windows (that have had multimodal physiological data therein modified according to an artificial task) may be provided as inputs to a machine learning model.The machine learning model may be implemented by at least one processor (see, e g., system 600). The plurality of sliding windows of data may be used as training data for the machine learning model. For example, if the plurality of sliding windows of multimodal physiological data comprises ten windows, the subset of the plurality of sliding windows that have had multimodal physiological data therein modified may comprise five windows. Accordingly, five windows of the plurality of sliding windows of multimodal physiological data may be left unmodified. The subset of the plurality of sliding windows and the unmodified windows may be provided to the machine learning model as training data.

[0022] At 110, the machine learning model is configured to identify a moment of opioid administration. The machine learning model may be configured to learn which of the windows provided as an input contains a signal that has been modified according to an artificial task. In this manner, the machine learning model may be configured to detect multimodal physiological signals that indicate an opioid administration. Further, the machine learning model may be configured to determine the time (e.g., the temporal coordinate of the multimodal physiological signal) at which the multimodal physiological was modified. In other words, the machine learning model may be configured to determine which portion of a given sliding window was altered in accordance with the artificial task. The machine learning model may thus be configured to determine a moment of opioid administration.

[0023] At 112, the pre-trained machine learning model is adjusted by being taught a pharmacokinetic equation. The machine learning model may be adjusted or fine-tuned with labeled data to detect opioid administration and predict the moment of administration in the time window while also learning the plasma drug concentration over time as governed by the pharmacokinetic equation. Labeled data refers to data corresponding to known opioid administrations. This stands in contrast to the unlabeled data acquired at 102 of the process 100, for which it is unknown if an opioid administration has occurred. The labeled data may further comprise information regarding the concentration of drugs as a function of time (e.g., opioids) within a user that has administered opioids. In certain implementations, a finite number of labeled data are used to fine tune the machine learning model. For example, between about forty and about eighty labeled data (e.g., windows contained labeled data) may be used to fine tune the machine learning model.

[0024] By providing labeled data to the machine learning model, the machine learning model may learn to detect opioid administrations with maximum accuracy. The machine learning model may further be configured to predict the moment at which an opioid administration occurs. The machine learning model may be configured to determine the drug plasma concentration as a function of time. In certain implementations, the drug plasma concentration may be proportional to the dose times an exponential factor. For example, the drug plasma concentration may be given by equation 1 :D (t) oc0(e_feet— e~kat) (1).Here, D(t) refers to the concentration of an opioid as a function of time. Do represents the initial drug concentration (e.g., the dose size). Keand Kaare, respectively, the drug elimination and absorption rate constants that depend on the particular opioid.

[0025] In some implementations, a hybrid loss model may be used to fine-tune the machine learning model so that it may determine the presence of an opioid administration and the timing thereof. For opioid detection, where the model has to predict whether an administration occurred in a time-window, a binary-weighted cross-entropy loss between the actual administration class and the predicted administration class can be used as the hybrid loss model. In some implementations, a weighted kappa index may be used as a loss for opioid moment prediction to measure the distance between the actual moment of administration and the predicted moment of administration (e.g., which minute or temporal coordinate) in the time-window. Unlike traditional classification loss functions such as cross-entropy or hinge, a weighted kappa index weighs the disagreement between ground truth and predicted moment based on how far apart they are.

[0026] FIG. IB illustrates a visual representation of the process 100 for the training of a machine learning model in accordance with some embodiments described herein. At 102 of FIG. IB, a plurality of multimodal physiological data are required. The plurality of multimodal physiological data may be acquired in the form of signals from, e.g., a wearable device. At 104, the plurality of multimodal data are divided into a plurality of sliding windows. At 106, the multimodal physiological signals in a subset of the plurality of sliding windows are modified. For example, the multimodal physiological signals in the subset of the plurality of sliding windowsmay be modified so as to mimic the effect of the administration of an opioid on the multimodal physiological signals. At 108, the plurality of sliding windows and the subset of the plurality of sliding windows in which signals have been modified are provided as inputs to a machine learning model. At 110, the machine learning model identifies a moment of opioid administration. At 112, the machine learning model is adjusted by teaching the machine learning model a pharmacokinetic equation.

[0027] FIG. 2A illustrates a process 200 for determining an amount of distress experienced by a user based on multimodal physiological signals. For example, a user may experience distress in the form of stress, pain, and / or cravings. A machine learning model may be used to implement the process 200 at least in part in order to predict at least one of a stress score, a pain score, and a craving score. In some implementations, the machine learning model may be configured to estimate at least one of a stress score, a pain score, and a craving score for a user based on multimodal physiological signals that are passively and continuously collected.

[0028] At 202, a plurality of multimodal physiological signals are acquired. The multimodal physiological signals used to predict stress, pain, and craving scores may be acquired, for example, from a user by a device (see, e.g., system 600 of FIG. 6). The input / output device used to acquire the multimodal physiological signals may comprise a wearable sensor, such as a smart watch.

[0029] At 204, the plurality of multimodal physiological signals are encoded. For example, a processor may be configured to encode the acquired plurality of multimodal physiological signals. The encoding of the plurality of multimodal physiological signals may be configured to preserve important information about the multimodal physiological signals while removing noise from the multimodal physiological signals. The result of the encoding of the plurality of multimodal physiological signals may be that the processor generates a plurality of encoded multimodal physiological signals.

[0030] The processor may be configured to encode 204 the acquired plurality of multimodal physiological signals using a root-level encoder. The root-level module comprises a data encoder configured to extract the global features from the plurality of multimodal physiological signals. For example, the data encoder may be configured to extract patterns that lie in the plurality of multimodal physiological signals.

[0031] At 205, the plurality of encoded multimodal physiological signals further may be embedded so as to preserve characteristics of the encoded multimodal physiological signals that explain information related to at least one of stress, pain, and cravings of a user. For example, a processor may be configured to generate a plurality of embeddings representing the plurality of encoded multimodal physiological signals. In some implementations, the plurality of encoded multimodal physiological signals may be embedded by a group-level feature encoder. The group- level feature encoder may be configured to extract group-level characteristics from the plurality of encoded multimodal physiological signals. For example, the group-level feature encoder may be configured to extract shared physiological patterns from the plurality of encoded multimodal physiological signals acquired from users with similar traits (e.g., demographic information, physical characteristics such as height and weight, etc.). In some implementations, the group-level feature encoder comprises an individual encoder for each of a plurality of heuristic user groups (e.g., user groups that are randomly initialized).

[0032] At 207, the plurality of embeddings generated at 205 may be combined with or associated with covariate information 206. For example, the covariate information may comprise demographic information for a user. The covariate information may comprise at least one of prescription history information and drug history information for a user. The covariate information may be acquired, for example, from electronic health records (EHRs) associated with users of the system that implements the process 200.

[0033] At 208, the plurality of embeddings and the covariate information may be provided as an input to a machine learning model. For example, the plurality of embeddings and the covariate information may be provided as an input to the machine learning model implemented by the processor 610 of FIG. 6.

[0034] In some implementations, machine learning model uses the group-level encoder to extract shared physiological patterns using a branching mechanism. For example, the branching mechanism may be configured to dynamically assign users to optimal groups during the training of the machine learning model. In other words, the branching mechanism may be configured to perform dynamic subject clustering during the training of the machine learning model. In some implementations, the branching mechanism may be configured to dynamically assign users tooptimal groups while the machine learning model is being trained to predict an affective state (e.g., a state associated with stress, pain, or craving) of a user. The group-level features capture shared physiological patterns among individuals with similar traits, reducing variability and enhancing model generalization for the target classification task.

[0035] At 210 of the process 200, the machine learning model may be configured to identify latent groups of users who have similar physiological signals. For example, the group-level encoder may be configured to identify latent groups of users having similar physiological signals. The machine learning model may be configured to predict stress, pain, and craving levels of users by dynamically clustering the users with dynamic learning. For example, the system may create a head for each user. By using, for example, a Gumble trick sampling method, the machine learning model can associate users with different branches of a machine learning network based on their common physiology. The machine learning model may be configured to predict stress, pain, and craving levels of users by dynamically clustering the users with dynamic learning.

[0036] The group-level encoder may be configured to output an embedding that represents which users are associated with a particular group based on the extracted group-level characteristics. The group-level characteristics extracted by the population-level feature encoder may be provided as inputs to a personalized user-level module 212. The group-level embedding fed to the personalized user-level module 212 may comprise a weighted sum of group-level embeddings with weights representing a probability with which a user belongs to a particular group. The personalized user-level module 212 may be constructed for each individual. The userlevel module may be configured to receive from the group-level encoder embeddings. The userlevel module is configured to capture a user-specific embedding from the group-level data.

[0037] In some implementations, the system is configured to compare an RR interval (e.g., the time elapsed between two successive R-waves of an electrocardiogram, which may be received with the multimodal physiological data, and which may be encoded and embedded as described) to a self-reported feeling of stress, pain, or craving by a user.

[0038] At 212, at least one of a stress score, a pain score, and a craving score is predicted. By training on the task of predicting levels of stress, pain, and cravings with heart rate variability dataas input, the machine learning model learns branches in the network that assign each individual from whom multimodal physiological signals was received to the optimal group.

[0039] FIG. 2B illustrates a process 250 for predicting an opioid misuse risk score based on multimodal physiological signals. As in FIG. 2A, the process 250 for predicting an opioid misuse risk score may comprise at 202 acquiring a plurality of multimodal physiological signals. The plurality of multimodal physiological signals may be acquired from, e.g., a wearable such as a smart watch. At 204, the plurality of multimodal physiological signals may be encoded and embedded. The plurality of multimodal physiological signals may be encoded and embedded by at least one processor. At 206, the plurality of multimodal physiological signals may be combined with covariate information. The covariate information may include, for example, a user’s prescription history, a user’s symptom history, a user’s demographic history, and / or a user’s selfreported scores regarding signs or feelings of distress. Such signs or feelings of distress for which a user may self-report scores include stress, pain, and cravings. At 208, the plurality of embeddings and the covariate information may be provided as an input to a machine learning model. At 210 of the process 200, the machine learning model may be configured to identify latent groups of users who have similar physiological signals. At 212, at least one of a stress score, a pain score, and a craving score is predicted.

[0040] As shown in FIG. 2B, in some implementations, the process 250 further comprises determining 214 a trajectory for at least one of stress, pain, and craving. For example, the machine learning model may be configured to predict a stress, pain, or craving trajectory using stress, pain, or craving scores that are computed periodically (e.g., hourly, or at some other regular temporal interval). A machine learning model may be configured to estimate an opioid misuse risk score for a user based at least in part on the determined trajectory for at least one of stress, pain, and craving.

[0041] In some implementations, the process 250 further comprises applying 216 a topological non-linear analysis to at least one of the predicted stress trajectory, the predicted pain trajectory, and the predicted craving trajectory to determine if the respective signal (e.g., stress, pain, or craving) is chaotic. It is known that stress, pain, and craving processes may follow deterministic trajectories in opioid misusers prior to administration. As such, if the topological non-linearanalysis fails to detect the presence of chaos in the at least one of the predicted stress trajectory, the predicted pain trajectory, and the predicted craving trajectory, then the predicted trajectory can be used to predict a risk of opioid misuse. For example, the machine learning model may predict a probability 218 that a user misuses an opioid based on the absence of chaos in an analyzed trajectory.

[0042] In some implementations, the covariate information including, for example, a user’s prescription history, symptom history, and demographics (such as age, education level, and years of opioid usage, etc.) may be provided 220 as an input to a large language model (LLM) 221. The large language model may be configured to leverage qualitative covariate information (e.g., prescription history, a body part of a user that may be experiencing pain) to provide additional context to the inferred misuse probability 218. By providing the covariate information with careful semantic prompt engineering, the LLM 221 may be configured to extract informative contextual features from the covariate information. For example, the LLM 221 may be configured to extract informative contextual features from the covariate information after being prompted with a user’s age, education level, income, prescription record (e g., medication type and dose), symptoms, and self-reported affective states (e.g., self-reported feelings of craving, pain, and stress). The contextual features extracted from the covariate information may be concatenated 222 with the inferred misuse probability 218 to compute a final misuse risk score 223.

[0043] FIG. 3 illustrates a system 300 for using multimodal physiological signals captured during the performance of a cognitive task to predict an opioid misuse risk score. A user 302 of the system 300 may wear a wearable device 304. The wearable device 304 may be configured to acquire a plurality of multimodal physiological signals 306 from the user. A processor may be configured to extract physiological features 307 from the plurality of multimodal physiological signals. Further, the user 302 of the system 300 may use a client device 308 to perform a cognitive task. The client device 308 may be configured to perform interactive cognitive task-driven data collection 310 to capture episodic multimodal physiological signals during the performance of the cognitive task by a user. The machine learning model may be configured to extract important features 312 from cognitive and physiological signals received during the performance of the cognitive task, for example, as multimodal data. The extracted features can be mapped to an opioid misuse score 314. The cognitive task may comprise, for example, a systematic way to understandif patient can control an inhibition. For example, the cognitive task may comprise or otherwise resemble a standard Go / No-go task that tests a user’s capacity not to respond to a stimulus.

[0044] FIG. 4 illustrates a system 400 for identifying an intervention trigger based on at least one of a detected opioid administration, a predicted stress score, a predicted pain score, a predicted craving score, and a predicted opioid misuse risk score. A user 402 of the system 400 may wear a wearable device 404 and may interact with a client device 405. Further, the user 402 of the system 400 may use the client device 405 to perform a cognitive task. The client device 405 may be configured to perform interactive cognitive task-driven data collection to capture episodic multimodal physiological signals during the performance of the cognitive task by a user. The wearable device 404 may be configured to acquire a plurality of multimodal physiological signals 406 from the user. A processor may be configured to identify digital biomarkers 408 in the plurality of multimodal physiological signals 406. The processor may be configured to implement a machine learning model, such as a neural network. The processor is configured to identify an intervention trigger 410 based on the digital biomarkers 408 and on the episodic multimodal physiological signals captured during the performance of the cognitive task by the user. If the processor identifies an intervention trigger 410, the processor may be configured to select an intervention 412 for the user. The machine learning model may be configured to select a certain intervention 412 based on the intervention trigger 410, as certain interventions 412 can be most effective for certain triggers 410.

[0045] FIG. 5 illustrates a system 500 for forecasting a change in a multimodal physiological signal after an intervention action has been taken. For example, the system 500 may forecast a change in a multimodal physiological signal after an intervention selected by the system 400 of FIG. 4 has been performed. The system 500 may comprise a digital twin simulator model 502. The digital twin simulator model 502 may be implemented, for example, by a processor.

[0046] As described with respect to FIG. 4, in some implementations, a system is configured to select an intervention for a user based on digital biomarkers. Responsive to the selection of the intervention, a user may take intervention actions 504 at a time t. A user may perform i such intervention actions 504. (where i = 1 , 2, . . . , N for N total actions taken). The digital twin simulator model 502 may analyze the taken intervention action 504, a plurality of acquired multimodalphysiological signals 506, and demographic information 508. Based on the intervention action 504, the plurality of acquired multimodal physiological signals 506, and the demographic information 508, the digital twin simulator model 502 is configured to forecast the plurality of acquired multimodal physiological signals 506 at a time subsequent to the time t. For example, the digital twin simulator model 502 may be configured to determine a forecasted plurality of multimodal physiological signals 510 at a time t + 1 (e.g., t plus one hour).

[0047] The system 500 may be further configured to extract a plurality of digital biomarkers 512 from the plurality of multimodal physiological signals 506. Similarly, the system 500 may be configured to extract a plurality of digital biomarkers 514 from the forecasted plurality of multimodal physiological signals 510 at the time t + 1. The system 500 may be further configured to compute an estimated change between the plurality of digital biomarkers 512 from the plurality of multimodal physiological signals 506 and the plurality of digital biomarkers 514 from the forecasted plurality of multimodal physiological signals 510.

[0048] The system 500 may also be configured to compute an estimated change between a parameter (e.g., a detected opioid administration, stress score, pain score, craving score, stress trajectory, pain trajectory, craving trajectory, and opioid misuse risk score) predicted based on the plurality of digital biomarkers 512 and the parameter predicted based on the plurality of digital biomarkers 514.

[0049] For example, the system 500 may be configured to determine an estimated change between a time of an opioid administration predicted based on the plurality of digital biomarkers 512 and a time of an opioid administration predicted based on the plurality of digital biomarkers 514.

[0050] The system 500 may use, for example, the systems 100-400 of FIGs. 1-4 to determine the plurality of digital biomarkers 512 and the plurality of digital biomarkers 514. The system 500 may use, for example, the systems 100-400 of FIGs. 1-4 to compute the estimated change between the plurality of digital biomarkers 512 from the plurality of multimodal physiological signals 506 and the plurality of digital biomarkers 514 from the forecasted plurality of multimodal physiological signals 510.

[0051] The system 500 is further configured to determine a cost / reward function 516 based on the estimated change. In some implementations, the cost / reward function 516 will be configured to compute a function of the estimate change. For example, the cost / reward function 516 may output +1 if an opioid misuse risk score computed based on the plurality of digital biomarkers 514 decreases relative to an opioid misuse risk score computed based on the plurality of digital biomarkers 512. The cost / reward function 516 may output -1 if an opioid misuse risk score computed based on the plurality of digital biomarkers 514 increases relative to an opioid misuse risk score computed based on the plurality of digital biomarkers 512. At 518, an optimal policy (e.g., a recommended intervention) is determined based on the output of the cost / reward function 516. The optimal policy 518 may be used to recommend an action (e.g., an intervention) to a user.

[0052] In some implementations, the current subject matter may be configured to be implemented in a system 600, as shown in FIG. 6. For example, aspects disclosed herein may be at least in part physically comprised on system 600. To illustrate further system 600 may further include an operating system, a hypervisor, and / or other resources, to provide the noted machine learning models. The system 600 may include a processor 610, a memory 620, a storage device 630, and an input / output device 640. Each of the components (e.g., 610, 620, 630 and 640) may be interconnected using a system bus 650. The processor 610 may be configured to process instructions for execution within the system 600. In some implementations, the processor 610 may be a single-threaded processor. In alternate implementations, the processor 610 may be a multithreaded processor.

[0053] In some implementations, the system 600 may be configured to implement a method for detecting opioid administrations using wearables and pharmacokinetics. For example, the input / output devices 640 may comprise wearables or wearable sensors (e.g., a smart watch or other health monitor) configured to acquire multimodal physiological data from a user. The multimodal data may include, for example, electrocardiogram (ECG) signals (e.g., heart rate variability (HRV) data) or skin temperature data.

[0054] The processor 610 may be configured to implement a machine learning model. The memory 620 may comprise instructions that are used by the processor 610 to implement the machine learning model. In some implementations, the machine learning model may beconfigured to detect opioid administrations based at least in part on pharmacokinetic principles. The processor 610 may be configured to implement a process for detecting an administration of an opioid, such as the process 100 of FIG. 1 A. The processor 610 may be configured to implement a process for determining at least one of a stress score, a pain score, and a craving score, such as the process 200 of FIG. 2 A.

[0055] The processor 610 may be further configured to process instructions stored in the memory 620 or on the storage device 630, including receiving or sending information through the input / output device 640. The memory 620 may store information within the system 600. In some implementations, the memory 620 may be a computer-readable medium. In alternate implementations, the memory 620 may be a volatile memory unit. In yet some implementations, the memory 620 may be a non-volatile memory unit. The storage device 630 may be capable of providing mass storage for the system 600. In some implementations, the storage device 630 may be a computer-readable medium. In alternate implementations, the storage device 630 may be a floppy disk device, a hard disk device, an optical disk device, a tape device, non-volatile solid state memory, or any other type of storage device.

[0056] The input / output device 640 may be configured to provide input / output operations for the system 600. In some implementations, the input / output device 640 may include a keyboard and / or pointing device. In alternate implementations, the input / output device 640 may include a display unit for displaying graphical user interfaces. For example, the input / output device 640 may comprise an Empatica E4 (Empatica, Milan, Italy). The E4 is a research-grade wrist-worn device that continuously and passively captures various physiological signals. It is equipped with several sensors that measure instantaneous heart rate (HR) sampled at fs=lHz, skin temperature (TEMP, / s=4Hz), electrodermal activity (EDA, / ^=4Hz), and triaxial acceleration (ACC, / .,=32Hz). Raw data is initially stored on the device’s onboard memory and may be transferred to Empatica’s secure cloud-based server (Empatica Connect) for further analysis.

[0057] One or more aspects or features of the subject matter described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs, field programmable gate arrays (FPGAs) computer hardware, firmware, software, and / or combinations thereof. These various aspects or features can include implementation in one or more computer programs that areexecutable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0058] These computer programs, which can also be referred to as programs, software, software applications, applications, components, or code, include machine instructions for a programmable processor, and can be implemented in a high-level procedural and / or object- oriented programming language, and / or in assembly / machine language. As used herein, the term “machine-readable medium” refers to any computer program product, apparatus and / or device, such as for example magnetic discs, optical disks, memory, and Programmable Logic Devices (PLDs), used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor. The machine-readable medium can store such machine instructions non-transitorily, such as for example as would a non-transient solid-state memory or a magnetic hard drive or any equivalent storage medium. The machine-readable medium can alternatively or additionally store such machine instructions in a transient manner, such as for example, as would a processor cache or other random access memory associated with one or more physical processor cores.

[0059] To provide for interaction with a user, one or more aspects or features of the subject matter described herein can be implemented on a computer having a display device, such as for example a cathode ray tube (CRT) or a liquid crystal display (LCD) or a light emitting diode (LED) monitor for displaying information to the user and a keyboard and a pointing device, such as for example a mouse or a trackball, by which the user may provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. For example, feedback provided to the user can be any form of sensory feedback, such as for example visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form,including acoustic, speech, or tactile input. Other possible input devices include touch screens or other touch-sensitive devices such as single or multi-point resistive or capacitive track pads, voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices and associated interpretation software, and the like.

[0060] In the descriptions above and in the claims, phrases such as “at least one of’ or “one or more of’ may occur followed by a conjunctive list of elements or features. The term “and / or” may also occur in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradicted by the context in which it used, such a phrase is intended to mean any of the listed elements or features individually or any of the recited elements or features in combination with any of the other recited elements or features. For example, the phrases “at least one of A and B;” “one or more of A and B;” and “A and / or B” are each intended to mean “A alone, B alone, or A and B together.” A similar interpretation is also intended for lists including three or more items. For example, the phrases “at least one of A, B, and C;” “one or more of A, B, and C;” and “A, B, and / or C” are each intended to mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together.” Use of the term “based on,” above and in the claims is intended to mean, “based at least in part on,” such that an unrecited feature or element is also permissible.

[0061] The subject matter described herein can be embodied in systems, apparatus, methods, and / or articles depending on the desired configuration. The implementations set forth in the foregoing description do not represent all implementations consistent with the subject matter described herein. Instead, they are merely some examples consistent with aspects related to the described subject matter. Although a few variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations can be provided in addition to those set forth herein. For example, the implementations described above can be directed to various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of several further features disclosed above. In addition, the logic flows depicted in the accompanying figures and / or described herein do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Other implementations may be within the scope of the following claims.

Claims

What is claimed:

1. A method compri sing : acquiring a plurality of multimodal physiological signals; dividing the plurality of multimodal physiological signals into a plurality of sliding windows containing a respective multimodal physiological signal; selecting a subset of the plurality of sliding windows that contain the respective multimodal physiological signal, modifying the respective multimodal physiological signal contained in the subset of the plurality of sliding windows; providing, as one or more inputs to a machine learning model, the plurality of sliding windows that contain the respective multimodal physiological signal and the subset of the plurality of sliding windows that contain the respective modified multimodal physiological signal; and identifying, using one or more outputs of the machine learning model, at least one moment of opioid administration.

2. The method of claim 1, further comprising: adjusting the machine learning model to detect drug plasma concentration in the plurality of multimodal physiological signals by teaching the machine learning model to learn pharmacokinetics of drug plasma concentrations.

3. The method of any of claims 1 and 2, wherein the plurality of multimodal physiological signals are acquired by a wearable device.

4. The method of any of claims 1-3, wherein a first sliding window of the plurality of sliding windows and a second sliding window of the plurality of sliding windows comprise overlapping multimodal physiological signals.

5. The method of any of claims 1-4, wherein modifying the multimodal physiological signal comprises changing the signal to mimic an opioid administration.

6. The method of any of claims 1 -5, wherein between forty five percent of the plurality of sliding windows and fifty five percent of the plurality of sliding windows are selected for modification.

7. The method of any of claims 1-6, wherein the machine learning model is trained to identify the moment of opioid administration on the plurality of sliding windows and the subset of the plurality of sliding windows.

8. The method of any of claims 1-7, wherein changing the signal to mimic the opioid administration comprises multiplying the signal by an amplification factor and by a deterministic function.

9. The method of any of claims 1-8, wherein identifying the moment of opioid administration comprises identifying a signal of the plurality of multimodal physiological signals that mimics an opioid administration.

10. A method comprising: acquiring a plurality of multimodal physiological signals; encoding the plurality of multimodal physiological signals to generate a plurality of encoded multimodal physiological signals; generating a plurality of embeddings representing the plurality of encoded multimodal physiological signals; combining the plurality of embeddings with covariate information; providing the plurality of embeddings and the covariate information to a machine learning model; identifying, using the machine learning model, similar groups of physiological signals; and determining an amount of user distress represented by the plurality of multimodal physiological signals.

11. The method of claim 10, wherein the amount of distress is determined as a score, the score comprising at least one of a stress score, a pain score, and / or a craving score.

12. The method of any of claims 10 and 11, further comprising predicting at least one of a stress trajectory, a pain trajectory, and a craving trajectory.

13. The method of claim 12, further comprising performing a non-linear topological analysis on at least one of the stress trajectory, the pain trajectory, and the craving trajectory, and, when the non-linear topological analysis fails to detect a presence of chaos in the at least one of the stress trajectory, the pain trajectory, and the craving trajectory, determining a probability of opioid misuse.

14. The method of any of claims 10-13, wherein identifying similar groups of physiological signals comprises, by the machine learning model, dynamically clustering users having similar physiological signals and demographic information into groups.

15. The method of any of claims 10-14, further comprising acquiring a plurality of cognitive signals.

16. The method of any of claims 12-15, further comprising identifying an intervention trigger based on at least one of the stress trajectory, the pain trajectory, the craving trajectory, or the probability of opioid misuse.

17. The method of claim 16, further comprising selecting an intervention based on the intervention trigger.

18. The method of any of claims 10-17, wherein the covariate information comprises at least one of prescription history, drug history, and demographic information.

19. A system comprising: at least one programmable processor; and a non-transitory machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform operations comprising: acquiring a plurality of multimodal physiological signals;dividing the plurality of multimodal physiological signals into a plurality of sliding windows containing a respective multimodal physiological signal; selecting a subset of the plurality of sliding windows that contain the respective multimodal physiological signal, modifying the respective multimodal physiological signal contained in the subset of the plurality of sliding windows; providing, as one or more inputs to a machine learning model, the plurality of sliding windows that contain the respective multimodal physiological signal and the subset of the plurality of sliding windows that contain the respective modified multimodal physiological signal; and identifying, using one or more outputs of the machine learning model, at least one moment of opioid administration.

20. A system comprising: at least one programmable processor; and a non-transitory machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform operations comprising: acquiring a plurality of multimodal physiological signals; encoding the plurality of multimodal physiological signals to generate a plurality of encoded multimodal physiological signals; generating a plurality of embeddings representing the plurality of encoded multimodal physiological signals; combining the plurality of embeddings with covariate information; providing the plurality of embeddings and the covariate information to a machine learning model; identifying, using the machine learning model, similar groups of physiological signals; and determining an amount of user distress represented by the plurality of multimodal physiological signals.

21. A computer program product comprising a non -transitory machine readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising: acquiring a plurality of multimodal physiological signals; dividing the plurality of multimodal physiological signals into a plurality of sliding windows containing a respective multimodal physiological signal; selecting a subset of the plurality of sliding windows that contain the respective multimodal physiological signal, modifying the respective multimodal physiological signal contained in the subset of the plurality of sliding windows; providing, as one or more inputs to a machine learning model, the plurality of sliding windows that contain the respective multimodal physiological signal and the subset of the plurality of sliding windows that contain the respective modified multimodal physiological signal; and identifying, using one or more outputs of the machine learning model, at least one moment of opioid administration.

22. A computer program product comprising a non-transitory machine readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising: acquiring a plurality of multimodal physiological signals; encoding the plurality of multimodal physiological signals to generate a plurality of encoded multimodal physiological signals; generating a plurality of embeddings representing the plurality of encoded multimodal physiological signals; combining the plurality of embeddings with covariate information; providing the plurality of embeddings and the covariate information to a machine learning model; identifying, using the machine learning model, similar groups of physiological signals; and determining an amount of user distress represented by the plurality of multimodal physiological signals.

Images