Predicting persistence or reduction in user interactions across sessions using machine learning models and event data

Abstract

To provide systems and methods for detecting expected reductions of interactions across sessions.SOLUTION: The method includes: identifying event data that identifies interactions by a user with a first session for digital therapeutic content; applying the event data to a machine learning (ML) model; generating a likelihood of the user interacting with a second session for digital therapeutic content based on applying the event data to the ML model; detecting an expected reduction in interactions by the user with the second session, responsive to the likelihood of the user not satisfying a threshold; and providing an output for the user based on detecting the expected reduction in interactions. In this manner, efficacy of medication that the user is taking to address his or her condition may be improved.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] Predicting persistence or decline of user interactions between sessions using machine learning models and event data. [Background technology]

[0002] During a session, the computing device and the remote server can communicate data related to user interactions detected through the user interface. For example, a digital therapeutic application running on the computing device can receive data from the remote server and present digital therapeutic content (e.g., in the form of audio, visual, or textual content) through the user interface. The digital therapeutic may include evidence-based therapeutic interventions delivered directly to the user through the application on the computing device. The digital therapeutic content presented on the user interface may prompt the user to perform activities that trigger specific parts of the user's nervous system to facilitate addressing a condition. Each time a user interaction is detected with one of the user interface elements, the computing device may generate data based on the user interaction for transmission over the network. The remote server hosting the application's resources may return response data to the computing device. Summary of the Invention [Problem to be solved by the invention]

[0003] A session may involve various processing operations on both the computing device and a remote server, such as processing user interactions to invoke functions, rendering user interface elements, and communicating data. Depending on the complexity of the processing operations and the rate of user interactions with user interface elements, the amount of processing power, memory, and network bandwidth consumed may be significant. Predicting user interactions, related to the allocation of computing resources, presents many technical challenges. One is that users are not uniform in how they behave, and a particular user may exhibit behaviors that vary from other users, making it difficult to treat all users the same. Another is that interaction data may be sparse and insufficient to make accurate predictions, especially for new users of an application. Furthermore, scalability in predicting user interactions with a large number of users and user interface elements may also be challenging.

[0004] The inability to accurately and precisely predict user interactions can result in inefficient allocation of resources (e.g., computing power and network bandwidth) from delivering data for the presentation of content and interactivity through user interface elements. Inaccurate predictions can lead to the selection of content that does not result in user interaction, thereby wasting computing resources and degrading the quality of human-computer interactions (HCI) between the user and the user interface. Furthermore, providing such content and interactivity and collecting poor-quality user interaction data can result in a negative feedback loop in predictive analytics.

[0005] In the context of digital therapeutics, for users to achieve full improvement, they should interact with the user interface and perform activities as directed by the digital therapeutic content intended to address their condition. Lack of adherence, such as little or no interaction with the user interface, significantly impacts user outcomes with digital therapeutics. Dropout rates can increase significantly as users progress through treatment. Lack of user engagement can impede progress and prolong the healing process for the user's condition. One user consequence of not interacting with a digital therapeutic can be a stagnation or worsening of the user's condition. Delivery of ineffective digital therapeutic content can lead to both inefficient allocation of resources (e.g., computing power and network bandwidth) and reduced quality of the HCI. [Means for solving the problem]

[0006] To address these and other technical challenges, digital therapeutic applications can use specific event data from prior sessions and machine learning (ML) models to predict the likelihood that a user will interact in a subsequent session provided to the user. The ability to predict the likelihood that a user will interact in a subsequent session has several technical advantages and benefits. For one, the prediction can be used to decide whether to continue a digital therapeutic session or to provide targeted countermeasures aimed at improving a user's adherence to a treatment regimen. This allows computing systems to better and more efficiently allocate computing resources and network bandwidth when performing operations to provide further digital therapeutics or countermeasures.

[0007] Second, improved treatment adherence allows users to receive the full benefits and effectiveness of the digital therapeutic in improving, mitigating, or otherwise addressing the user's condition. Thus, computing resources consumed in delivering digital therapeutic content to which the user did not respond may be saved. Additionally, the computing system may alert a healthcare provider or digital therapeutic administrator when a user is predicted to drop out, so the healthcare provider (or support member) may identify which users to manually intervene with (e.g., contact). For example, a signal may be output indicating that a user is predicted to drop out, which may be used to indicate to the provider that intervention is needed. In-person intervention may not only improve the effectiveness of the digital therapeutic provided to the user, but also the medications the user is taking in conjunction with the digital therapeutic regimen.

[0008] To that end, the computing system can present digital therapeutic content via a user interface and instruct the user in performing activities. The activities may be according to specific tasks, such as an implicit association test (IAT), attention bias modification training (ABMT), an emotional faces memory task (EFMT), a digital support tool (DST), and adaptive goal setting (AGS), among others. During a session, the computing system can monitor event data of interactions during the session to provide digital therapeutic content to address the condition. The event data can include various information related to the interaction and contextual factors, such as the identity of the content provided, the duration of the activity, metrics indicating the number or rate of interactions, metrics related to the activity, task, or condition, prior dropouts, and an indication of whether the user contacted a support center.

[0009] The computing system may apply the event data to a model to determine the likelihood that a user will interact with the digital therapeutic content in a subsequent session. A subsequent session may correspond to digital therapeutic content provided to the user in the next time period (e.g., hour, day, or week). The model may be initialized, trained, and established using training data formed from prior sessions or previous users with similar profiles (e.g., the same condition) or other factors. The training data may include a user's past interactions in a session and their performance in subsequent sessions. From the training data, the model can learn to predict the likelihood of user interaction in a subsequent session. The model may show that certain factors in the event data, such as recent dropout or use of the digital therapeutic application on the previous day, may be more predictive of future dropout than other parameters.

[0010] The computing system can compare the likelihood to a threshold to determine whether the user is predicted to drop out of the digital therapeutic session or to continue. If the likelihood exceeds the threshold, the computing system can predict that the user will continue interaction in a subsequent session. Alternatively, the computing system may continue the subsequent session as defined above. On the other hand, if the likelihood does not exceed the threshold, the computing system can predict a decrease in user interaction in subsequent sessions.

[0011] The computing system can select countermeasures to address the expected drop in interaction. Countermeasures can include, for example, pre-generated notifications to the user to encourage engagement, notifying a support center that the user is likely to drop out, triggering a generative artificial intelligence (AI) model to create a custom notification, or modifying the digital therapeutic content. The computing system can provide the selected countermeasure as an output to the application or support center. If the likelihood indicates a low risk of dropping out in subsequent sessions, the computing system can select and provide a notification to the user or support center indicating that the user is at low risk of dropping out. If the likelihood indicates a high risk of the user dropping out in subsequent sessions, the computing can select and provide additional support and interventions to increase the user's adherence.

[0012] By using ML models to predict the likelihood of a user interacting in a subsequent session based on event data from a previous session, an application can decide whether to continue the session or provide countermeasures to counter a potential decline in user interaction. Reliance on previous event data from the user and other users can overcome issues related to a lack of data, as well as the type of event data (e.g., information about the user interaction and related contextual factors), in accurately and precisely predicting user interactions. Accurate predictions allow an application to efficiently allocate computing resources (e.g., computing power and network bandwidth) when deciding whether to continue digital therapeutic content or provide countermeasures.

[0013] By providing targeted interventions when a user is predicted to drop out, the application may increase the user's adherence to the digital treatment delivered through the session, further reducing the waste of computing resources that would otherwise be consumed delivering sessions in which the user does not interact. Increased adherence may also improve the quality of the HCI between the user and the application, more effectively treating, improving, or otherwise addressing the user's condition. Providing a digital treatment through the application in this manner may increase the effectiveness of medications the user is taking to address their condition.

[0014] Aspects of the present disclosure are directed to systems and methods for detecting an expected decrease in interaction between sessions. One or more processors coupled with a memory can identify event data identifying interactions by a user in a first session for digital therapeutic content to address a condition. The one or more processors can apply the event data to a machine learning (ML) model. The ML model can be trained using a plurality of examples. Each of the plurality of examples can include (i) respective event data identifying interactions by a corresponding user with the respective session for the digital therapeutic content, and (ii) respective identifications of whether the corresponding user interacted with a subsequent session after the respective session. The one or more processors can generate a likelihood that the user will interact with a second session for the digital therapeutic content to address the condition following the first session based on applying the event data to the ML model. The one or more processors can detect an expected decrease in the user's interaction with the second session in response to a likelihood that the user will not meet a threshold. The one or more processors can provide an output to the user based on detecting the expected decrease in interaction.

[0015] In some embodiments, the one or more processors may generate a likelihood that the second user will interact with a fourth session following the third session for the second user based on applying the event data identifying the second user's interaction with the third session to the ML model. In some embodiments, the one or more processors may detect an expected persistence of the second user's interaction with the fourth session in response to the likelihood that the second user satisfies a threshold. In some embodiments, the one or more processors may provide an output for the second user based on detecting the expected persistence of the interaction.

[0016] In some embodiments, the one or more processors can communicate an output including the notification to a computing device at a support center, the computing device being configured to initiate communication with the user after receiving the notification. In some embodiments, the one or more processors can select an output from a plurality of output candidates based on a likelihood that the user will interact with the second session, the plurality of output candidates including at least one of (i) a first output candidate to display on the computing device to initiate communication via at least one of a phone call, an email, or an in-app chat, or (ii) a second output candidate to display on the computing device to monitor subsequent interaction by the user.

[0017] In some embodiments, the one or more processors can communicate an output including a predefined notification to direct the user to interact with the second session provided to the user. In some embodiments, the one or more processors can identify profile information associated with the user in response to detecting an expected decrease in interaction by the user. In some embodiments, the one or more processors can apply the profile information to a generative transformation model to generate a notification that directs the user to interact with the second session that is provided to the user. In some embodiments, the one or more processors can provide a notification that directs the user to interact with the second session that is provided to the user.

[0018] In some embodiments, the one or more processors may modify the digital therapeutic content presented in the second session based on detecting the expected decrease in interaction, and providing the output further includes providing the second session of the modified digital therapeutic content to the user. In some embodiments, the one or more processors may identify second event data identifying interactions by the user with the second session of digital therapeutic content to address the condition. In some embodiments, the one or more processors may apply the second event data to a machine learning (ML) model to generate a likelihood that the user will interact with a third session of digital therapeutic content to address the condition following the second session.

[0019] In some embodiments, the one or more processors can determine whether a change in responsiveness occurs in the user based on a comparison of the likelihood that the user will interact with the second session and the likelihood that the user will interact with the third session. The event data further includes at least one of: (i) an identifier of the digital therapeutic content provided in the first session; (ii) a time at which the first session is provided; (iii) a sequence identifier of the first session within the plurality of sessions; (iv) a metric indicative of the degree of interaction in the first session; (v) an indication of whether the user contacted the computing device for support; (vi) a type of user device associated with the user; (vii) a number of pre-expected declines; (ix) a metric associated with a task performed in the first session; or (x) a metric associated with a condition. The user is taking medication to address a condition at least partially concurrently with at least one of the first session or the second session. [Brief explanation of the drawings]

[0020] The above and other objects, aspects, features and advantages of the present disclosure will become more apparent and will be better understood by referring to the following description taken in conjunction with the accompanying drawings. [Figure 1] 1 illustrates a block diagram of a system for detecting expected decline or persistence of interaction between sessions, according to an example embodiment. [Figure 2] FIG. 1 illustrates a block diagram of a process for generating a user's likelihood of interacting in a subsequent session based on event data from a previous session in a system to detect an expected decline or persistence of interaction between sessions, according to an exemplary embodiment. [Figure 3]1 illustrates a block diagram of a process for generating an output based on a user's likelihood of interacting in a subsequent session within the system to detect expected decline or persistence of interaction between sessions, according to an illustrative embodiment. [Figure 4] 1 illustrates a block diagram of a session provisioning process in a system for detecting expected decline or persistence of interaction between sessions, according to an example embodiment. [Figure 5] 1 illustrates a flow diagram of a method of a system for detecting expected decline or persistence of interaction between sessions, according to an example embodiment. [Figure 6] 1 illustrates a block diagram of a server system and a client computer system according to an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] For purposes of reading the following description of the various embodiments, the sections of this specification and their respective contents are listed below.

[0022] Section A describes systems and methods for detecting expected interaction decline or persistence between sessions.

[0023] Section B describes network and computing environments useful for practicing the embodiments described herein.

[0024] A. Systems and Methods for Detecting Expected Decrease or Persistence of Interaction Between Sessions Referring now to FIGURE 1, a block diagram of a system 100 for detecting expected decline or persistence of interaction between sessions is depicted. In overview, the system 100 may include at least one session management service 105, a set of user devices 110A-N (hereinafter generally referred to as user devices 110), and a support center 180 having at least one support computing device 175 communicatively coupled to each other via at least one network 115. At least one of the user devices 110 (e.g., first user device 110A as depicted) includes at least one application 125. The application 125 may include or provide at least one user interface 130 having one or more user interface (UI) elements 135A-N (hereinafter generally referred to as UI elements 135). The session management service 105 may include, among other things, at least one session handler 140, at least one model applier 145, at least one interaction evaluator 150, at least one feedback generator 155, and at least one predictive model 160. The session management service 105 may include or have access to at least one database 120. The database 120 may store, maintain, or otherwise include one or more user profiles 165A-N (hereinafter generally referred to as user profiles 165) and training data 170. The functionality of the application 125 on the user device 110 may be partially executed on the session management service 105, or vice versa. Each component of the system 100 may be implemented using a computing system such as those described in Section B.

[0025] More particularly, session management service 105 (sometimes referred to generally herein as a messaging service) may be any computing device including one or more processors coupled with memory and software and capable of performing the various processes and tasks described herein. Session management service 105 may communicate with one or more user devices 110 and database 120 via network 115. Session management service 105 may be located, collocated, or otherwise associated with at least one computer system. The computer system may correspond to a data center, branch office, or site where one or more computers corresponding to session management service 105 are located.

[0026] Within the session management service 105, a session handler 140 can manage sessions and receive interactions from users. An interaction evaluator 150 can detect interactions with the application 125. A model applier 145 can use a predictive model 160 to determine the likelihood that a user will interact with a subsequent session. A feedback generator 155 can generate an output based on the likelihood of interaction to guide the user to interact with the digital therapeutic application. The predictive model 160 can include any machine learning model for determining the likelihood that a user will interact in a subsequent session.

[0027] The machine learning model architecture of the predictive model 160 may include, for example, a deep learning neural network (e.g., a convolutional neural network model architecture), a regression model (e.g., a linear or logistic regression model), a random forest, a support vector machine (SVM), a clustering algorithm (e.g., k-nearest neighbor), or a naive Bayes model. Generally, the predictive model 160 has at least one input and one output. The input and output are related via a set of weights. The input may include event data from at least one previous session. The output may include a likelihood that a user will interact in at least one subsequent session. The set of weights may follow a machine learning architecture. The machine learning model of the predictive model 160 may be trained using training data 170 (e.g., according to supervised learning).

[0028] User device 110 (sometimes referred to herein as an end-user computing device) may be any computing device that includes one or more processors coupled with memory and software and is capable of performing the various processes and tasks described herein. User device 110 may communicate with session management service 105 and database 120 via network 115. User device 110 may be a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, eyeglasses), or laptop computer. User device 110 may be used to access application 125. In some embodiments, application 125 may be downloaded (e.g., via a digital distribution platform) and installed on user device 110. In some embodiments, application 125 may be a web application having resources accessible via network 115.

[0029] The application 125 executing on the user device 110 may be a digital therapeutic application. The application 125 may present or provide sessions (sometimes referred to herein as therapy sessions) to address at least one condition of the user. End-user conditions may include, for example, chronic pain (e.g., related to or including arthritis, migraine, fibromyalgia, back pain, Lyme disease, endometriosis, repetitive stress injury, irritable bowel syndrome, inflammatory bowel disease, and cancer pain), skin pathologies (e.g., atopic dermatitis, psoriasis, atrophy of the skin, and eczema), cognitive disorders (e.g., mild cognitive impairment (MCI), Alzheimer's disease, multiple sclerosis, and schizophrenia), psychiatric disorders (e.g., affective disorders, bipolar disorder, obsessive-compulsive disorder, borderline personality disorder, and attention-deficit / hyperactivity disorder), substance use disorders (e.g., opioid use disorder, alcohol use disorder, tobacco use disorder, hallucinogen use disorder), other diseases (e.g., narcolepsy, tumors or cancer), and the like.

[0030] The end user may be taking medication to address a condition at least in part in association with use of the application 125 (e.g., any number of sessions). For example, if the medication is for pain, the end user may be taking acetaminophen, a nonsteroidal anti-inflammatory composition, an antidepressant, an anticonvulsant, or other composition, among others. For skin pathology, the end user may be taking a steroid, an antihistamine, or a topical antiseptic, among others. For cognitive impairment, the end user may be taking a cholinesterase inhibitor or memantine, among others. For narcolepsy, the end user may be taking a stimulant or an antidepressant, among others. The end user may also be participating in other psychotherapies for these conditions. In some embodiments, digital therapeutic content may be provided to the end user within the digital therapeutic application toward achievement of the end user's endpoint. The endpoint may be, for example, the end user's physical or mental goal, the completion of a medication regimen, or an endpoint indicated by a physician or the end user. At least one of the end user devices 110 may have a digital therapeutic application and may provide sessions (sometimes referred to herein as therapy sessions) to address at least one condition of the end user.

[0031] The application 125 may include, present, or otherwise provide to a user of the user device 110 at least one user interface 130, including one or more user interface elements 135A-N (hereinafter generally referred to as UI elements 135). The user interface 130 may be provided according to a configuration on the application 125. The UI elements 135 may correspond to visual components of the user interface 130, such as command buttons, text boxes, check boxes, radio buttons, menu items, and sliders, among others. In some embodiments, the application 125 may be a digital therapeutic application and may provide sessions (sometimes referred to herein as therapy sessions) for addressing a condition via the user interface 130. The user interface 130 may include a set of UI elements 135 for presenting digital therapeutic content.

[0032] The digital therapeutic content may be of any modality, such as text, images, audio, video, or multimedia content, among others, or a combination thereof. Content items are stored and maintained in database 120 using one or more files. For example, in the case of text, the digital therapeutic content may be stored as text files (TXT), rich text files (RTF), extensible markup language (XML), and hypertext markup language (HTML), among others. For images, the digital therapeutic content may be stored as Joint Photographic Experts' Group (JPEG), Portable Network Graphics (PNG), Graphics Interchange Format (GIF), or Scalable Vector Graphics (SVG) formats, among others. For audio, the digital therapeutic content may be stored as waveform audio files (WAV), motion picture expert group formats (e.g., MP3 and MP4), and Ogg Vorbis (OGG) formats, among others. For video, the digital therapeutic content may be stored as motion picture expert group formats (e.g., MP3 and MP4), QuickTime movie (MOV), and Windows Movie Video (WMV), among others.For multimedia content, the digital therapeutic content can be in audio video interleave (AVI), Motion Picture Experts Group formats (e.g., MP3 and MP4), QuickTime Movie (MOV), and Windows Movie Video (WMV), among others.

[0033] Digital therapeutic content can include a set of stimuli (e.g., in the form of audio, visual, or text) that guide a user to perform a specific task, such as an implicit association task (IAT) (e.g., associating stimuli with concepts), attention bias modification training (ABMT) (e.g., training a user to shift attention away from specific stimuli), an emotional face memory task (EFMT) (e.g., providing a message based on the user's anticipated state), a digital support tool (DST), adaptive goal setting (AGS) (e.g., providing a message based on the user's dynamic goals), etc.

[0034] In some embodiments, the IAT of the digital therapeutic content may be an IAT as described in U.S. Patent Application No. 18 / 111,084 (published as U.S. Patent Application Publication No. 2023 / 0268037), which is incorporated herein by reference in its entirety. The ABMT of the digital therapeutic content may be an ABMT as described in U.S. Patent Application No. 18 / 237,567 (published as U.S. Patent Application Publication No. 2024 / 0071602), which is incorporated herein by reference in its entirety. In some embodiments, the EFMT of the digital therapeutic content may be an EFMT as described in U.S. Patent No. 10,123,737, which is incorporated herein by reference in its entirety. In some embodiments, the DST of the digital therapeutic content may be a DST as described in U.S. Patent Application No. 18 / 130,813 (published as U.S. Patent Application Publication No. 2023 / 0360773), which is incorporated herein by reference in its entirety. In some embodiments, the AGS of the digital therapeutic content may be an AGS as described in U.S. Patent Application No. 18 / 208,067 (published as U.S. Patent Application Publication No. 2023 / 0410967), the entire contents of which are incorporated herein by reference.

[0035] Database 120 may store and maintain various resources and data related to session management service 105 and applications 125. Database 120 may include a database management system (DBMS) for arranging and organizing data maintained thereon, such as user profiles 165 and training data 170, among other things. Database 120 may communicate with session management service 105 and one or more user devices 110 via network 115. While performing various operations, session management service 105 and applications 125 may access database 120 to retrieve identified data therefrom. Session management service 105 and applications 125 may also write data to database 120 from the performance of such operations.

[0036] On the database 120, each user profile 165 (sometimes referred to herein as a user account, user information, or subject profile) can store and maintain information related to a user of the application 125 through the user device 110. Each user profile 165 can be associated with or correspond to a respective user of the application 125. The user profile 165 can identify various information about the user, such as a user identifier, a condition to be addressed, information about sessions conducted by the user (e.g., completed activities or lessons), message preferences, user characteristic information, and progress in addressing the condition (e.g., endpoint completion), among other information. The session information can include various parameters of previous sessions conducted by the user and may initially be null. The message preferences can include treatment preferences and user input preferences, such as preferred message types or timing. The message preferences can also include preferences determined by the session management service 105, such as the types of messages the user can respond to. The progress status is initially set to a starting value (e.g., null or "0") and can correspond to the alleviation, mitigation, or treatment of the condition. The user profile 165 may be continually updated by the application 125 and the session management service 105 .

[0037] In some embodiments, user profile 165 may identify or include information regarding a treatment regimen undertaken by a user, such as the type of treatment (e.g., therapy, medication, or psychotherapy), duration (e.g., day, week, month, or year), and frequency (e.g., day, week, quarter, year), among other things. User profile 165 may include an activity log of at least one of messages provided to the user, interactions by the user identifying the particular user's performance, and responses from user devices 110 associated with the user, among other things. User profile 165 may be stored and maintained in database 120 using one or more files (e.g., extensible markup language (XML), comma-separated values ​​(CSV), delimited text files, or structured query language (SQL) files). User profile 165 may be iteratively updated as the user performs additional sessions or responds to additional messages.

[0038] Support center 180 can accommodate remote services including support computing devices 175 that include one or more processors coupled with memory and software capable of performing the various processes and tasks described herein. Support center 180 and support computing devices 175 may be associated with an entity such as a healthcare provider (e.g., a clinician or staff member) or a digital therapeutics administrator (e.g., an administrator of session management service 105). For example, support center 180 may be a centralized hub within an organization that provides assistance, guidance, or instruction to end users using application 125. Support center 180 can include support computing devices 175 using various communication channels via network 115. Users of support computing devices 175 can communicate with users of user devices 110 via application 125.

[0039] Referring now to FIG. 2 , a block diagram of a process 200 is shown for generating a user's likelihood of interacting in a subsequent session based on event data from a previous session in the system 100 to detect an expected decrease or persistence of interaction between sessions. The process 200 may include or correspond to operations performed by the system 100 to generate event data from a session and predict the likelihood that a user will interact in a subsequent session. Under the process 200, a session handler 140 executing on the session management service 105 may send, communicate, or otherwise provide instructions for at least one session 210 to be presented via an application 125. The instructions may specify or include digital therapeutic content to be presented to the user 205 via a user interface 130 for the application 125. The instructions may include, for example, specifications regarding which UI element 135 should be used and may specify content to be displayed on the UI element 135 of the user interface 130. The instructions may be code, a data packet, or a control for presenting a session to the user 205 via an application 125 executing on the user device 110.

[0040] For a session 210, the session handler 140 can create, write, or otherwise generate instructions. In some embodiments, the generation of instructions may be based on the user profile 165 of the user 205. For example, the session handler 140 may select digital therapeutic content, including lessons, based on a condition of the user 205 identified in the user profile 165. In some embodiments, the session handler 140 may generate instructions for a session 210 that includes digital therapeutic content that includes a rhetorical message for the user 205. For example, the message may include an informational guide on how to address a particular medical condition. In some embodiments, the session handler 140 may generate instructions for a session 210 that includes digital therapeutic content to prompt the user 205 to perform a particular task. The digital therapeutic content may include a set of stimuli (e.g., audio, visual, or text) for the user 205. Tasks may include, for example, an Implicit Association Task (IAT), Attentional Bias Modification Training (ABMT), Emotional Face Memory Task (EFMT), Digital Support Tools (DST) (sometimes referred to herein as Decision Support Tools), and Adaptive Goal Setting (AGS). A session may correspond to a set number of tasks to be performed by the user 205 within a defined period of time. Upon creation, the session handler 140 can communicate instructions for the session 210 to the user device 110.

[0041] An application 125 on the user device 110 may obtain, identify, or otherwise receive instructions for the session 210. The application 125 may perform, carry out, or otherwise execute the instructions for the session 210. In accordance with the instructions, the application 125 may render, display, or otherwise present digital therapeutic content via a UI element 135 of the user interface 130. For example, the user 205 may suffer from high blood pressure and may be recovering from a hand injury. To treat the high blood pressure, the session 210 may include a message instructing the user 205 to perform one or more aerobic exercises (e.g., walking, running, or hiking), while to treat the hand injury, the user 205 may complete one or more vigorous exercises. The user 205 may interact with the content for treating high blood pressure because the content for treating high blood pressure is better tailored to the user 205 than the content for treating the hand injury. Content for treating high blood pressure may include motivational quotes, workout music, and various games to play during breaks for one or more cardiovascular exercises.

[0042] In conjunction with the presentation of digital therapeutic content for session 210, application 125 on user device 110 may record, detect, or monitor one or more interactions 215 by user 205 with application 125 on user device 110. Each interaction 215 may correspond to one or more events on application 125 triggered by user 205 interacting with UI elements 135 of user interface 130 while session 210 is being presented. For example, if session 210 is for EFMT, interaction 215 may include or identify the user's response when prompted to indicate whether a first facial expression is the same as a last facial expression. Application 125 may use event listeners or handlers on UI elements 135 of user interface 130 to monitor interactions 215. In some embodiments, application 125 may use listeners associated with input / output devices on user device 110 to monitor interactions 215. For example, the application 125 may use, among other things, a camera on the user device 110 to track the gaze of the user 205 or may use an inertial sensor on the user device 110 to detect the movement or orientation of the user device 110.

[0043] Based on the interactions 215, the application 125 can write, create, or otherwise generate event data 220 that identifies the session 210 of digital therapeutic content and the interactions 215 by the user 205. In some embodiments, the application 125 may generate event data 220 that identifies the interactions 215 across a set of prior sessions (e.g., the prior 3-5 sessions provided to the user 205). The event data 220 may include a log of the interactions 215 detected on the UI element 135 of the user interface 130 that presents the digital therapeutic content and information derived from the set of interactions 215. In some embodiments, the event data 220 may include an identifier for the digital therapeutic content provided in the session 210. For example, the identifier may be a set of alphanumeric characters to reference an individual instance of the session 210, the digital therapeutic content (e.g., a lesson, message, or task) provided in the session 210, or the number of sessions 210 provided to the user 205.

[0044] In some embodiments, the event data 220 may identify or include a time at which a session 210 is provided. This time may identify, for example, the time at which the digital therapeutic content for the session 210 was presented to the user 205, the time at which instructions for the session 210 were sent to the user device 110, the time at which the digital therapeutic content for the session 210 finished being presented to the user 205, the time at which the digital therapeutic content for the session 210 was no longer presented via the user interface 130, etc. In some embodiments, the event data 220 may identify or include a sequence identifier for the session 210 within a set of sessions. The sequence identifier may be used to distinguish which session 210 the user 205 is currently interacting with. For example, if the user 205 suffers from a speech impediment and has undergone n sessions to address the speech impediment, the sequence identifier may be “n” to identify the nth session.

[0045] In some embodiments, the event data 220 may identify or include at least one metric to indicate the degree of interactions 215 with the session 210. The metrics may include, for example, the total number of interactions 215, the rate or frequency of interactions 215, the rate of correct responses to prompts, and the average response time between the presentation (e.g., of a stimulus) and the corresponding user interaction by the user 205. The metrics may be determined by the application 125 based on the interactions 215. In some embodiments, the event data 220 may identify or include an indication of whether the user 205 contacted the support computing device 175. The indication may be a flag, a binary value, an alphanumeric value, or metadata associated with the event data 220. In some embodiments, the event data 220 may include or identify the device type of the user device 110, such as a smartphone, laptop, desktop, wearable, or smartwatch, among others. In some embodiments, the event data 220 may include a pre-expected number of attrition events (eg, as determined using the predictive model 160 detailed herein).

[0046] In some embodiments, the event data 220 may identify or include at least one metric related to a task presented in the digital therapeutic content of the session 210. For example, for an IAB task, the event data 220 may include the user's 205 response time to the presentation of a stimulus, the percentage of correct classifications of the stimulus, and the user's 205 score for the association between the stimulus and the concept, etc. For an ABMT task, the event data 220 may include the user's 205 response time to the presentation of each stimulus, the percentage of correct responses by the user 205, the percentage of incorrect responses by the user 205, the level of attention allocation (e.g., from eye-tracking data), etc. For an EFMT task, the event data 220 may include, among other things, the response time, the percentage of correct responses, and the difficulty level (e.g., the number of faces between the first and last presented expressions). For a DST task, the event data 220 may identify whether the user 205 performed a specified activity (e.g., behavioral therapy including cognitive behavioral therapy, biofeedback, and relaxation therapy). In some embodiments, the event data 220 may identify or include at least one metric related to the condition of the user 205. The metric may identify the severity level of the symptoms of the condition or the condition itself. For example, the event data 220 may include or identify a pain catastrophizing score (PCS) or number of migraine days in a defined period to measure the severity level of the user 205's migraine. The score may be collected through interaction with prompts presented via the application 125.

[0047] The session handler 140 may retrieve, identify, or otherwise receive event data 220 from the user device 110. Upon receipt, the session handler 140 may process or parse the event data 220 to extract or identify a log of interactions 215 detected on UI elements 135 of the user interface 130 that present digital therapeutic content and information derived from the set of interactions 215. In some embodiments, the session handler 140 may parse the event data 220 to identify one or more of an identifier for the digital therapeutic content provided in the session 210, the time the session 210 is provided, a sequence identifier, a metric indicative of the degree of interaction 215 with the session 210, an indication of whether the user 205 contacted a support computing device 175, a device type, a pre-expected number of declines, task-related metrics, or status-related metrics, etc. In some embodiments, the session handler 140 may supplement or add information from a user profile 165 (e.g., maintained on the database 120) to the event data 220. For example, the session handler 140 may identify a pre-expected number of attritions as determined by the predictive model 160 of the user 205 .

[0048] Additionally, prior to applying the event data 220 to the predictive model 160, the predictive model 160 may be initialized, trained, and established using training data 170 (e.g., by the model applier 145 or another device other than the session management service 105). The model applier 145 (or another device) may use the training data 170 to initialize, train, or otherwise establish the predictive model 160. The training data 170 may include a set of examples. For example, the training data 170 may be created from historical data and activity performance of a pool of users. The pool of users may have characteristics similar to the user 205 in terms of, among other things, the condition to be addressed, preferences (e.g., preferences for types of digital content or messages, such as empathetic, motivational, or rhetorical), age, or location. Each example may include sample event data 225A-N (hereinafter generally referred to as event data 225) and associated sample identifications 230A-N (hereinafter generally referred to as sample identifications 230). The sample event data 225 can identify a corresponding user's interaction with each session of digital therapeutic content. The sample event data 225 can include information in a similar format to the event data 220. The sample identification 230 can indicate whether the user interacted with a subsequent session after the session associated with the sample event data 225.

[0049] The model applier 145 (or another device) can initialize a set of weights for the predictive model 160 according to the model architecture. Upon initialization, the model applier 145 may begin training the predictive model 160 according to the learning methodology of the model architecture. The model applier 145 may identify a set of examples, including sample event data 225 and sample identification 230, in the training data 170 to update the weights and architecture of the predictive model 160. For example, if a random forest is used as the predictive model 160, the model applier 145 may randomly sample examples from the training data 170 (e.g., via bootstrapping or bagging) to create multiple decision trees for the random forest. At each node, the model applier 145 can determine whether to further split the decision tree through feature selection to decorrelate the decision trees in the random forest of the predictive model 160. The model applier 145 may perform recursive splits to isolate the target variable. The model applier 145 can then combine the decision trees to perform ensemble learning. Upon completion of ensemble learning, the model applier 145 can use the remaining set of examples in the training data 170 to evaluate the prediction accuracy and update the weights of the predictive model 160.

[0050] Using the trained predictive model 160, the model applier 145 may input, provide, or apply the event data 220 to the predictive model 160. Based on providing or applying the event data 220 to the predictive model 160, the model applier 145 may calculate, determine, or generate at least one likelihood 235 that the user 205 will interact with at least one subsequent session. The likelihood 235 may indicate a value (e.g., a probability) that the user 205 will interact with a session following the session 210. In some embodiments, the likelihood 235 may be the likelihood that the user 205 will not interact with the subsequent session. The subsequent session may correspond to a defined period of time after the current session 210. To generate the likelihood, the model applier 145 may provide the event data 220 to the predictive model 160. Once provided, the model applier 145 may process the event data 220 according to the weights and model architecture of the predictive model 160. After processing, the model applier 145 generates a likelihood 235. Different factors may affect or influence the likelihood 235 that the user 205 will interact with a subsequent session. For example, the likelihood 235 that the user 205 will interact with a subsequent session may be higher if the user 205 contacts the support computing device 175, shows improvement in a metric related to their condition, or uses a particular type of user device 110. Conversely, if the user 205 does not contact the support computing device 175 or does not show improvement in their condition, the likelihood 235 that the user 205 will interact with a subsequent session may be lower.

[0051] Referring now to FIG. 3 , a block diagram of a process 300 is depicted for generating an output based on a user's likelihood of interacting in a subsequent session in the system 100 to detect an expected decrease or persistence of interaction between sessions. The process 300 may include or correspond to operations performed by the system 100 for generating an output 310 based on the likelihood 235. Under the process 300, the interaction evaluator 150 executing on the session management service 105 may identify or determine at least one classification 305. To determine the classification 305, the interaction evaluator 150 may compare the likelihood 235 to a threshold. The threshold may define, identify, or otherwise define a value for the likelihood 235 for predicting or detecting an expected decrease in interaction on the part of the user 205 in subsequent sessions. The classification 305 may indicate whether the user is expected to decrease or persist in interaction in subsequent sessions.

[0052] If the likelihood 235 meets (e.g., is greater than or equal to) the threshold, the interaction evaluator 150 may identify, predict, determine, or otherwise detect persistence of user interaction by the user 205 in the next session. The interaction evaluator 150 may determine a classification 305 to indicate an expected (or predicted) persistence. On the other hand, if the likelihood 235 does not meet (e.g., is less than) the threshold, the interaction evaluator 150 may identify, predict, determine, or otherwise detect an expected decrease in user interaction by the user 205 in the next session. The interaction evaluator 150 may determine a classification 305 to indicate an expected (or predicted) decrease. The interaction evaluator 150 may store and maintain the classification 305 in the database 120. For example, the interaction evaluator 150 may use the classification 305 to update the user profile 165 of the user 205. Additionally, the interaction evaluator 150 may compare the likelihood 235 to a set of ranges, each of which corresponds to a level of severity for the user 205 and can be used to select an appropriate countermeasure to address the expected decline in user interaction.

[0053] The feedback generator 155 executing on the session management service 105 can select or identify at least one of the output candidates 310A-N (hereinafter generally referred to as output candidates 310) based on the possibilities 235 (or classifications 305). The set of output candidates 310 can include, for example, one or more of a first output candidate that indicates on the support computing device 175 to initiate communication with the user 205 via at least one of a phone call, email, or in-app chat, a second output candidate that indicates on the support computing device 175 to monitor subsequent interactions by the user 205, a third output candidate that notifies the user 205 (e.g., using a predefined notification) to increase engagement with the digital therapeutic, or a fourth candidate output that indicates on the support computing device 175 to refrain from initiating communication or monitoring the user 205.

[0054] Based on a comparison of the likelihood 235 to the set of ranges, the feedback generator 155 can select at least one output 310'. At least some of the output candidates can be associated with a set of ranges and a level of severity. For example, a first output candidate can be associated with a high level of severity and can be a high-touch intervention. A second output candidate can be associated with a low level of severity and can be a low-level intervention. A third output candidate can be associated with a medium level of severity and can be a base-case intervention. If the likelihood 235 does not meet a threshold indicating an expected decline, the feedback generator 155 can select an output 310' from one of the first through third output candidates. If the likelihood 235 meets a threshold indicating an expected persistence, the feedback generator 155 can select a fourth output candidate as the output 310' and instruct the support computing device 175 to refrain from initiating communication with or monitoring the user 205.

[0055] In some embodiments, the feedback generator 155 can write, generate, or otherwise generate the output 310′ using at least one generative model 315. The generative model 315 can be, among other things, a generative transformer model such as a large language model (LLM) (e.g., ChatGPT or bidirectional encoder representations from transformers (BERT)). The generative model 315 can be hosted on the session management service 105 or on a separate server. To generate the output 310′, the feedback generator 155 can retrieve, select, or otherwise identify a user profile 165 associated with each user 205 upon detecting an expected decline. The user profile 165 can identify or include profile information such as name, status, preferences, and a number of pre-expected declines, among other things. The feedback generator 155 can provide, feed, or otherwise apply the profile information to the generative model 315. The generative model 315 may use the profile information to generate custom notifications that instruct the user 205 to interact with subsequent sessions provided to the user 205. The feedback generator 155 may use the notifications generated by the generative model 315 as output 310′.

[0056] Upon generation, the feedback generator 155 may communicate, transmit, or otherwise provide the output 310′ to the user 205. In some embodiments, the feedback generator 155 may provide the output 310′ to the application 125 of the user device 110. The output 310′ provided to the application 125 may include, for example, a predefined or custom notification generated using the generative model 315 to direct the user 205 to interact with a subsequent session. In some embodiments, the feedback generator 155 may communicate the output 310′ to a support computing device 175 of the support center 180. For example, the feedback generator 155 may communicate the output 310′ to the support computing device 175 so that the support computing device 175 can initiate communication with the user upon receiving the notification or begin monitoring the user's 205 interaction with a subsequent session. In some embodiments, the feedback generator 155 may communicate an output 310′ to the support computing device 175 to refrain from initiating communication or monitoring the user 205 if the classification 305 indicates persistence of the interaction.

[0057] Referring now to FIG. 4 , a block diagram of a process 400 for providing sessions in the system 100 is depicted to detect expected inter-session interaction decline or persistence. The process 400 may include or correspond to operations performed by the system 100 to provide subsequent sessions. Under the process 400, the session handler 140 may send, communicate, or otherwise provide instructions for at least one session 410 to be presented via the application 125. The session 410 (also referred to herein as the next session or subsequent session) may correspond to a defined time following the previous session 210. The instructions may specify or include digital therapeutic content to be presented to the user 205 via the user interface 130 for the application 125. The instructions may include, for example, specifications regarding which UI element 135 should be used and may specify content to be displayed on the UI element 135 of the user interface 130. The instructions may be code, a data packet, or control for presenting the session 410 to the user 205 via the application 125 running on the user device 110.

[0058] In some embodiments, the feedback generator 155 may change, alter, or otherwise modify the digital therapeutic content presented in a subsequent session 410. Modification of the digital therapeutic content may be implemented when an expected decrease in interaction is detected. Modification may be based on the user profile 165, the likelihood 235, or the classification 305, among others. In some embodiments, the feedback generator 155 may identify or select a difficulty level for the digital therapeutic content. For example, the feedback generator 155 may select an easier difficulty level to increase engagement on the part of the user 205. In some embodiments, the feedback generator 155 may identify or select a task for the digital therapeutic content. For example, the feedback generator 155 may select a task that is different from a task with which the user 205 was previously engaged. In some embodiments, the feedback generator 155 may select or identify a tone (e.g., empathy, clarity, positivity, trust, respect, encouragement, or personalization) for the message of the digital therapeutic content. In some embodiments, the feedback generator 155 may change or alter the sequence of the digital therapy content (e.g., the order of presentation of UI elements 135 on the user interface 130). The feedback generator 155 may provide the modified digital therapy as part of the session 410 or output 310'.

[0059] An application 125 on the user device 110 can obtain, identify, or otherwise receive instructions for the session 410. The application 125 may perform, carry out, or otherwise execute the instructions for the session 410. In accordance with the instructions, the application 125 may render, display, or otherwise present digital therapeutic content via a UI element 135 of the user interface 130. For example, the user 205 may suffer from high blood pressure and be recovering from a hand injury. To treat the high blood pressure, the session 410 may include a message instructing the user 205 to perform one or more aerobic exercises (e.g., walking, running, or hiking), while to treat the hand injury, the user 205 may complete one or more vigorous exercises. The user 205 may interact with the content for treating the high blood pressure because the content for treating the high blood pressure is better tailored to the user 205 than the content for treating the hand injury. Content for high blood pressure may include motivational quotes, workout music, and various games to play during breaks for one or more cardiovascular exercises.

[0060] Along with the presentation of digital therapeutic content for session 410, application 125 on user device 110 can record, detect, or monitor one or more interactions 415 by user 205 with application 125 on user device 110. Each interaction 415 can correspond to one or more events on application 125 triggered by user 205 interacting with UI elements 135 of user interface 130 while session 410 is being presented. For example, if session 410 is a go / no-go task, interaction 415 can include or identify the user's response when prompted to indicate whether or not to respond to the presentation of a stimulus. Application 125 can use event listeners or handlers on UI elements 135 of user interface 130 to monitor interactions 415.

[0061] Based on the interactions 415, the application 125 can write, create, or otherwise generate event data 420 identifying the interactions 415 by the user 205 with the session 410 of digital therapeutic content. In some embodiments, the application 125 may generate event data 420 identifying the interactions 215 across a set of prior sessions (e.g., the prior 3-5 sessions including the session 210 provided to the user 205). The event data 420 can identify or include, for example, one or more of: an identifier for the digital therapeutic content provided in the session 410; a time at which the session 410 is provided; a sequence identifier; a metric indicating the degree of interaction 415 with the session 410; an indication of whether the user 205 contacted a support computing device 175; a type of device; a number of prior expected attrition events; a metric related to a task presented in the digital therapeutic content; or a metric related to the user's state. In some embodiments, the session handler 140 can supplement or add information from a user profile 165 (e.g., maintained on the database 120) to the event data 420.

[0062] The session handler 140 may retrieve, identify, or otherwise receive event data 420 from the user device 110. Upon receipt, the session handler 140 may process or parse the event data 420 to extract or identify a log of interactions 415 detected on UI elements 135 of the user interface 130 presenting the digital therapeutic content and information derived from the set of interactions 415. In some embodiments, the session handler 140 may parse the event data 420 to identify, for example, one or more of an identifier for the digital therapeutic content provided in the session 410, the time at which the session 410 is provided, a sequence identifier, a metric indicative of the degree of interaction 415 with the session 410, an indication of whether the user 205 contacted a support computing device 175, the type of device, a pre-expected number of attrition events, a metric related to a task presented in the digital therapeutic content, or a metric related to the user's state, etc. In some embodiments, the session handler 140 may supplement or add information from a user profile 165 (e.g., maintained on the database 120) to the event data 420. For example, the session handler 140 may identify a pre-expected number of attritions as determined by the predictive model 160 for the user 205 .

[0063] Using the trained predictive model 160, the model applier 145 can input, provide, or apply the event data 420 to the predictive model 160. Based on providing or applying the event data 420 to the predictive model 160, the model applier 145 may calculate, determine, or generate at least one likelihood 425 that the user 205 will interact with a subsequent session. The likelihood 425 may indicate a value (e.g., a probability) that the user 205 will interact with a session following the session 410. The subsequent session may correspond to a defined period of time after the current session 410. To generate the likelihood 425, the model applier 145 can provide the event data 420 to the predictive model 160. Once provided, the model applier 145 can process the event data 420 according to the weights and model architecture of the predictive model 160. From the processing, the model applier 145 generates the likelihood 125.

[0064] The interaction evaluator 150 can identify or determine at least one classification 430. To determine this, the interaction evaluator 150 may compare the likelihood 425 to a threshold. The threshold can define, identify, or otherwise define a value for the likelihood 425 of detecting an expected decrease in interaction on the part of the user 205 in a subsequent session. The classification 430 can indicate whether the user is expected to decrease or persist in interaction in a subsequent session. If the likelihood 425 meets (e.g., is greater than or equal to) the threshold, the interaction evaluator 150 can identify, determine, or otherwise detect persistence of user interaction by the user 205 in the next session. The interaction evaluator 150 can determine the classification 430 to indicate the expected persistence.

[0065] On the other hand, if the likelihood 425 does not meet (e.g., is less than) the threshold, the interaction evaluator 150 may identify, determine, or otherwise detect an expected decrease in user interaction by the user 205 in the next session. The interaction evaluator 150 may determine a classification 430 to indicate the expected decrease. The interaction evaluator 150 may store and maintain the classification 430 in the database 120. For example, the interaction evaluator 150 may update the user profile 165 of the user 205 with the classification 430. Further, the interaction evaluator 150 may compare the likelihood 425 to a set of ranges. Each range corresponds to a severity level for the user 205 and can be used to select an appropriate measure to address the expected decrease in user interaction.

[0066] In some embodiments, the interaction evaluator 150 may identify or determine whether a change (e.g., improvement or deterioration) in responsiveness (or interaction) will occur for the user 205 based on a comparison of the probabilities 235 and 425 (or classifications 305 and 430). To make the comparison, the interaction evaluator 150 may calculate or determine the difference between the probabilities 235 and 425. If the difference is negative by at least a threshold margin, the interaction evaluator 150 may determine a decrease (or deterioration) in responsiveness between sessions 210 and 410. If the difference is positive by at least a threshold margin, the interaction evaluator 150 may determine an increase (or improvement) in responsiveness between sessions 210 and 410. If the difference is within the threshold margin, the interaction evaluator 150 may determine no change in responsiveness.

[0067] In some embodiments, the feedback generator 155 can select or identify at least one of a set of output candidates 310 based on a change in responsiveness. If there is a decrease (or deterioration) in responsiveness, the feedback generator 155 may select a first output candidate for display on the support computing device 175 to initiate communication with the user 205. If there is no change in responsiveness, the feedback generator 155 may select a second output candidate for the support computing device 175 to monitor subsequent interactions by the user 205. If there is an increase (or improvement) in responsiveness, the feedback generator 155 can select a third output candidate for notifying the user 205 (e.g., using a predefined or custom notification) to increase engagement with the digital therapeutic. The feedback generator 155 may also provide output in a manner similar to process 300 described above.

[0068] By using event data from previous sessions and machine learning (ML) models to predict the likelihood that a user will interact with subsequent sessions provided to the user, the session management service 105 can decide whether to continue the digital therapy session or provide more targeted measures (in the form of output 310′) to improve the user's adherence. This allows the session management service 105 and user device 110 to better and more efficiently allocate computing resources and network bandwidth based on this decision. A healthcare provider or other entity, such as a digital therapy administrator, may be able to identify which users should receive manual intervention (e.g., by initiating communication). From the user's 205 perspective, improvements in therapy adherence across multiple sessions allow the user 205 to receive the full benefits and effectiveness of the digital therapy in addressing the user's 205 condition. If a decision is made to provide the user 205 with more targeted measures, the human intervention (e.g., initiating communication) may not only improve the effectiveness of the digital therapy delivered to the user 205, but may also improve any medications the user may be taking in conjunction with the digital therapy regimen.

[0069] Furthermore, reliance on prior event data 220 and 420 may overcome issues of data scarcity as well as types of contextual factors in accurately and precisely predicting user interactions between sessions. Accurate predictions may enable the session management service 105 and user device 110 to more efficiently allocate computing resources (e.g., computing power and network bandwidth) when determining whether to continue digital therapeutic content or provide countermeasures. As a result of providing targeted countermeasures when a user is predicted to drop out, the application 125 may improve the user's 205 adherence to the digital therapeutic provided throughout the session, thereby improving the quality of human-computer interaction (HCI) between the user 205 and the application 125.

[0070] Referring now to FIG. 5, a flow diagram of a method 500 for providing an output to prompt interaction with a digital therapeutic application is depicted. Method 500 may be implemented or performed using any of the components described herein, such as session management service 105 and user device 110, or any combination thereof. Under method 500, a computing system (e.g., session management service 105 or user device 110) may identify event data for a first session (505). The computing system may apply the event data to a machine learning (ML) model (e.g., predictive model 160) (510). The computing system may generate a likelihood (e.g., likelihood 235) that the user will interact with the second session (515). The computing system may determine (520) whether the likelihood exceeds a threshold. If the likelihood does not meet (e.g., is less than) the threshold, the computing system may detect an expected decrease in interaction (525). The computing system may select a countermeasure (e.g., output 310′) to address the expected decrease in interaction (530). If the likelihood meets (e.g., is greater than or equal to) a threshold, the computing system can detect persistence of the user interaction 535. The computing system can provide an output based on the determination of whether there is an expected decrease or persistence of the interaction 540.

[0071] Example 1: Using event prediction models to improve adherence to digital treatments by ABMT The session handler 140 can provide a session 210 to the user 205 that includes ABMT digital therapeutic content to address chronic pain associated with an underlying condition. The application 125 on the user device 110 can collect data related to interactions 215 in response to the presentation of the session 210. The session handler 140 can receive event data 220 related to the ABMT, such as the response time by the user 205 to the presentation of each stimulus, the percentage of correct responses by the user 205, the percentage of incorrect responses by the user 205, and the level of attention allocation (e.g., from eye-tracking data). The model applier 145 can input the event data 220 into a predictive model 160 to generate a likelihood 235 that the user 205 will interact in a subsequent session. Based on the likelihood 235, the interaction evaluator 150 can determine a classification 305 to predict whether the user 205 will interact with a subsequent session. If the classification 305 predicts that the user 205 is likely not to interact, the feedback generator 155 may select at least one output 310′ to provide to the user 205 to provide an intervention directly to the user and / or a provider, enabling the provider to intervene.

[0072] By providing sessions 210 and 410 and output 310′ based on the probabilities 235 generated by the predictive model 160, a higher rate of return for subsequent sessions can be achieved, and metrics related to the user's 205 status also show improvement. For example, by considering response time, percentage of correct responses, or level of attention allocation, among other data, the predictive model 160 is expected to provide more targeted and accurate probabilities 235. Furthermore, providing output 310′ based on the probabilities 235 indicates that the user 205 will have a higher rate of interaction with subsequent sessions. With exposure to additional sessions 210 and 410 (for ABMT), metrics of the user 205 related to chronic pain (e.g., Pain Severity Scale (PCS) value) are expected to show improvement.

[0073] Example 2: Using event prediction models to improve adherence to digital treatments with EFMT The session handler 140 may provide a session 210 including digital therapeutic content for EFMT to address affective disorders (e.g., major depressive disorder) for presentation to the user 205. The application 125 on the user device 110 may collect data related to interactions 215 in response to the presentation of the session 210. The session handler 140 may receive event data 220 related to the EFMT, such as response time, percentage of correct responses, and difficulty level (e.g., the number of facial expressions presented between the beginning and end). The model applier 145 may input the event data 220 into a predictive model 160 to generate a likelihood 235 that the user 205 will interact with a subsequent session. Based on the likelihood 235, the interaction evaluator 150 can determine a classification 305 to predict whether the user 205 will interact with a subsequent session. If the classification 305 predicts that the user 205 is likely not to interact, the feedback generator 155 may select at least one output 310′ to provide to the user 205 to provide an intervention directly to the user and / or a provider, enabling the provider to intervene.

[0074] By providing sessions 210 and 410 and output 310′ based on the probabilities 235 generated by the predictive model 160, a higher return rate for subsequent sessions can be achieved, and metrics related to the user's 205 status also show improvement. For example, by considering data such as response time, percentage of correct responses, and difficulty level, the predictive model 160 is expected to provide more targeted and accurate probabilities 235. Furthermore, providing output 310′ based on the probabilities 235 indicates a higher rate of interaction with subsequent sessions by the user. Exposure to additional sessions 210 and 410 (for EFMT) is expected to result in improvement in the user's 205 metrics related to affective disorders (e.g., Hamilton Depression Rating Scale-17 values ​​(Ham-D)).

[0075] Example 3: Using event prediction models to improve adherence to digital treatments with DST The session handler 140 may provide the user 205 with a session 210 including digital therapeutic content for DST to address migraine headaches. The application 125 on the user device 110 may collect data related to the interaction 215 in response to presentation of the session 210. The session handler 140 may receive event data 220 related to the DST, such as whether an activity prompted by the digital therapeutic content was performed (e.g., behavioral therapy including cognitive behavioral therapy, biofeedback, and relaxation therapy). The model applier 145 may input the event data 220 into a predictive model 160 to generate a likelihood 235 that the user 205 will interact in a subsequent session. Based on the likelihood 235, the interaction evaluator 150 may determine a classification 305 to predict whether the user 205 will interact with the subsequent session. If the classification 305 predicts that the user 205 is likely not to interact, the feedback generator 155 may select at least one output 310′ to provide to the user 205 to provide an intervention directly to the user and / or provider, enabling the provider to intervene.

[0076] By providing sessions 210 and 410 and output 310′ based on the probabilities 235 generated by the predictive model 160, a higher return rate for subsequent sessions can be achieved, and metrics related to the user's 205 status also show improvement. For example, by considering whether a particular activity was performed, among other data, the predictive model 160 is expected to provide a more targeted and accurate probabilities 235. Furthermore, providing output 310′ based on the probabilities 235 indicates that the user 205 will have a higher rate of interaction with subsequent sessions. With exposure to additional sessions 210 and 410 (for DST), metrics of the user 205 related to migraine or disability (e.g., monthly migraine days (MDD) or pain severity scale (PCS) value) are expected to show improvement.

[0077] (B. Network and Computing Environment) Various operations described herein may be implemented on a computer system. FIG. 6 illustrates a simplified block diagram of a representative server system 600, a client computer system 614, and a network 626 that may be used to implement certain embodiments of the present disclosure. In various embodiments, the server system 600 or a similar system may implement the services or servers described herein, or portions thereof. The client computer system 614 or a similar system may implement the clients described herein. The system 100 described herein may be similar to the server system 600. The server system 600 may have a modular design incorporating multiple modules 602 (e.g., blades in a blade server embodiment); although two modules 602 are shown, any number may be provided. Each module 602 may include a processing unit 604 and local storage 606.

[0078] Processing unit 604 may include a single processor or multiple processors, each of which may have one or more cores. In some embodiments, processing unit 604 may include a general-purpose primary processor and one or more special-purpose coprocessors, such as a graphics processor, digital signal processor, etc. In some embodiments, some or all of processing unit 604 may be implemented using customized circuitry, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuitry itself. In other embodiments, processing unit(s) 604 may execute instructions stored on local storage 606. Any type of processor, in any combination, may be included in processing unit 604.

[0079] The local storage 606 may include volatile storage media (e.g., DRAM, SRAM, SDRAM, etc.) and / or non-volatile storage media (e.g., magnetic or optical disks, flash memory, etc.). The storage media incorporated in the local storage 606 may be fixed, removable, or upgradeable, as desired. The local storage 606 may be physically or logically divided into various sub-units, such as system memory, read-only memory (ROM), and permanent storage devices. The system memory may be a volatile read-write memory, such as a read-write memory device or dynamic random access memory. The system memory may store some or all of the instructions and data needed by the processing unit 604 at runtime. The ROM may store static data and instructions needed by the processing unit 604. The permanent storage device may be a non-volatile read-write memory device that can store instructions and data even when the module 602 is powered down. As used herein, the term "storage medium" includes any medium capable of storing data indefinitely (subject to overwriting, electrical disturbance, power loss, etc.), and does not include carrier waves and ephemeral electronic signals propagated wirelessly or via wired connections.

[0080] In some embodiments, local storage 606 may store one or more software programs performed by processing unit 604, such as an operating system and / or programs that implement various server functions, such as functions of system 100 or any other system described herein, or any other server associated with system 100 or any other system described herein.

[0081] "Software" generally refers to sequences of instructions that, when executed by processing unit 604, cause server system 600 (or portions thereof) to perform various operations, and thus define one or more specific machine embodiments that perform and execute the operations of the software program. The instructions may be stored as firmware resident in read-only memory and / or as program code stored on a non-volatile storage medium that can be read into volatile working memory for execution by processing unit 604. Software may be implemented as a single program or a collection of individual programs or program modules that interact as desired. From local storage 606 (or non-local storage, as described below), processing unit 604 may retrieve program instructions to perform and data to process in order to perform the various operations described above.

[0082] In some server systems 600, multiple modules 602 may be interconnected via a bus or other interconnect 608 to form a local area network that supports communication between the modules 602 and other components of the server system 600. The interconnect 608 may be implemented using a variety of technologies, such as a server rack, a hub, a router, etc.

[0083] A wide area network (WAN) interface 610 may provide data communication capabilities between a local area network (e.g., through interconnect 608) and a network 626, such as the Internet. Other technologies may be used to communicatively couple the server system to the network 626, including wired technologies (e.g., Ethernet, IEEE 802.3 standard) and / or wireless technologies (e.g., Wi-Fi, IEEE 802.11 standard).

[0084] In some embodiments, local storage 606 is intended to provide working memory for processing unit 604, providing fast access to programs and / or data being processed while reducing traffic on interconnect 608. Storage for larger amounts of data may be provided over a local area network by one or more mass storage subsystems 612 connected to interconnect 608. Mass storage subsystems 612 may be based on magnetic, optical, semiconductor, or other data storage media. Direct-attached storage, storage area networks, network-attached storage, etc. may be used. Any data store or other collection of data described herein as produced, consumed, or maintained by a service or server may be stored in mass storage subsystem 612. In some embodiments, additional data storage resources may be accessible via WAN interface 610 (potentially increasing latency).

[0085] The server system 600 may operate in response to requests received via the WAN interface 610. For example, one of the modules 602 may implement a monitoring function and assign individual tasks to other modules 602 in response to received requests. Work allocation techniques may be used. Once a request is processed, results may be returned to the requester via the WAN interface 610. Such operations may generally be automated. Furthermore, in some embodiments, the WAN interface 610 may connect multiple server systems 600 together to provide a scalable system capable of managing large volumes of activity. Other techniques for managing server systems and server farms (collections of operating server systems) may be used, including dynamic resource allocation and reallocation.

[0086] Server system 600 can interact with various user-owned or user-operated devices over a wide area network such as the Internet. One example of a user-operated device is shown in Figure 6 as client computing system 614. Client computing system 614 can be implemented as a consumer device, such as, for example, a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, eyeglasses), desktop computer, laptop computer, etc.

[0087] For example, client computing system 614 can communicate via WAN interface 610. Client computing system 614 can include computer components such as a processing unit 616, a storage device 618, a network interface 620, a user input device 622, and a user output device 624. Client computing system 614 can be a computing device implemented in a variety of form factors, such as a desktop computer, a laptop computer, a tablet computer, a smartphone, other mobile computing device, a wearable computing device, etc.

[0088] The processing unit 616 and storage device 618 may be similar to the processing unit 604 and local storage 606 described above. Appropriate devices may be selected based on the requirements placed on the client computing system 614. For example, the client computing system 614 may be implemented as a "thin" client having limited processing capabilities or as a high-powered computing device. The client computing system 614 may provide program code executable by the processing unit 616 to enable various interactions with the server system 600.

[0089] The network interface 620 may provide a connection to a network 626, such as a wide area network (e.g., the Internet) to which the WAN interface 610 of the server system 600 is also connected. In various embodiments, the network interface 620 may include a wired interface (e.g., Ethernet) and / or a wireless interface implementing various RF data communication standards, such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, LTE, etc.).

[0090] User input device 622 may include any device (or devices) that allows a user to provide signals to client computing system 614, which may interpret the signals as indicating particular user requests or information. In various embodiments, user input device 622 may include any or all of a keyboard, touchpad, touchscreen, mouse or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, etc.

[0091] The user output device 624 can include any device that enables the client computing system 614 to provide information to a user. For example, the user output device 624 can include a display-to-display image generated by or delivered to the client computing system 614. The display can incorporate various image generation technologies, such as liquid crystal display (LCD), light-emitting diode (LED) displays including organic light-emitting diodes (OLED), projection systems, cathode ray tubes (CRT), etc., along with supporting electronics (e.g., digital-to-analog or analog-to-digital converters, signal processors, etc.). Some embodiments can include devices such as a touchscreen that function as both an input and output device. In some embodiments, other user output devices 624 can be provided in addition to or instead of a display. Examples include indicator lights, speakers, tactile “display” devices, printers, etc.

[0092] Some embodiments include electronic components, such as a microprocessor, storage, and memory, that store computer program instructions on a computer-readable storage medium. Many of the features described herein can be implemented as processes specified as a set of program instructions encoded on a computer-readable storage medium. These program instructions, when executed by one or more processing units, cause the processing units to perform various operations indicated in the program instructions. Examples of program instructions or computer code include machine code, such as produced by a compiler, and files containing high-level code executed by a computer, electronic component, or microprocessor using an interpreter. Through appropriate programming, processing units 604 and 616 can provide various functionalities for server system 600 and client computing system 614, including any functionality or other functionality described herein as being performed by a server or client.

[0093] It will be understood that the server system 600 and the client computing system 614 are exemplary and that variations and modifications are possible. Computer systems used in connection with embodiments of the present disclosure may have other capabilities not specifically described herein. Furthermore, while the server system 600 and the client computing system 614 are described with reference to particular blocks, it should be understood that these blocks are defined for convenience of explanation and are not intended to imply a particular physical arrangement of component parts. For example, different blocks may, but need not, be located in the same facility, in the same server rack, or on the same motherboard. Furthermore, the blocks need not correspond to physically distinct components. The blocks may be configured to perform various operations, for example, by programming a processor or providing appropriate control circuitry, and various blocks may or may not be reconfigurable depending on how the initial configuration is obtained. Embodiments of the present disclosure may be realized in a variety of apparatuses, including electronic devices implemented using any combination of circuitry and software.

[0094] While the present disclosure has been described with reference to specific embodiments, those skilled in the art will recognize that numerous modifications are possible. Embodiments of the present disclosure can be implemented using various computer systems and communication technologies, including, but not limited to, the specific examples described herein. Embodiments of the present disclosure can be implemented using any combination of dedicated components and / or programmable processors and / or other programmable devices. The various processes described herein can be implemented on the same processor or different processors, in any combination. Where components are described as being configured to perform particular operations, such configuration can be achieved, for example, by designing electronic circuitry to perform the operations, by programming programmable electronic circuitry (such as a microprocessor) to perform the operations, or any combination thereof. Furthermore, while the above-described embodiments may refer to particular hardware and software components, those skilled in the art will recognize that different combinations of hardware and / or software components can also be used, and that particular operations described as being implemented in hardware can also be implemented in software, or vice versa.

[0095] A computer program incorporating various features of the present disclosure may be encoded and stored on a variety of computer-readable storage media, suitable media including magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, and other non-transitory media. The computer-readable medium on which the program code is encoded may be packaged with a compatible electronic device, or the program code may be provided separately from the electronic device (e.g., via internet download or as a separately packaged computer-readable storage medium).

[0096] Therefore, although the present disclosure has been described in terms of specific embodiments, it will be understood that the present disclosure is intended to cover all modifications and equivalents that fall within the scope of the following claims.

Claims

1. 1. A method for detecting an expected decline in interaction between sessions, comprising: identifying, by the one or more processors, event data identifying interactions by the user in a first session with digital therapeutic content to address the condition; applying, by the one or more processors, the event data to a machine learning (ML) model, the ML model being trained using a plurality of examples, each of the plurality of examples including (i) respective event data identifying interactions with a respective session of digital therapeutic content by a corresponding user, and (ii) a respective identification of whether the corresponding user interacted with a subsequent session after the respective session; generating, by the one or more processors, a likelihood that the user will interact with a second session of digital therapeutic content to address the condition after the first session based on applying the event data to the ML model; detecting, by the one or more processors, an expected decrease in the user's interaction with the second session in response to the likelihood that the user will not meet a threshold; and providing, by the one or more processors, an output to the user based on detecting the expected decrease in interaction; A method comprising:

2. 10. The method of claim 1 further comprising: generating, by the one or more processors, a likelihood that the second user will interact with a fourth session after the third session based on applying event data identifying an interaction by a second user with a third session to the ML model; detecting, by the one or more processors, an expected persistence of interaction with the fourth session by the second user in response to the likelihood that the second user will satisfy the threshold; and providing, by the one or more processors, an output for the second user based on detecting the expected persistence of the interaction; A method comprising:

3. 10. The method of claim 1, wherein providing the output further comprises communicating the output including a notification to a computing device at a support center, the computing device configured to initiate communication with the user after receiving the notification.

4. 2. The method of claim 1, further comprising: selecting, by the one or more processors, the output from a plurality of output candidates based on the likelihood that the user will interact with the second session, the plurality of output candidates including at least one of (i) a first output candidate to be displayed on a computing device to initiate communication via at least one of a phone call, an email, or an in-app chat, or (ii) a second output candidate to be displayed on the computing device to monitor subsequent interaction by the user.

5. 10. The method of claim 1, wherein providing the output further comprises communicating the output including a predefined notification to instruct the user to interact with the second session provided to the user.

6. 10. The method of claim 1 further comprising: identifying, by the one or more processors, profile information associated with the user in response to detecting an expected decrease in the interaction by the user; and applying, by the one or more processors, the profile information to a generative transformation model to generate a notification that instructs the user to interact with the second session, the notification being provided to the user; Equipped with The method, wherein providing the output further comprises providing the notification instructing the user to interact with the second session provided to the user.

7. 10. The method of claim 1, further comprising modifying, by the one or more processors, the digital therapeutic content presented in the second session based on detecting the expected decrease in interaction; and The method, wherein providing the output further comprises providing the second session of the modified digital therapeutic content to the user.

8. 10. The method of claim 1 further comprising: identifying, by the one or more processors, second event data identifying the user's interaction with the second session for digital therapeutic content to address the condition; applying, by the one or more processors, the second event data to a machine learning (ML) model to generate a likelihood that the user will interact with a third session of digital therapeutic content to address the condition after the second session; and determining, by the one or more processors, whether a change in responsiveness will occur for the user based on a comparison of the likelihood that the user will interact with the second session and the likelihood that the user will interact with the third session; A method for providing the above.

9. 10. The method of claim 1, wherein the event data further includes at least one of: (i) an identifier of the digital therapeutic content provided in the first session; (ii) a time the first session was provided; (iii) a sequence identifier of the first session within a plurality of sessions; (iv) a metric indicating a degree of interaction in the first session; (v) an indication of whether the user contacted a computing device for support; (vi) a type of user device associated with the user; (vii) a number of pre-estimated attrition events; (ix) a metric associated with a task performed in the first session; or (x) a metric associated with the condition.

10. 10. The method of claim 1, wherein the user is taking medication to address the condition at least partially concurrently with at least one of the first session or the second session.

11. 1. A system for detecting an expected decline in interaction between sessions, comprising one or more processors coupled with a memory, the system comprising: the one or more processors identifying event data identifying a first session and an interaction by the user with digital therapeutic content to address the condition; applying the event data to a machine learning (ML) model, the ML model being trained using a plurality of examples, each of the plurality of examples including (i) respective event data identifying interactions with a respective session of digital therapeutic content by a corresponding user, and (ii) a respective identification of whether the corresponding user interacted with a subsequent session after the respective session; generating, based on applying the event data to the ML model, a likelihood that the user will interact with a second session of digital therapeutic content to address the condition following the first session; Detecting an expected decrease in the user's interaction with the second session in response to the likelihood that the user will not meet a threshold; and providing an output to the user based on detecting the expected decrease in interaction; A system that is configured to:

12. 12. The system of claim 11, wherein the one or more processors: generating a likelihood that the second user will interact with a fourth session subsequent to the third session for the second user based on applying event data identifying an interaction of the second user with a third session to the ML model; Detecting an expected persistence of interaction with the fourth session by the second user in response to the likelihood that the second user satisfies the threshold; and providing an output for the second user based on detecting the expected persistence of the interaction; A system that is configured to:

13. 12. The system of claim 11, wherein the one or more processors are configured to communicate the output including a notification to a computing device at a support center, the computing device configured to initiate communication with the user after receiving the notification.

14. 12. The system of claim 11, wherein the one or more processors are configured to select the output from the plurality of output candidates based on the likelihood that the user will interact with the second session, the plurality of output candidates including at least one of (i) a first output candidate to display on a computing device to initiate communication via at least one of a phone call, an email, or an in-app chat, or (ii) a second output candidate to display on the computing device to monitor subsequent interaction by the user.

15. 12. The system of claim 11, wherein the one or more processors are configured to communicate the output including a predefined notification to direct the user to interact with the second session provided to the user.

16. 12. The system of claim 11, wherein the one or more processors: In response to detecting an expected decrease in the interaction by the user, identifying profile information associated with the user; applying the profile information to a generative transformation model to generate a notification instructing the user to interact with the second session provided to the user; providing the notification instructing the user to interact with the second session provided to the user; A system that is configured to:

17. 12. The system of claim 11, wherein the one or more processors: modifying the digital therapeutic content presented in the second session based on detecting the expected decrease in interaction; and The step of providing an output, wherein the one or more processors are further configured to provide the second session of the modified digital therapeutic content to the user.

18. 12. The system of claim 11, wherein the one or more processors: identifying second event data identifying an interaction by the user with the second session for digital therapeutic content to address the condition; applying the second event data to a machine learning (ML) model that generates a likelihood that the user will interact with a third session for digital therapeutic content to address the condition subsequent to the second session; and determining whether a change in responsiveness will occur for the user based on a comparison of the likelihood that the user will interact with the second session and the likelihood that the user will interact with the third session; A system that is configured to:

19. 12. The system of claim 11, wherein the event data further includes at least one of: (i) an identifier of the digital therapeutic content provided in the first session; (ii) a time the first session was provided; (iii) a sequence identifier of the first session within a plurality of sessions; (iv) a metric indicating a degree of interaction in the first session; (v) an indication of whether the user contacted a computing device for support; (vi) a type of user device associated with the user; (vii) a number of pre-estimated attrition events; (ix) a metric associated with a task performed in the first session; or (x) a metric associated with the condition.

20. 12. The system of claim 11, wherein the user is taking a medication to address the condition at least partially concurrently with at least one of the first session or the second session.

Images