Adaptively selecting messages for transmission in networked environment to increase session compliance

Abstract

Systems and methods for controlling the transmission of messages in a networked environment are provided herein. The server may send a set of instructions to a user device associated with the user, each instruction presenting at least one of a set of variants of the message. The server may receive a set of responses corresponding to the instruction, each response of the user being made to a respective presentation of at least one variant of the message. The server may use the response to determine a respective performance metric for each variant of the message, and may update the selection model in response to the response. The server may select the variants based on the performance metrics of the variants. The server may send a second instruction to a user device associated with the user to present the selected variant of the message.

Description

Background Art

[0001] Poor adherence to medical treatment is a serious problem in the healthcare industry. Patients may find it difficult to adhere to treatment, resulting in ineffective clinical outcomes and higher costs. Despite industry efforts, medication adherence rates have not changed significantly over time and remain low. There are no effective and affordable methods to help patients adhere to prescribed treatments. A method is needed to improve adherence among patients and ultimately impact clinical outcomes. Summary of the invention

[0002] In the messaging service of the present invention, the service can select which variant of a message to send to a user among multiple variants. These variants can correspond to different types of content, themes, tones, or layouts for a given message, or based on factors such as the user's personality, literacy, health knowledge, learning style, personal stimulation or motivation, messaging style, or preference for including the user's name. The purpose of the selection can be to provide a variant that maximizes user engagement as measured by interaction rate. For messaging platforms in the context of digital therapy, additional purposes may include compliance with sessions that address user conditions (e.g., behavioral, psychological, or spiritual), as measured by various factors over a period of time. In this way, the amount of computing resources and network bandwidth consumed when sending otherwise invalid message variants can be avoided.

[0003] A challenge with other approaches when selecting variants of a message may be the initial lack of data (e.g., history or response data) about a given user. This may be because the service may not have any information about content preferences or about the user's typical behavior regarding interactions with messages before the user begins to participate in the platform. Therefore, the service may have poor determinants for selecting which specific messages to send to the user's device. In the absence of any information, the service may end up selecting and sending message variants that result in little or no user interaction, indicating low-quality human-computer interaction (HCI). This may also result in wasteful consumption of computing resources on both the service and the user's device, as well as expenditure of network bandwidth.

[0004] In the context of digital therapeutic applications, among other approaches, messages may be provided via an application on a user's device to administer a treatment to a user, where in some cases the user is prompted to open the application to self-administer the therapeutic intervention. The service may have, at best, information about the user's condition to be addressed, but little or no data about the user's performance in response to the messages or compliance with the session. This lack of information on the part of the service may result in the presentation of message variations that have little or no relevance to the user. As a result, the user may experience message fatigue due to repeated presentation of little to no relevance to the user, resulting in lower compliance with the treatment regimen for the session. Low compliance with the session may, in turn, result in no change or even worsening of the user's condition to be addressed by the digital therapeutic application.

[0005] One attempt to address these issues may be to implement A / B testing. For example, a messaging platform service may define user groups where one group of users receives message variant "A" and another group of users receives message variant "B". In this setup, the messaging platform service may then collect response data and measure the performance of the various message variants from these user groups during a testing phase. Thereafter, the service may select one variant to serve across all users based on performance.

[0006] However, the A / B testing approach may have many problems. For example, since message variants are selected for the entire user group, it may be difficult to personalize the selection of message variants for a specific user. This may be the case even though certain message variants may have high interaction rates with different users. On the other hand, since personalization is not taken into account, the service may excessively collect measurements of user responses on each group, resulting in wasteful consumption of computer resources and network bandwidth and poor quality of HCI. In addition, the A / B testing approach may not be suitable for digital therapy applications because users are restricted to seeing message variants for their own group. This may limit the customizability of sessions that address their condition and leave the problem of low treatment compliance unaddressed.

[0007] To address these and other technical challenges, the service in the present invention can use a model of a specific user's response to a message variant to perform personalized selection for a given user. When a user begins to participate in an application, the service can establish a machine learning model (or statistical model). The machine learning model can be, for example, a multi-armed bandit model updated according to Thompson sampling. The model can be used to define a selection probability for each message variant, which is updated based on a performance metric indicated by the user's response to the message variant. When the service begins to collect data about user behavior, certain user preferences may begin to emerge. The data may include, for example, whether the user interacts with the message variant, whether the user responds to the message correctly, and performance data on sessions that solve user symptoms, etc.

[0008] In the context of digital therapeutic applications, message variations (e.g., notifications) may have a causal effect on user behavior. For example, a message received from a service to a user's device may cause the user to perform a task in a treatment regimen provided by the service. To promote ease and consistency for users participating in a treatment regimen, a messaging application or platform running on the device may be used to generate and push messages to encourage users to participate in sessions of the regimen. Sessions may be incorporated into an application on a user's device to address the user's mental and physical goals. For example, an application executing on a user's device may receive a message encouraging a user to participate in a task of their training regimen, and the user may subsequently participate in the task.

[0009] The message may include a mechanism for responding, such as a link within the message (e.g., a uniform resource locator (URL)), a text box, or using a different platform or application that is also application-dependent. For example, given an initial presentation of two types of message variants to a user, the user may indicate a preference for the first message over the second message by responding to the first message rather than the second message. Based on the results of the message (e.g., upon eliciting a response from the user), the prior probabilities may be updated. The service may then again randomly select the next message to send with a probability weighted by the updated prior. For example, the service may perform a Thompson sampling algorithm on multiple variants and corresponding responses to establish a set of prior probabilities for each variant of the message for the user. This iterative process may continue until the service is able to converge on the user preference or the treatment ends.

[0010] Users may prefer certain messages over other messages. The messages sent to users may be different variants. For example, one or more different types or variants of messages may be sent to a user device corresponding to the tasks of the solution. Users are more likely to engage with the messages and content they prefer. Users may prefer or respond more frequently to one variant of a message than another. This may indicate that users engage more with a particular variant of a message related to their solution than another, thereby engaging more fully with their solution as a result of receiving that variant. However, some users may not show any obvious or statistically significant preference, in which case the user may continue to receive variants of messages from the server randomly or with equal frequency.

[0011] As these preferences crystallize as indicated by the machine learning model, the service can begin to personalize the user's content by favoring the types of messages to which the user reacts. The service can aggregate the data in a machine learning model (e.g., a multi-armed bandit). The more user response data collected, the more confidence the model can have in selecting a message variation. For a digital therapeutics application, by the time the user completes a session, the service may have updated the model to converge on the user's specific messaging preferences, thereby increasing the likelihood of compliance with the therapeutic intervention provided through the message.

[0012] In this way, the consumption of computing resources (such as processing and memory) and network bandwidth of services and user devices can be reduced. By using machine learning models to estimate the variants of messages preferred by users, servers and user devices can reduce bandwidth, power or transmission of messages. In addition, the use of machine learning models can allow personalization of message variants selected for users. By customizing the variants of messages sent to users within sessions that address user symptoms, the quality of HCI between users and devices is improved. In addition, in the context of digital therapy applications, personalized selection of such message variants can result in the delivery of user-specific interventions to improve user compliance with treatment. This can not only lead to higher compliance with therapeutic interventions delivered through the session throughout the duration of the session, but also lead to potential improvements in the user's symptoms to be addressed.

[0013] The message to be delivered may include multiple variants. At least one variant may include an identification of a type or action as part of a session to be performed by the user. In some embodiments, at least one variant of the message may include an identification of a subject (e.g., health, family, work, study) of the content to be presented via the user device. In some embodiments, at least one variant of the message may include an identification of a type of content (e.g., audio, text, image, video, or other multimedia content) to be presented via the user device. In some embodiments, at least one variant may include an identification of including user information (e.g., including or excluding the user's name on the message when presented on the application). For example, a variant may specify that the user's name is included, followed by "You have one or more unfinished tasks. Click here to get started!" In some embodiments, at least one variant may specify, identify, or define the layout (e.g., positioning, size, and color) of various UI elements when presenting the message via the user interface. Various variants of the message may be intended to promote user participation or interaction. Participation may be measured by the ratio of messages sent to messages interacted with by the user. Selecting which variant of the message can encourage user participation by selecting a variant to which the user is more likely to respond.

[0014] Variants may be based on a user profile or user. For example, a set of variants may correspond to a personality associated with a user profile. Personality may indicate differences in humor, such as a message that may be interpreted by a user in a variant as "silly" or "funny," in contrast to a variant that may be interpreted by a user in a variant as "serious" or "solemn." Variants may be based on the tone of the content included in the message. For example, one variant may be interpreted as empathetic, while a second variant of the same message may be interpreted as direct. An empathetic variant may say "It's okay to have strong feelings. Through this lesson, you'll learn how to better manage them," while a direct variant may read "Perseverance and self-mastery are the keys to recovery. Log in now to complete your lesson!"

[0015] Variants can also be related to the user's health knowledge or health literacy. Health literacy can correspond to the amount of technical or scientific details given to them based on the application user's past experience. For example, regarding migraine, a variant of the message can explain to a user with health literacy via the application what exactly happens scientifically in the brain, or describe the type of symptoms they have. However, variants of messages for users with lower literacy may include simpler levels of content. For example, if the user profile indicates that the user is interested in or lacks knowledge of the cause of the disease, treatment, or anatomical structure, the variant can teach the user about knowledge in this field. In another example, the variant can say "this drug increases the dopamine level in your brain", in contrast to saying "this drug will make you feel better". Variants can also be related to the user's cognitive ability. Cognitive ability can be defined as the ability to perform the following operations: reading, learning, memory, logical reasoning, and attention. Variant 225 can be selected based on the user's cognitive ability to perform the above tasks, and different levels of content can be included based on ability.

[0016] In addition, variants may be related to the user's learning style. For example, the user may prefer to learn through visual aids rather than text. The variants presented to the user may include graphics or images that are more conducive to the user's learning style. Variants may be related to the user's personal stimulation or motivation. For example, the variant may include motivational messages such as "Good job!" or "Good job!" or otherwise provide motivation to the user (e.g., through sound effects, point systems, or other means). Variants may be related to the user's preferred messaging style. For example, the user may prefer messages with emoticons, or prefer other images than messages with only plain text that lack emoticons. Variants may be related to the user's values. For example, the user may set a value in their family or their study, and the received variant may correspond to a positive reminder of the value or encouragement based on the value. For example, the message may read "These courses can help you enjoy being with family and friends again. Start your first lesson: " or "Think about why you took these steps. Rehabilitation can help you focus on tasks at home, school, or work. Start your first lesson: " or "Remember that you are doing this for your child" or "This will help make learning much easier!" Individual variants can be selected based at least on the user profile.

[0017] Aspects of the present invention relate to systems, methods, and computer-readable media for controlling the sending of messages to users. A server may send one or more first instructions to a user device associated with a user during a first time period of a session in which the user implements a behavior endpoint, each of the one or more first instructions presenting at least one of one or more variants of a message. The server may receive one or more responses corresponding to the one or more first instructions from the user device during the first time period, each of the one or more responses of the user being a corresponding presentation of at least one of the one or more variants of the message that implements the behavior endpoint. The server may use the one or more responses to determine a corresponding performance metric for each of the one or more variants of the message in response to the passage of the first time period of the session. The server may select a variant from one or more variants of the message based on the performance metric of the variant during a second time period of the session. The server may send a second instruction to a user device associated with the user during the second time period to present the selected variant of the message.

[0018] In some embodiments, the server may determine that a second performance metric of the selected variant of the message during the second time period differs from a corresponding performance metric during the first time period by a threshold margin. The server may select a second variant from the plurality of variants of the message to send in response to determining that the second performance metric differs.

[0019] In some embodiments, the server may select a second variant of the one or more second variants of the second message to send within a third time period of the session based on the selection of the variant of the message within the second time period. In some embodiments, the server may use the one or more responses to build a model to track, for each of the one or more variants, (i) a corresponding performance metric identifying the degree of interactivity of the user with the corresponding variant of the message and (ii) a variance of the performance metric.

[0020] In some embodiments, the server may determine to select one of the one or more variants of the message for the second time period in response to determining that the variance of the performance metric is below a threshold. In some embodiments, the server may determine, based on the user's profile, a plurality of a priori probabilities of selecting corresponding plurality of variants of the message for sending during the first time period of the session.

[0021] In some embodiments, the server may provide a dashboard interface for presentation that identifies one or more statistics for the session based on a plurality of responses from a user device associated with the user. In some embodiments, determining the corresponding performance metric further comprises determining a corresponding performance metric for each of the one or more variations of the message using a profile of a user of the one or more responses and the information identifying the session.

[0022] In some embodiments, each of the one or more variations of the message may include at least one of the following: (i) identification of the type of action to be performed by the user via the user device, (ii) identification of the subject matter of the content to be presented via the user device, (iii) identification of the type of content to be presented via the user device, (iv) identification of user information included, (v) definition of the corresponding layout of the content, (vi) identification of a personality associated with the user, (v) identification of the tone of the content to be presented via the user device, (vi) identification of content based on the user's literacy, (vii) identification of content based on the user's health knowledge, (viii) identification of a learning style associated with the user, (ix) identification of personal stimulation or motivation for the user, (x) identification of the user's messaging style, (xi) identification of the user's values, or (xii) the user's cognitive abilities.

[0023] In some embodiments, the server may maintain a profile of the user on a database that identifies (i) a behavioral endpoint to be achieved, (ii) a log record of one or more responses from the user, and (iii) one or more performance metrics for one or more variations of the message. In some embodiments, the user may take medication associated with addressing the behavioral endpoint, at least partially concurrently with the session. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The foregoing and other objects, aspects, features and advantages of the present invention will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0025] Figure 1 A block diagram of a system for controlling the sending of messages to a user device according to an illustrative embodiment is described;

[0026] Figure 2 A block diagram depicting a process for selecting and sending a variant of a message during a first time period of a conversation in a system for controlling sending according to an illustrative embodiment;

[0027] Figure 3 A block diagram depicts a process for receiving a response to a message during a first time period of a session in a system for controlling transmissions according to an illustrative embodiment;

[0028] Figure 4 A block diagram depicting a process for selecting and sending a variant of a message during a second time period of a conversation in a system for controlling sending according to an illustrative embodiment;

[0029] Figure 5 depicts a set of example graphs of prior probabilities of message variants in accordance with an illustrative embodiment;

[0030] FIG. 6A to FIG. 6B A method of predicting a variant of a message sent to a user device based on a response of the user device according to an illustrative embodiment is described respectively; and

[0031] Figure 7 is a block diagram of a server system and a client computer system in accordance with an illustrative embodiment. DETAILED DESCRIPTION

[0032] In order to read the description of the various embodiments below, it may be helpful to list the sections of the specification and their corresponding contents below:

[0033] Section A describes systems and methods for adaptive selection of messages for session dependency; and

[0034] Section B describes a network and computing environment that can be used to practice the embodiments described herein.

[0035] A. Systems and methods for adaptive selection of session-dependent messages

[0036] Reference now Figure 1, depicts a block diagram of a system 100 for controlling the sending of messages to users. In general, the system 100 may include at least one session management service 105 and a set of user devices 110A-N (hereinafter generally referred to as user devices 110), which are communicatively coupled to each other via at least one network 115. At least one user device 110 (e.g., the first user device 110A as described) may include at least one application 120. The application 120 may include or provide at least one user interface 145 having one or more user interface (UI) elements 150A-N (hereinafter generally referred to as UI elements 150). The session management service 105 may include at least one session manager 155, a variant selector 160, a response processor 165, or at least one performance calculator 170, etc. The session management service 105 may include or access at least one database 175. The database 175 may store, maintain, or otherwise include one or more user profiles 180A-N (hereinafter generally referred to as user profiles 180) and one or more messages 185A-N, etc. The functionality of application 120 may be executed in part on session management service 105 .

[0037] In more detail, the session management service 105 can (sometimes generally referred to herein as a computing system or service) be any computing device that includes one or more processors coupled with memory and software and capable of performing the various processes and tasks described herein. The session management service 105 can communicate with one or more user devices 110 and a database 175 via a network 115. The session management service 105 can be located, situated, or otherwise associated with at least one server group. The server group can correspond to a data center, a branch office, or a site where one or more servers of the session management service 105 are located.

[0038] Within the session management service 105, the session manager 155 may execute, initiate, or send a message 185 related to a session initiated by a user of the application 120 on the corresponding user device 110. The variant selector 160 may select a variant of the message to be sent via the session manager to the user device 110. The response processor 165 may receive a response to the message from the user device 110. The representation calculator 170 may calculate or update a model of the response and the message.

[0039] The user device 110 (sometimes referred to herein as an end-user computing device) can be any computing device including one or more processors coupled to memory and software and capable of performing the various processes and tasks described herein. The user device 110 can communicate with the session management service 105 and the database 175 via the network 115. The user device 110 can be a smart phone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, glasses), or laptop computer. The user device 110 can be used to access the application 120. In some embodiments, the application 120 can be downloaded and installed on the user device 110 (e.g., via a digital distribution platform). In some embodiments, the application 120 can be a web application with resources accessible via the network 115.

[0040] The application 120 executed on the user device 110 may be a digital therapy application and may provide a session (sometimes referred to herein as a therapy session) that addresses at least one condition of the user. The user's condition may include, for example, chronic pain (e.g., associated with or including arthritis, migraines, fibromyalgia, back pain, Lyme disease, endometriosis, repetitive stress injuries, irritable bowel syndrome, inflammatory bowel disease, and cancer pain), skin diseases (e.g., atopic dermatitis, psoriasis, pruritus, and eczema), cognitive disorders (e.g., mild cognitive impairment (MCI), Alzheimer's disease, multiple sclerosis, and schizophrenia), and other diseases (e.g., narcolepsy and tumors), etc.

[0041] The user may simultaneously take medications to address the condition, at least in part, while being provided a session through the application 120. For example, if the medication is for pain, the user may take: acetaminophen; a nonsteroidal anti-inflammatory combination; an antidepressant; an anticonvulsant; or other combination; etc. For skin conditions, the user may take: a steroid, an antihistamine, or a topical antibacterial agent, etc. For cognitive impairment, the user may take a cholinesterase inhibitor or memantine, etc. For narcolepsy, the user may take a stimulant or an antidepressant, etc. The user of the application 120 may also perform other psychological treatments for these conditions.

[0042] Depending on the configuration on the application 120, the application 120 may include, present or otherwise provide a user interface 145 including one or more UI elements 150 to a user of the user device 110. The UI elements 150 may correspond to visual components of the user interface 145, such as command buttons, text boxes, check boxes, radio buttons, menu items, and sliders, etc. In some embodiments, the application 120 may be a digital therapy application and may provide a session (sometimes referred to herein as a therapy session) via the user interface 145 to achieve a behavioral endpoint for a user (sometimes referred to herein as a patient, person, or subject). The behavioral endpoint may be, for example, completion of a session, a physical or mental goal of a user, completion of a medication regimen, or a behavioral endpoint indicated by a physician or user.

[0043] The database 175 may store and maintain various resources and data associated with the session management service 105 and the applications 120. The database 175 may include a database management system (DBMS) that arranges and organizes the data maintained thereon. The database 175 may communicate with the session management service 105 and one or more user devices 110 via the network 115. While running various operations, the session management service 105 and the applications 120 may access the database 175 to retrieve identified data therefrom. The session management service 105 and the applications 120 may also write data to the database 175 by running such operations.

[0044] Such operations may include the sending and receiving of messages 185. Messages 185 may be MMS, SMS, or via a messaging interface of application 120. Each message 185 may identify or include information to be presented via the user interface 145 of application 120. For example, the information of message 185 may include reminders to perform session tasks. Messages 185 may be delivered periodically (e.g., daily, weekly, monthly, etc.). Messages 185 may be derived from a pre-generated psychotherapy message library and / or a pre-generated engagement (reminder) message library. Messages 185 may include reminders for subjects to complete therapy sessions, take medications, and / or complete program tasks. Messages 185 may be personalized based on the user's activity, compliance, and / or performance associated with the program.

[0045] The message 185 may also include a mechanism for responding, such as a link, a chat box, or instructions to respond to the message. The session manager 155 may coordinate the sending of the message 185 to the user device 110. The response processor 165 may coordinate the receipt of the response to the message 185 and subsequently update the user profile 180 in response to the user's response.

[0046] User profiles 180 (sometimes referred to herein as user accounts, user information, or subject profiles) can store and maintain information related to users of application 120 via user device 110A. Each user profile 180 can be associated or correspond to a corresponding user 210 (described herein) of application 120. User profiles 180 can identify various information about user 210, such as a user identifier, a condition to be addressed, information about a session conducted by user 210, message preferences, user demographic information, and progress in addressing the condition, etc. Information about a session can include various parameters of a previous session performed by user 210, and can be initially null. Message preferences can include treatment preferences and user input preferences, such as message types or timing of preferred messages. Message preferences can also include preferences determined by session management service 105, such as the type or variation of messages to which user 210 can respond. Progress can be initially set to a starting value (e.g., null or "0") and can correspond to relief, alleviation, or treatment of a condition.

[0047] In some embodiments, the user profile 180 may identify or include information about a treatment regimen taken by the user 210, such as the type of treatment (e.g., therapy, medication, or psychotherapy), duration (e.g., day, week, or year), and frequency (e.g., daily, weekly, quarterly, annually), etc. The user profile 180 may be stored and maintained in the database 175 using one or more files (e.g., extensible markup language (XML), comma-separated value (CSV) delimited text files, or structured query language (SQL) files). The user profile 180 may be iteratively updated as the user 210 performs additional sessions or responds to additional messages 185. For example, the response processor 165 may update the user profile 180 based on the responses provided by the user. The responses provided by the user may indicate engagement with the message 185, changes in demographic information, or updates to user preferences.

[0048] Reference now Figure 2 , depicts a block diagram of a process 200 for selecting and sending a variant of a message during a first time period of a conversation in a system for controlling sending. Process 200 may include or correspond to a process performed in system 100 for selecting and sending a variant of a message during a first time period of a conversation (at time T A Under process 200, a session manager 155 executing on a session management service 105 may access a database 175 to retrieve or identify a user profile 180 of a user 210 for an application 120 on a user device 110. Based on the user profile 180, the session manager 155 may determine or identify a behavior endpoint of the user 210.

[0049] The session manager 155 may determine or mark a session in which the user 210 achieves a behavioral endpoint. The application 120 executed on the user device 110 may be a digital therapy application, and may provide a session (sometimes referred to herein as a therapy session) via the user interface 145 to address at least one condition of the user 210 (sometimes referred to herein as a patient, person, or subject). The session provided via the application 120 may include a set of messages with routines or activities to be performed by the user 210 or information to be read or viewed by the user 210. The session may identify or have a set duration for the condition to be addressed. The duration may have any range of time, such as from a few hours (e.g., 2 hours) to several years (e.g., 3 years), etc. The session management service 105 presents a prompt for the message on the user interface 145 on the user device 110.

[0050] To begin a session, an application 120 on a user device 110 may provide, send, or otherwise communicate to a session management service 105 at least one request for a session for a user 210 of the application 120. The session may correspond to a set of routines or activities that the user 210 is to perform and be evaluated. The request may include or identify the user 210 to whom the session is to be provided. In some embodiments, the application 120 may send the request in response to detecting a corresponding interaction with an interface 145 of the application 120. In some embodiments, the application 120 may send the request independent of the user 210's interaction with the interface 145 (e.g., according to a schedule for addressing a condition of the user 210). The session manager 155 may initiate the session in response to receiving the request.

[0051] Using the identification of the user profile 180, the conversation manager 155 may select or label a set of messages 185 for a conversation. For example, the database 175 may have multiple sets of messages 185 for different conditions. The conversation manager 155 may select the set of messages 185 for the condition of the user 210 as identified by the user profile 180. The set of messages 185 may define or form a conversation for the user 210. Each message 185 may identify or have a set of variations 225A-N. Each variation 225 of the message 185 may convey or include at least a common portion of the information of the message 185 across all variations 225. For example, a first variation 225A may express the same prompt to perform a specific action of the same routine as a second variation 225B, but with a different format (e.g., layout), content type (e.g., text, audio, or video), or theme (e.g., with a different tone).

[0052] In some embodiments, at least one variation 225 may include an identification of a type or action as part of a session to be performed by user 210. In some embodiments, at least one variation 225 of message 185 may include an identification of a subject (e.g., health, family, work, study) of content to be presented via user device 110. In some embodiments, at least one variation 225 of message 185 may include an identification of a type of content (e.g., audio, text, image, video, or other multimedia content) to be presented via user device 110. In some embodiments, at least one variation 225 may include an identification of including user information (e.g., including or excluding the user's name on message 185 when presented on application 120). For example, variation 225 may specify that the user's name is included, followed by "You have one or more unfinished tasks. Click here to get started!" In some embodiments, at least one variation 225 may specify, identify, or define the layout (e.g., positioning, size, and color) of various UI elements 150 when presenting message 185 via user interface 145. Various variations of messages may be intended to promote engagement or interaction of user 210. Engagement may be measured by the ratio of messages 185 sent to messages 185 with which the user interacts. Selecting which variation 225 of the message 185 may encourage user engagement by selecting a variation 225 to which the user 210 is more likely to respond.

[0053] The message 185 may include or contain multiple variations 225. The variations 225 may be based on the user profile 180 or the user 210. For example, a set of variations 225 may correspond to a personality associated with the user profile 185. The personality may indicate a difference in humor, such as a message 185 that may be interpreted by the user 210 in a variation as "silly" or "funny," in contrast to a variation that may be interpreted by the user 210 in a variation as "serious" or "dignified." The variations 225 may be based on the tone of the content included in the message 185. For example, one variation 225 may be interpreted as empathetic, while a second variation 225 of the same message 185 may be interpreted as direct. The empathetic variation 225 may say "It's OK to have strong feelings. Through this lesson, you'll learn how to better manage them," while the direct variation 225 may read "Persistence and self-mastery are the keys to recovery. Log in now to complete your course!" The tone of the content may be identified in the variations 225 of the message 185.

[0054] Variant 225 may relate to the health knowledge or health literacy of user 210. Health literacy may correspond to the amount of technical or scientific details given to the user based on the past experience of the application. For example, regarding migraine, variant 225 may explain to a user with health literacy via the application what exactly happens scientifically in the brain, or describe the type of symptoms they have. However, variant 225 for users with lower literacy may include content at a simpler level. For example, if user profile 180 indicates that user 210 is interested in or lacks knowledge of the cause of disease, treatment, or anatomical structure, variant 225 may teach user 210 about knowledge in this field. For example, variant 225 may say "this drug increases the dopamine level in your brain", in contrast to saying "this drug will make you feel better". Health knowledge or literacy may be identified in variant 225 of message 185. Variant 225 may also be related to the cognitive ability of user 210. Cognitive ability may be defined as the ability to perform the following operations: reading, learning, memory, logical reasoning, and attention. Variations 225 may be selected based on the user's cognitive ability to perform the above-described tasks, and may include different levels of content based on the ability.

[0055] In addition, variants 225 may be related to the learning style of user 210. For example, user 210 may prefer to learn through visual aids rather than text. Variants 225 presented to user 210 may include graphics or images that are more conducive to the user's learning style. The user's learning style may be identified in variants 225. Variants 225 may relate to personal stimulation or incentives for user 210. For example, variants 225 may include motivational messages such as "Good job!" or "Good job!" or otherwise provide incentives to user 210 (e.g., through sound effects, a points system, or other means). Personal stimulation or incentives for user 210 may be identified in variants 225 of message 185. Variants 225 may relate to the preferred messaging style of user 210. For example, user 210 may prefer messages 185 with emoticons, or prefer other images than messages 185 with only plain text lacking emoticons. Messaging styles may be identified in variants 225 of message 185. Variants 225 may relate to the values ​​of user 210. For example, a user may set a value in their family or their studies, and the received variant 225 may correspond to a positive reminder of the value or encouragement based on the value. The value of the user 210 may be identified in the variant 225 of the message 285. For example, the message may read "These lessons can help you enjoy being with family and friends again. Start your first lesson:" or "Think about why you are taking these steps. Recovery can help you focus on tasks at home, school, or work. Start your first lesson:" or "Remember that you are doing this for your child" or "This will help make studying a lot easier!" Individual variants 225 may be selected by the variant selector 160 based at least on the user profile 180.

[0056] The variant selector 160 executing on the session management service 105 can initiate or establish at least one selection model 205 for controlling and selecting variants 225 of the message 185 provided to the user 210 associated with the user device 110. The selection model 205 (sometimes referred to herein as a machine learning model or a statistical model) can be used during a first period of time T during which the session management service 105 collects data. A Develop iteratively within a period of time. A Data collection within a specified time period (T) can facilitate the development of an accurate and robust selection model 205. In some cases, collecting data within a short period of time or collecting an insignificant amount of data may lead to an inaccurate model. A) or until a specified number of responses, sufficient responses to the variants may be recorded for the variant selector 160 to establish a selection model 205. The selection model 205 may be based on various models or algorithms, such as statistical models (e.g., multi-armed bandit or multi-class queuing network) or reinforcement models (e.g., Q-learning or Markov decision process), etc. In general, the selection model 205 may be used to track the selection probability of each variant 225, the performance metric of each variant 225, and the statistical variance of the performance of each variant 225, etc.

[0057] For example, the selection model 205 can be implemented using a multi-armed bandit model updated with Thompson sampling. The multi-armed bandit model can be a probabilistic heuristic in which multiple resources have unknown return probabilities associated with various rewards and in which the reward is maximized. In addition, each resource can provide a random reward based on an unknown probability distribution for a given resource. The multi-armed bandit model can seek to develop a measure of the reward (e.g., gain) of each resource and a probability distribution of the rewards of each resource by iteratively selecting each resource over a period of time to converge on maximizing the reward. The multi-armed bandit model can be represented as

[0058]

[0059] Among them, μ * is the maximum mean reward (e.g., when the user will interact with each variant), ρ is the regret, Τ is the number of rounds (e.g., message variants sent), and is the reward in round t (e.g., user interacted with a variant, no user interacted with a message variant, or other performance measure).

[0060] Thompson sampling can define a heuristic for selecting variants to display in a multi-armed bandit model. Sampling can include a model for selecting variants that can be expected to maximize rewards (e.g., user responses or performance metrics). Thompson sampling can include several elements, including: (1) a likelihood function: P(r|Θ, a, x); (2) a set Θ of parameters θ of the distribution of r; (3) a prior distribution P(θ); (4) past observation triples D = {(x; a; r)}; and (5) a posterior distribution P(θ|D), where x refers to the context (e.g., the user presented a message), a is the action (e.g., the user received a variant of the message), and r is the reward (e.g., the reward or the user interacted with the variant). Thompson sampling can rely on various sampling rounds (e.g., multiple message rounds to the user in different variants) to maximize the expected results over time or rounds.

[0061] The selection model 205 can be updated and developed in conjunction with the heuristics detailed herein or in conjunction with other probability models. The selection model 205 can be developed from iterative sampling of messages sent to users. In an implementation, the iterative sampling can be used to develop the success probability of each variant 225 of the message 185 (e.g., rejecting a response from a user). For example, the variant selector 160 can initially assign equal success probabilities (e.g., selection probabilities 220) to each variant 225 of the message 185. When new information related to the selection probability 220 is obtained, the prior probability (e.g., selection probability 220) can be iteratively updated. For example, the response to the message 185, the preference selected by the user, or the demographics of the user can affect or change the selection probability 220 of the variant 225. The selection probability 220 can change or change from its initial equilibrium probability as more information is obtained. In this way, the probability distribution of the success probability of each variant 225 of the message 185 can be converged.

[0062] The response pattern of the user may create statistical variance. For example, the user may not show any predictable behavior in preference of variant 225. This may result in large variance in preference of variants. In this case, variants 225 of message 185 may be sent randomly or with equal frequency. In another example, user 210 may respond to variant 225 of message 185 in a manner inconsistent with the developed selection model 205 or the prior probabilities associated with variant 225 (stored in selection model 205). This may create statistical variance within the success probability distribution of variant 225. The variance may be the expected value of the squared deviation of variant 225 from its mean. In some cases, selection model 205 may generate statistical variances over time period T. A Message response data continues to be collected from user 210 until the variance between variants 225 is below a threshold. Variant selector 160 may include other information besides user 210's response to message 185 in order to develop selection model 205.

[0063] In some embodiments, the selection model 205 may utilize information contained in a database (e.g., user profile 180) to determine the probability of a message variant. The user profile 180 may include demographic information of the user 210, such as age, weight, or health status, etc. The demographic information may be provided by the user 210 (e.g., via the user interface 145) or may be collected about the user, and may identify various characteristics about the user 210. For example, the session management service 105 may indicate that the user 210 has started a session corresponding to managing prediabetes, and may update the user profile 180 to infer that the user 210 has prediabetes. The demographic information about the user may affect the selection probability 220 of the variant 225.

[0064] In some embodiments, the variant selector 160 may use the user profile 180 to calculate, generate, or determine the prior probability of selecting each variant 225 of the message 185 in the selection model 205. For example, the variant selector 160 may use the user profile 180 to create a subgroup of users with similar demographics. The variant selector 160 may use these subgroups to generate more accurate selection probabilities for the user group. For example, the variant selector 160 may define groupings related to specific characteristics of the user (e.g., weight, height, health status, or any combination thereof) to determine the commonality between the preferred variants 225 of the group. Using this definition, the variant selector 160 may mark the selection model 205 of the variants 225 across the user group. The variant selector 160 may establish the selection model 205 of the user 210 based on the combination of the selection models 205 of the variants 225 across the user group. Based on the selection models 205 of other users from the same group, the combined selection model 205 may have a priori selection probability biased towards at least one variant 225. In this way, variation selector 160 may present variations 225 that may be more likely to cause user 210 to engage with message 185 .

[0065] Variant selector 160 may indicate or otherwise select at least one variant 225 from a set of variants 225 of message 185 to send. The selection may be made during a first time period of the session (e.g., at time T A The variants 225 may be selected by the variant selector 160 according to the selection model 205 during the period of time. For each variant 225, the variant selector 160 may calculate, determine, or indicate a corresponding probability of selection 220, as described herein. The variant selector 160 may continuously update the selection model 205, or may update the selection model 205 periodically or in response to new data (e.g., user responses, changes in user demographics). The variant selector 160 may select a variant 225 of the message 185 for transmission based on the variant 225 having the highest probability of selection 220. In this way, the variant 225 may be presented to the user 210 (via the application 120 on the user device 110) with a high likelihood of prompting the user to participate.

[0066] By selecting the variant 225, the session manager 155 may generate at least one instruction 215 that is sent to the user device 110. The instruction may include the selected variant 225 of the message 185. The instruction 215 may be code, data packet, or control that presents the message to the user 210 in at least one variant 225. The instruction 215 may include processing instructions for displaying the message on the application 120. The instruction 215 may also include instructions for the user 210 to perform related to their session. For example, the instruction 215 may display a message 185 instructing the user to take medication associated with their session. When generated, the session manager 155 may send the instruction 215 to the user device 110.

[0067] The application 120 on the user device 110 may retrieve, indicate, or otherwise receive an instruction 215 identifying a selected variation 225 of the message 185 from the session management service 105. Upon receipt, the application 120 may display the message 185 via the user interface 145 via the application 120. The user interface 145 may display a variation 225A of the message 185A that is different from a variation 225B of the same message 185A. For example, the variation 225 may refer to a different display or presentation of the same message. For example, the variation 225 may be a different layout of the UI elements 150 displayed on the user interface 145. The message 185 may include or display a prompt, a button, a text box, a link (e.g., a URL), or other means for the user 210 to interact or engage with the message 185. For example, the presentation of the message 185 may include a button that asserts that the user 210 has performed the task of his session. The prompt to engage with the message 185 may be an interaction 305. The interaction 305 may indicate that the user 210 has engaged with the variation 225 of the message 185.

[0068] Reference now Figure 3 , depicts a block diagram of a process 300 for receiving a response to a message during a first time period of a session in a system for controlling transmission. The process 300 may include or correspond to receiving a response to a message during a first time period of a session (e.g., at time T A 145 ). Under process 300, application 120 running on user device 110 may monitor at least one interaction 305 with user interface 145. Interaction 305 may include a key press, a touch screen, a click, or any other event associated with presenting message 185 via user interface 145. In some embodiments, interaction 305 may be after the presentation of variation 225 of message 185. For example, after presenting variation 225 of message 185 prompting user 210 to perform a routine, interaction 305 may correspond to a set of user interactions that record the completion of a specified routine. In some embodiments, interaction 305 may occur simultaneously with presenting message 185 via user interface 145.

[0069] Upon detecting the interaction 305, the application 120 may generate or produce at least one response 315 for sending to the session management service 105. The response 315 may identify or include information related to the user's completion of the task, the user 210's response, or general participation in the message 185 or the session. In some embodiments, the application 120 may maintain a timer to track the time that has elapsed since the presentation of the message 185. The application 120 may compare the elapsed time to the time limit of the variation 225 of the message 185. When the elapsed time exceeds the time limit, the application 120 may generate a response 315 indicating that there was no user interaction with the variation 225 of the message 185. Upon generation, the application 120 may provide, transmit, or otherwise send the response 315 to the session management service 105.

[0070] The response processor 165 executing on the session management service 105 can retrieve, mark, or otherwise receive the response 315 from the user device 110. Upon receipt, the response processor 165 can parse the response 315 to extract or mark the interaction 305 of the user 210 with the presented variation 225 of the message 185. The response processor 165 can record or load data from the response 315 into the database 175. The response processor 165 can store an association between the response 315 and the user profile 180. In some embodiments, the response processor 165 can store and maintain an identification of the variation 225 of the session in the database 175. The response processor 165 can store an association between the response 315 of the user 210 and the variation 225.

[0071] The performance calculator 170 executing on the session management service 105 may calculate, generate, or otherwise determine a performance metric 310 for the variant 225 of the message 185. The performance metric 310 may be a value (e.g., a numerical value) that identifies or corresponds to the user 210's interaction 305 with the variant 225 of the message 185 presented on the user interface 145. The performance metric 310 may be assigned or set to one value when the presented variant 225 causes the interaction 305, and may be assigned or set to another value when the variant 225 does not cause the interaction 305. In general, more interactions with the variant 225 of the message 185 may result in a higher performance metric 310, while fewer or no interactions with the variant 225 of the message 185 when presented may result in a lower performance metric 310. In some embodiments, the performance metric 310 may correspond to a rate of correct or appropriate responses by the user 210 to the variant 225 of the message 185. The rate of correct or appropriate responses may be related to adherence to the session to address the condition. The performance calculator 170 can analyze information related to the responses 315 and the variants 225 and can perform statistical calculations. For example, the performance calculator can calculate the variance of the frequency of the responses 315, the relationship between a specific variant 225 and a specific response 315 rate, or the relationship between the user profile 180 and the variant 225.

[0072] In some embodiments, performance metric 310 may be related to the variance of responses 315 associated with variants 225 for a given user 210. For example, performance calculator 170 may calculate the variance between variants 225 that have prompted interactions 305 from user 210 and variants 225 that have not prompted interactions 305 from user 210. Performance calculator 170 may also use the variance of responses 315 associated with variants 225 at time T. A The performance metric 310 of each variant 225 of the message 185 is calculated based on the responses 315 collected within and the user profile 180 corresponding to the responses 315. In this way, the performance metric 310 can relate to the degree of interactivity of the user 210 with the corresponding variant 225 of the message 185.

[0073] Using performance metrics 310, performance calculator 170 may modify or update selection model 205. As described above, selection model 205 may track, among other things, the probability of selection of each variant 225, the performance metrics of each variant 225, and the statistical variance of the performance of each variant 225. Performance calculator 170 may update selection model 205 based on performance metrics 310 to change the probability of selection 220 of variant 225A. For example, if user 210 never responds to variant 225A, performance calculator 170 may assign a lower probability of selection 220 to the presented variant 225. Conversely, if user 210 responds to variant 225, performance calculator 170 may assign a higher probability of selection 220. In either scenario, by adding responses 315, performance calculator 170 may reduce the statistical variance of variants 225 in selection model 205.

[0074] The performance calculator 170 may modify or update the user profile 180 to identify or include the performance metrics 310, the selection probabilities 220, or other statistics associated with the user 210 represented by the user profile 180. The performance calculator 170 may maintain information related to the user 210 on the user profile 180 in the database 175. The user profile 180 may identify the user's behavioral endpoints, a log record of responses 315 associated with the user, or a set of performance metrics 310 associated with the user 210 for the variations 225 presented to the user 210. The performance calculator 170 may update the selection model 205 based on the information within the user profile 180, the performance metrics 310, or a combination thereof.

[0075] By updating the selection model 205, the performance calculator 170 can determine whether the first time period T of the session has passed. A Time period T A The time period T may correspond to a portion of the session during which any variant 225 of the message 185 may be selected for sending to the user 210. During this time, the session management service 105 may continue to collect responses 315 from the user device 110 for use in selecting a variant 225 of the message 185 to provide at least a portion of the remainder of the session. A The passage of can correspond to at least one variance of the selection model 205 being below a threshold, a threshold number of responses 315 received by the session management service 105, a set time limit, etc.

[0076] In some embodiments, the performance calculator 170 may determine whether a time limit of the first time period of the session has passed. The time limit may be defined relative to the start of the session and may range from a few seconds (e.g., 3 seconds) to a few weeks (e.g., 10 weeks), etc. When the time limit has passed, the performance calculator 170 may determine that the first time period T of the session has passed. AOtherwise, when the time limit has not yet passed in the session, the performance calculator 170 may determine the first time period T of the session. A Not passed yet.

[0077] In some embodiments, the performance calculator 170 can determine whether at least one statistical variance of the corresponding variant 225 of the message 185 in the selection model 205 is below a threshold. The threshold can describe or identify a value of the statistical variance at which, for at least a portion of the remainder of the session, sufficient confidence is obtained for selecting the variant 225. When the variance is less than the threshold, the performance calculator 170 can determine that a first time period T of the session has passed. A Otherwise, when the variance is greater than the threshold, the performance calculator 170 may determine the first time period T of the session A The session management service 105 may then repeat processes 200 and 300 discussed herein.

[0078] In some embodiments, performance calculator 170 may determine whether the number of responses 315 from user 210 is greater than a threshold. The threshold may describe or identify a value of the number of responses 315 at which sufficient confidence is obtained for selecting variant 225 for at least a portion of the remainder of the session. When the number of responses is greater than or equal to the threshold, performance calculator 170 may determine that a first time period T of the session has elapsed. A Otherwise, when the number of responses is less than the threshold, the performance calculator 170 may determine the first time period T of the session. A The session management service 105 may then repeat processes 200 and 300 discussed herein.

[0079] After determining that the first time period has elapsed, a second time period T of the session may be started. B In some embodiments, after the first time period has elapsed, the session management service 105 may stop recording responses, calculating variances, or combining T A In some embodiments, the passage of the first time period can indicate that enough data has been captured in the selection model 205 to make a variant 225 selection by the session management service 105 that is likely to result in a response 315 from the user 210 .

[0080] Reference now Figure 4 , describes a block diagram of a process 400 for selecting and sending a variant of a message during a second time period of a conversation in a system for controlling sending. The process 400 may include a process related to selecting and sending a message during a second time period of the conversation (at time T BProcess 400 may include one or more operations discussed in conjunction with process 200 or 300. Under process 400, session manager 155 may continue the session (now in a second time period). In the second time period of the session, session manager 155 may mark or select at least one message 185' sent to user 210. In some embodiments, message 185' may be the same as message 185 from the first time period in the session. In some embodiments, message 185' may be different from message 185 sent during the first time period of the session.

[0081] In some embodiments, the conversation manager 155 may indicate or select another set of messages 185' to send to the user 210 during at least a portion of the second time period of the conversation. A Afterwards, the conversation manager 155 may indicate or select a second group of messages 185'. The second group of messages 185' may be for the same conversation established to address the condition of the user 210. The second group of messages 185' may have a relationship with the first group of messages 185 used in the first time period. The relationship between the message groups may be defined according to the specification of the conversation. For example, the second group of messages 185' may be used to send messages in the time period T A The subsequent routine or activity prompted by message 185 is in the example. A The message may involve performing breathing exercises, B The message in may relate to performing a running workout.

[0082] Each message 185' may also have a set of variants 225'AN, similar to the set of variants 225 described above. Each variant 225' of the message 185' may convey or include at least a common portion of the information of the message 185' across all variants 225', with different action types, subjects, content types, inclusion of user information (e.g., name), or layout, etc. Each variant 225' of the message 185' may have a corresponding relationship with a variant 225 of the message 185. For example, a variant 225' and a variant 225 may have a corresponding relationship with the same type of action, subject, content type, whether or not user information or layout is included, etc.

[0083] Using this identification, variant selector 160 may use selection model 205 to identify or select variant 225' of message 185'. Variant selector 160 may select variant 225' of message 185' determined from the first time period of the session as having the highest probability of selection 220'. In some embodiments, variant selector 160 may use selection model 205 to select a variant 225 of message 185' that is different from message 185 selected during the first time period. To select, variant selector 160 may identify variant 225 with the highest probability of selection 220' from selection model 205. Using this identification, variant selector 160 may find, determine, or otherwise select a variant 225' of message 185' that has a corresponding relationship with variant 225 of message 185. For example, variant selector 160 may select a variant 225' of message 185' that has the same layout as variant 225 of message 185.

[0084] In some embodiments, variant selector 160 may indicate or select a variant 225' of message 185' that is different from the highest probability of selection 220 determined from the first time period using selection model 205. The probability of selection 220' of performance metric 310 of variant 225 may be determined from the first time period T A To the second time period T B Such changes may be due to various reasons, such as changes in user 210 preferences, changes in the user's condition or circumstances, changes in one or more variants 225 (e.g., adding or removing additional variants 225), changes in user device 110 that will result in a different presentation of variants 225 to the user (e.g., a user purchases a different mobile device to display application 120 on), and other factors.

[0085] To determine, variant selector 160 may determine that probability of selection 220' in the second time period differs from probability of selection 220 during the first time period by a threshold margin. The threshold margin may identify or define the difference between probability of selection 220 and probability of selection 220' by which the preference of user 210 deviates. In some embodiments, performance metrics from user 210 may be used instead of or in addition to probability of selection. When the difference is greater than the threshold margin, variant selector 160 may select a different variant 225' that is different from the initially selected variant 225. Otherwise, when the difference is less than the threshold margin, variant selector 160 may maintain the initially selected variant 225.

[0086] Upon selection, the session manager 155 may generate at least one instruction 215' that is sent to the user device 110. The instruction may include a selected variation 225' of the message 185'. The instruction 215' may be code, a data packet, or a control that presents the message to the user 210 in at least one variation 225'. The instruction 215' may include processing instructions for displaying the message on the application 120. The instruction 215' may also include instructions for the user 210 to perform related to their session. For example, the instruction 215' may display the message 185' that instructs the user to perform a specific activity associated with their session. Upon generation, the session manager 155 may send the instruction 215' to the user device 110.

[0087] The application 120 on the user device 110 may retrieve, indicate, or otherwise receive an instruction 215' identifying a selected variation 225' of the message 185' from the session management service 105. Upon receipt, the application 120 may display the message 185' via the user interface 145 by the application 120. Through the presentation, the application 120 may monitor at least one interaction 305' with the user interface 145. In some embodiments, the interaction may be subsequent to the presentation of the variation 225' of the message 185'. For example, after presenting the variation 225' of the message 185' prompting the user 210 to perform a routine, the interaction 305' may correspond to a set of user interactions that record the completion of the specified routine. In some embodiments, the interaction 305' may occur simultaneously with the presentation of the message 185' via the user interface 145.

[0088] Upon detecting the interaction 305', the application 120 may generate or produce at least one response 315' for sending to the session management service 105. The response 315' may identify or include information related to the user's completion of the task, the user 210's response, or general participation in the message 185' or the session. In some embodiments, the application 120 may maintain a timer to track the time that has elapsed since the message 185' was presented. The application 120 may compare the elapsed time to the time limit of the variant 225' of the message 185'. When the elapsed time exceeds the time limit, the application 120 may generate a response 315' indicating that there was no user interaction with the variant 225' of the message 185'. Upon generation, the application 120 may provide, transmit, or otherwise send the response 315' to the session management service 105.

[0089] The response processor 165 may retrieve, mark, or otherwise receive a response 315' from the user device 110. Upon receipt, the response processor 165 may parse the response 315' to extract or mark the interaction 305 of the user 210 with the presented variation 225' of the message 185'. The response processor 165 may record or load data from the response 315' into the database 175. The response processor 165 may store an association between the response 315' and the user profile 180. In some embodiments, the response processor 165 may store and maintain identification of the variation 225' of the conversation in the database 175. The response processor 165 may store an association between the response 315' of the user 210 and the variation 225'.

[0090] In some embodiments, the performance calculator 170 may calculate, generate, or otherwise determine a performance metric 310' of a variant 225' of a message 185'. The performance metric 310' may be a value (e.g., a numerical value) that identifies or corresponds to an interaction 305 of a user 210 with a variant 225' of a message 185' presented on the user interface 145. The performance metric 310' may be assigned or set to one value when the presented variant 225' causes an interaction 305, and may be assigned or set to another value when the variant 225' does not cause an interaction 305. In general, more interactions with the variant 225' of the message 185' may result in a higher performance metric 310', while fewer or no interactions with the variant 225' of the message 185' when presented may result in a lower performance metric 310'. In some embodiments, the performance metric 310' may correspond to a rate of correct or appropriate responses of the user 210 to the variant 225' of the message 185'. The performance calculator 170 may analyze information related to the responses 315' and the variants 225', and may perform statistical calculations. For example, the performance calculator may calculate the variance of the frequency of responses 315', the relationship between a particular variant 225' and a particular response 315' rate, or the relationship between a user profile 180 and a variant 225'.

[0091] Using the performance metrics 310', the performance calculator 170 may modify or update the selection model 205. As described above, the selection model 205 may track the selection probabilities of the individual variants 225', the performance metrics of the individual variants 225', and the statistical variance of the performance of the individual variants 225', etc. The performance calculator 170 may update the selection model 205 based on the performance metrics 310' to change the selection probability 220' of the variant 225'A. In addition, the performance calculator 170 may modify or update the user profile 180 to identify or include the performance metrics 310', the selection probabilities 220, or other statistics associated with the user 210 represented by the user profile 180. The performance calculator 170 may maintain information related to the user 210 on the user profile 180 in the database 175. The user profile 180 may identify the user's behavioral endpoints, a log record of responses 315 associated with the user, or a set of performance metrics 310' associated with the user 210 for the variants 225' presented to the user 210. Performance calculator 170 may update selection model 205 based on information within user profile 180, performance metric 310', or a combination thereof. In some embodiments, performance calculator 170 may forgo calculation of performance metric 310' during a second time period of the session.

[0092] In some embodiments, the developed selection model 205 may no longer fit the preferences of the user 210. For example, the user 210 may provide a response 315' that is outside the estimation of the selection model or exceeds a threshold variance within the selection model 205. Therefore, the selection model 205 may need to be modified or changed. The selection model 205 may need to be modified in response to changes in the variants 225 selected by the user 210 that result in a statistical deviation or variance exceeding a threshold amount. The performance calculator 170 may determine that at the second time period T B The performance metric 310 of the variants during the period exceeds a threshold. For example, the variance of the responses 315 to the message 185 may exceed a threshold. The selection model 205 may need to be edited as a result of changes to the user input. For example, the user may add or remove data related to the user's demographic information, which may affect the selection probability 220 of the variant 225. For example, the user may select a variant 225 during the period T. A The prediabetic condition has not yet been listed during the period, but is subsequently indicated via the user interface 145 during the time period T B The prediabetic condition is listed in . Subsequently, the performance calculator 170 may recalculate the performance metric 310 associated with the one or more variants 225 .

[0093] In some embodiments, the performance calculator 170 may provide a dashboard interface for presentation. The dashboard interface may be presented on a display of the session management service 105 or on a computing device communicatively coupled to the session management service 105. The dashboard interface may provide a presentation of one or more statistics for the session based on responses received from the user device 110 associated with the user 210. For example, the dashboard interface may display or present individual responses 315 from the user 210, performance metrics 310 associated with individual variations 225 of the message 185, selection probabilities and statistical variances indicated by the selection model 205, and information associated with the user 210, etc. The dashboard interface may be displayed at different time periods (e.g., T A or T B ) during the display of the selection model 205. The dashboard interface can be presented as a graphical user interface (e.g., GUI), a table, a series of graphics, or another display medium on the server. The dashboard interface can change dynamically as the response processor 165 collects responses 315.

[0094] In this way, the session management service 105 can provide control over the sending of messages in a networked environment. By using the response data to update the selection model 205 during the first time period, the session management service 105 can be able to identify the variants 225 of the message 185 that have the highest performance metrics for the particular user 210. In doing so, the session management service 105 can reduce computing resources across the system 100 by providing the variants 225 of the message 185 that are most likely to prompt the user 210 to engage. In addition, the session management service 105 can generally reduce wasteful sending of instructions 215 for messages 185 that will not elicit interaction from the user 210. By selecting the variants 225 of the message 185 in this way, the session management service 105 can also improve the quality of human-computer interaction (HCI) between the user 210 and the user interface 145 of the application 120 on the user device 110.

[0095] Reference now Figure 5 , depicting a set of example graphs 500 of prior probabilities of message variants. The examples depicted may involve iterative development of a selection model 205 in which each variant 225 in a variant group has an equal probability of selection 220. As responses 315 are received from a user 210, the prior probabilities of each variant (e.g., probability of selection 220) may vary and be updated over time periods T. A ends and converges to the final distribution.

[0096] In an illustrative example, variants 225A and 225B may be provided by Figure 5 Each variant can be represented by the curve described in the time period T A225A is sent to user 210 within 1 second. In this example, variant 225A may be more attractive to user 210 than variant 225B and may have an actual interaction rate of 30%, while variant 225B may have an interaction rate of approximately 20%. Initially, there may be no reason to believe that either of the variants is more attractive, so each user in a group of users may receive either variant with equal probability.

[0097] As the response processor 165 collects more responses 315, it may appear that variant 225A appears to have a slightly higher interaction rate. Therefore, the session management service 105 may associate variant 225A with a higher probability (i.e., more frequent than variant 225B). However, the session manager 155 may not stop sending variant 225B until the system is statistically confident that variant 225A is preferred. In this way, more engaging messages may be favored.

[0098] Reference now Fig. 6A and Figure 6B , describes a method 600 for predicting a variant of a message to be sent to a client based on a response of the client. The various figures describe operations performed by the client and operations performed by the service. The method 600 can be implemented or performed using any of the components detailed herein (e.g., the session management service 105 and the user device 110). Fig. 6A Initially, under method 600, a service (e.g., session management service 105) may identify a user profile (e.g., user profile 180) (605). The service may build a model (e.g., selection model 205) of variants (e.g., group of variants 225) of a message (e.g., message 185) (610). The service may select a variant (e.g., variant 225) based on a prior (e.g., selection probability 220) (615). The service may send an instruction (e.g., instruction 215) for a variant of the message to a user device (e.g., user device 110) (620). The user device may receive the instruction (e.g., instruction 215) (625). The user device may present the variant of the message identified in the instruction (630). The user device may send a response (e.g., response 315) to the service (635). The service may receive a response from the user device (640). The service may determine a performance metric (e.g., performance metric 310) (645). The service may update the model (650).

[0099] Move to Figure 6B, the service may check whether there is enough data (655). This determination may be based on the amount of time that has passed in the session or the degree of variance in the model of the variants. If there is not enough data, the service may return to (615) and repeat method 600. If there is enough data, the service may select the variant with the highest performance (660). The service may send an instruction (e.g., instruction 215') (665) including the selected variant of the message. The client may receive the instruction from the service (670). The user device may present the variant of the message (675). The user device may send a response (e.g., response 315') (680) to the server. The service may receive a response from the user device (685). The service may check whether the session is complete (690). If the session is not complete, the service may return to (660) and repeat method 600. If the session is complete, the service may terminate the session (695).

[0100] B. Network and computing environment

[0101] Various operations described herein can be implemented on a computer system. Figure 7 A simplified block diagram of a representative server system 700, a client computer system 714, and a network 726 that can be used to implement certain embodiments of the present invention is shown. In various embodiments, the server system 700 or a similar system can implement the services or servers described herein or portions thereof. The client computer system 714 or a similar system can implement the clients described herein. The system 100 described herein can be similar to the server system 700. The server system 700 can have a modular design incorporating multiple modules 702 (e.g., blades in a blade server embodiment); although two modules 702 are shown, any number of modules can be provided. Each module 702 can include one or more processing units 704 and local storage 706.

[0102] One or more processing units 704 may include a single processor (which may have one or more cores) or multiple processors. In some embodiments, one or more processing units 704 may include a general-purpose main processor and one or more special-purpose coprocessors, such as a graphics processor, a digital signal processor, etc. In some embodiments, some or all of the processing units 704 may be implemented using custom circuits, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA). In some embodiments, such an integrated circuit executes instructions stored on the circuit itself. In other embodiments, one or more processing units 704 may execute instructions stored in local storage 706. Any type of processor in any combination may be included in one or more processing units 704.

[0103] The local storage 706 may include volatile storage media (e.g., DRAM, SRAM, SDRAM, etc.) and / or non-volatile storage media (e.g., magnetic disks or optical disks, flash memory, etc.). The storage media incorporated into the local storage 706 may be fixed, removable, or upgradeable as desired. The local storage 706 may be physically or logically divided into various subunits, such as system memory, read-only memory (ROM), and permanent storage devices. The system memory may be a read-write memory device or a volatile read-write memory, such as a dynamic random access memory. The system memory may store some or all instructions and data required by one or more processing units 704 at runtime. The ROM may store static data and instructions required by one or more processing units 704. The permanent storage device may be a non-volatile read-write memory device that may store instructions and data even when the module 702 is powered off. The term "storage medium" as used herein includes any medium in which data may be stored indefinitely (subject to overwriting, electrical interference, power loss, etc.), and does not include carrier waves and transient electronic signals that are propagated wirelessly or through wired connections.

[0104] In some embodiments, local storage 706 may store one or more software programs to be executed by one or more processing units 704, such as an operating system and / or programs that implement various server functions (e.g., functions of system 100 or any other system described herein or any other one or more servers associated with system 100 or any other system described herein).

[0105] "Software" generally refers to a sequence of instructions that, when executed by one or more processing units 704, causes the server system 700 (or portions thereof) to perform various operations, thus defining one or more specific machine embodiments that execute and perform the operations of the software program. Instructions may be stored as firmware residing in read-only memory and / or program code stored in non-volatile storage media, which may be read into volatile working memory for execution by one or more processing units 704. Software may be implemented as a single program or as a collection of separate programs or program modules that interact as desired. From local storage 706 (or non-local storage described below), one or more processing units 704 may retrieve program instructions to be executed and data to be processed in order to perform the various operations described above.

[0106] In some server systems 700, multiple modules 702 may be interconnected via a bus or other interconnect 708, forming a local area network that supports communications between modules 702 and other components of the server system 700. The interconnect 708 may be implemented using a variety of technologies, including server racks, hubs, routers, and the like.

[0107] Wide area network (WAN) interface 710 can provide data communication capabilities between a local area network (e.g., via interconnect 708) and a network 726, such as the Internet. Other technologies can be used to communicatively couple the server system to network 726, including wired (e.g., Ethernet, IEEE 802.3 standards) and / or wireless technologies (e.g., Wi-Fi, IEEE 802.11 standards).

[0108] In some embodiments, local storage 706 is intended to provide working memory for one or more processing units 704, thereby providing fast access to programs and / or data to be processed while reducing traffic on interconnect 708. Storage for larger amounts of data can be provided on a local area network by one or more mass storage subsystems 712 that can be connected to interconnect 708. Mass storage subsystem 712 can be based on magnetic, optical, semiconductor or other data storage media. Direct attached storage, storage area networks, network attached storage, etc. can be used. Any data storage or other data collection described herein as being generated, used or maintained by a service or server can be stored in mass storage subsystem 712. In some embodiments, additional data storage resources can be accessed via WAN interface 710 (potentially with increased latency).

[0109] The server system 700 can operate in response to requests received via the WAN interface 710. For example, one module 702 can implement a supervisory function and assign discrete tasks to other modules 702 in response to received requests. Work distribution techniques can be used. When a request is processed, the result can be returned to the requester via the WAN interface 710. Such operations can typically be automated. Further, in some embodiments, the WAN interface 710 can connect multiple server systems 700 to each other, thereby providing a scalable system capable of managing a large number of activities. Other techniques for managing server systems and server farms (a collection of collaborative server systems) can be used, including dynamic resource allocation and reallocation.

[0110] The server system 700 can interact with various user-owned or user-operated devices via a wide area network such as the Internet. Examples of user-operated devices are shown in FIG. Figure 7 714. The client computing system 714 may be implemented as, for example, a consumer device such as a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, glasses), desktop computer, laptop computer, etc.

[0111] For example, client computing system 714 may communicate via WAN interface 710. Client computing system 714 may include computer components such as one or more processing units 716, storage devices 718, network interfaces 720, user input devices 722, and user output devices 724. Client computing system 714 may be a computing device implemented in a variety of form factors, such as a desktop computer, a laptop computer, a tablet computer, a smartphone, other mobile computing devices, a wearable computing device, etc.

[0112] The processing unit 716 and storage device 718 may be similar to the one or more processing units 704 and local storage 706 described above. Suitable devices may be selected based on the requirements to be placed on the client computing system 714; for example, the client computing system 714 may be implemented as a "thin" client with limited processing capabilities or as a high-power computing device. The client computing system 714 may be provided with program code that can be executed by the one or more processing units 716 to implement various interactions with the server system 700.

[0113] The network interface 720 may provide a connection to a network 726, such as a wide area network (e.g., the Internet), to which the WAN interface 710 of the server system 700 is also connected. In various embodiments, the network interface 720 may include a wired interface (e.g., Ethernet) and / or a wireless interface implementing various RF data communication standards, such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, LTE, etc.).

[0114] User input device 722 may include any device (or devices) through which a user may provide signals to client computing system 714; client computing system 714 may interpret the signals as indicating a particular user request or information. In various embodiments, user input device 722 may include any or all of a keyboard, touch pad, touch screen, mouse or other pointing device, scroll wheel, click wheel, dial, buttons, switches, keypad, microphone, etc.

[0115] The user output device 724 may include any device through which the client computing system 714 can provide information to the user. For example, the user output device 724 may include a display-to-display image generated by or delivered to the client computing system 714. The display may incorporate various image generation technologies, such as a liquid crystal display (LCD), a light emitting diode (LED) display including an organic light emitting diode (OLED), a projection system, a cathode ray tube (CRT), etc., and supporting electronic devices (e.g., digital-to-analog or analog-to-digital converters, signal processors, etc.). Some embodiments may include devices such as touch screens, which are used as both input devices and output devices. In some embodiments, other user output devices 724 may be provided in addition to or in place of a display. Examples include indicator lights, speakers, tactile "display" devices, printers, etc.

[0116] Some embodiments include electronic components, such as microprocessors, storages and memories, which store computer program instructions in computer-readable storage media. Many features described in this specification can be implemented as a process designated as a group of program instructions encoded on a computer-readable storage medium. When these program instructions are executed by one or more processing units, they cause one or more processing units to perform the various operations indicated in the program instructions. Examples of program instructions or computer codes include machine codes such as those produced by a compiler and files including high-level codes executed by a computer, electronic components or microprocessors using an interpreter. Through appropriate programming, one or more processing units 704 and 716 can provide various functions for server system 700 and client computing system 714, including any function or other functions described herein as being executed by a server or client.

[0117] It will be understood that the server system 700 and the client computing system 714 are illustrative, and various changes and modifications are possible. The computer system used in conjunction with an embodiment of the present invention may have other capabilities not specifically described herein. Further, although the server system 700 and the client computing system 714 are described with reference to specific blocks, it should be understood that these blocks are defined for the convenience of description and are not intended to imply a specific physical arrangement of component parts. For example, different blocks may but need not be located in the same facility, in the same server rack, or on the same motherboard. Further, these blocks do not need to correspond to physically different components. Blocks may be configured to perform various operations, such as by programming a processor or providing appropriate control circuits, and various blocks may be reconfigurable or may not be reconfigurable, depending on how the initial configuration is obtained. Embodiments of the present invention may be implemented in various devices including electronic devices implemented using any combination of circuits and software.

[0118] Although the present invention has been described with respect to specific embodiments, it will be appreciated by those skilled in the art that many modifications are possible. Embodiments of the present invention may be implemented using various computer systems and communication technologies, including but not limited to the specific examples described herein. Embodiments of the present invention may be implemented using any combination of dedicated components and / or programmable processors and / or other programmable devices. The various processes described herein may be implemented on the same processor or on different processors in any combination. Where components are described as being configured to perform certain operations, such configurations may be implemented, for example, by designing electronic circuits to perform operations, by programming programmable electronic circuits (e.g., microprocessors) to perform operations, or any combination thereof. Further, although the above-described embodiments may refer to specific hardware and software components, it will be appreciated by those skilled in the art that different combinations of hardware and / or software components may also be used, and specific operations described as being implemented in hardware may also be implemented in software, and vice versa.

[0119] The computer program incorporating the various features of the present invention may be encoded and stored on a variety of computer-readable storage media; suitable media include disks or tapes, optical storage media such as compact disks (CDs) or digital versatile disks (DVDs), flash memory, and other non-transient media. The computer-readable medium encoded with the program code may be packaged with a compatible electronic device, or the program code may be provided separately from the electronic device (e.g., via Internet download or as a separately packaged computer-readable storage medium).

[0120] Therefore, although the invention has been described with respect to particular embodiments, it will be understood that the invention is intended to cover all modifications and equivalents within the scope of the appended claims.

Claims

1. A method for controlling the sending of a message to a user, comprising: sending, by a server, a plurality of first instructions to a user device associated with a user during a first time period of a session in which the user implements a behavior endpoint, each of the plurality of first instructions presenting at least one of a plurality of variations of a message; receiving, by the server, a plurality of responses corresponding to the plurality of first instructions from the user device within the first time period, each of the plurality of responses of the user being a respective presentation of at least one of the plurality of variations of the message that implements the behavior endpoint; determining, by the server in response to the passage of the first time period of the session, a respective performance metric for each of the plurality of variants of the message using the plurality of responses; selecting, by the server, a variant from the plurality of variants of the message based on the performance metric of the variant during a second time period of the session; as well as A second instruction is sent by the server to the user device associated with the user during the second time period to present the selected variation of the message.

2. The method according to claim 1, further comprising: determining, by the server, that a second performance metric of the selected variant of the message during the second time period differs from the corresponding performance metric during the first time period by a threshold margin; as well as A second variant of the message is selected to be sent by the server in response to determining that the second performance metric is different from the plurality of variants.

3. The method according to claim 1, further comprising: A second variant of a plurality of second variants of a second message is selected by the server to send during a third time period of the session based on the selection of the variant of the message during the second time period.

4. The method according to claim 1, further comprising: The plurality of responses are used by the server to build a model to track, for each of the plurality of variations, (i) a respective performance metric identifying a degree of interactivity of the user with the corresponding variation of the message and (ii) a variance of the performance metric.

5. The method according to claim 1, further comprising: The selection of one of the plurality of variants of the message within the second time period is determined by the server in response to determining that a variance of the performance metric is below a threshold.

6. The method according to claim 1, further comprising: A plurality of a priori probabilities of selecting a corresponding plurality of variations of the message for sending during the first time period of the conversation are determined by the server based on a profile of the user.

7. The method according to claim 1, further comprising: A dashboard interface is provided for presentation by the server, the dashboard interface identifying one or more statistics for the session based on the plurality of responses from the user devices associated with the user.

8. The method according to claim 1, wherein: Determining the respective performance metrics further includes determining the respective performance metrics for each of the plurality of variations of the message using the plurality of responses and a profile of the user identifying the information of the conversation.

9. The method according to claim 1, wherein: Each of the multiple variations of the message includes at least one of the following: (i) identification of the type of action to be performed by the user via the user device, (ii) identification of the subject matter of the content to be presented via the user device, (iii) identification of the type of content to be presented via the user device, (iv) identification of user information included, (v) definition of the corresponding layout of the content, (vi) identification corresponding to the personality associated with the user, (v) identification of the tone of the content to be presented via the user device, (vi) identification of content based on the user's literacy, (vii) identification of content based on the user's health knowledge, (viii) identification of a learning style associated with the user, (ix) identification of personal stimulation or motivation for the user, (x) identification of a messaging style based on the user, (xi) identification of the user's values, or (xii) the user's cognitive abilities.

10. The method according to claim 1, further comprising: A profile of the user is maintained by the server on a database, the profile identifying (i) the behavioral endpoint to be implemented, (ii) a log record of the plurality of responses from the user, and (iii) a plurality of performance metrics of the plurality of variations of the message.

11. The method according to claim 1, wherein: The user takes a medication associated with addressing the behavioral endpoint at least partially concurrently with the session.

12. A system for controlling the sending of messages to a user, comprising: At least one server having one or more processors coupled to a memory, configured to: sending a plurality of first instructions to a user device associated with a user during a first time period of a session in which the user implements a behavior endpoint, each of the plurality of first instructions presenting at least one of a plurality of variations of a message; receiving, from the user device, a plurality of responses corresponding to the plurality of first instructions within the first time period, each of the plurality of responses of the user being a respective presentation of at least one of the plurality of variations of the message that implements the behavior endpoint; determining a respective performance metric for each of the plurality of variations of the message using the plurality of responses in response to the passage of the first time period of the conversation; selecting a variant from the plurality of variants of the message based on the performance metric of the variant during a second time period of the session; as well as A second instruction is sent to the user device associated with the user during the second time period to present the selected variation of the message.

13. The system according to claim 12, wherein: The at least one server is further configured to: determining that a second performance metric of the selected variant of the message during the second time period differs from the corresponding performance metric during the first time period by a threshold margin; as well as A second variant of the message is selected to be sent from the plurality of variants in response to determining that the second performance metric is different.

14. The system according to claim 12, wherein: The at least one server is further configured to select a second variant of a plurality of second variants of a second message to send within a third time period of the session based on the selection of the variant of the message within the second time period.

15. The system of claim 12, wherein: The at least one server is also configured to use the plurality of responses to build a model to track, for each of the plurality of variants, (i) a corresponding performance metric identifying a degree of interactivity of the user with the corresponding variant of the message and (ii) a variance of the performance metric.

16. The system of claim 12, wherein: The at least one server is further configured to determine to select one of a plurality of variants of the message within the second time period in response to determining that a variance of the performance metric is below a threshold.

17. The system of claim 12, wherein: The at least one server is further configured to determine, based on the profile of the user, a plurality of a priori probabilities of selecting the corresponding plurality of variants of the message for sending during the first time period of the conversation.

18. The system of claim 12, wherein: The at least one server is further configured to provide a dashboard interface for presentation that identifies one or more statistics for the session based on the plurality of responses from the user devices associated with the user.

19. The system of claim 12, wherein: The at least one server is further configured to determine the respective performance metric for each of the plurality of variations of the message using the plurality of responses and a profile of the user identifying the information of the conversation.

20. The system of claim 12, wherein: Each of the multiple variations of the message includes at least one of the following: (i) identification of the type of action to be performed by the user via the user device, (ii) identification of the subject matter of the content to be presented via the user device, (iii) identification of the type of content to be presented via the user device, (iv) identification of user information included, (v) definition of the corresponding layout of the content, (vi) identification corresponding to the personality associated with the user, (v) identification of the tone of the content to be presented via the user device, (vi) identification of content based on the user's literacy, (vii) identification of content based on the user's health knowledge, (viii) identification based on a learning style associated with the user, (ix) identification of personal stimulation or motivation for the user, (x) identification of the user's messaging style, (xi) identification of the user's values, or (xii) the user's cognitive abilities.

21. The system of claim 12, wherein: The at least one server is further configured to maintain a profile of the user on a database, the profile identifying (i) the behavioral endpoint to be implemented, (ii) a log record of the multiple responses from the user, and (iii) multiple performance metrics of the multiple variations of the message.

22. The system of claim 12, wherein: The user takes a medication associated with addressing the behavioral endpoint at least partially concurrently with the session.

Images