Adaptive selection of messages transmitted in a network environment to improve session adherence.

Abstract

This specification provides a system and method for controlling the transmission of messages in a network environment. A server can send a command to a user device associated with a user, each commanding the user to present a message in at least one of a set of variants. The server can receive a set of responses corresponding to the command, each response being from the user for each presentation of the message in at least one of the variants. The server can use the responses to determine each performance metric for the message in each variant and update a selection model in accordance with the responses. The server can select one variant from the variants based on its performance metric. The server can send a second command to the user device associated with the user, causing the user to present a message in the selected variant.

Description

[Technical Field]

[0001] [Cross-reference of related applications] This application claims priority to international patent application PCT / US / 2023 / 021208, filed on 5 May 2023, entitled “Adaptive Selection of Messages Transmitted in a Network Environment for Improving Session Adherence,” which is incorporated herein by reference in its entirety. [Background technology]

[0002] Poor adherence to treatment is a serious problem in the healthcare industry. Patients' difficulty in complying with treatment can render clinical outcomes ineffective and increase costs. Despite industry efforts, medication adherence rates remain low and have not changed significantly over time. There is no effective and manageable way to help patients adhere to prescribed treatments. A method is needed to improve patient adherence and ultimately impact clinical outcomes. [Overview of the project] [Problems that the invention aims to solve]

[0003] The messaging service described herein allows the service to select which of several types of messages to send to a user. The types may correspond to different types of content, themes, tones, or layouts of a given message, or may be based on factors such as the user's personality, literacy, health knowledge, learning style, personal extrinsic or intrinsic motivation, messaging style, or preference for including the user's name. The purpose of the selection may be to provide variants that maximize user engagement, as measured by interaction rates. In messaging platforms related to digital therapy, a further objective may include session adherence to address the user's state over time (e.g., behavioral, psychological, or mental), as measured by various factors. This avoids the amount of computing resources and network bandwidth consumed by transmitting invalid message variants.

[0004] A challenge with other methods for selecting message variants may be the initial lack of data about a given user (e.g., history or response data). This is because, before a user begins engaging with the platform, the service may not have information about the user's typical behavior regarding content preferences or interactions with messages. As a result, the service may lack the decision-making factors to select a particular message to send to the user's device. Without this information, the service is essentially experiencing no user interaction during message variant selection and sending, which indicates a low quality of human-computer interaction. This can also lead to wasted computing resources on both the service and the user's device, as well as depletion of network bandwidth.

[0005] In other approaches to digital therapeutic applications, messages can be delivered to the user via an application on the user's device, and in some cases, the user may be prompted to open an application to self-manage therapeutic interventions. The service may have only information about the user's condition to be addressed, and little to no data on the user's actions in response to messages or adherence to sessions. This lack of information on the service side can lead to the presentation of message variants that are little to no relation to the user. As a result, the user may experience message fatigue from repeated presentations that are little to no relation to them, which can lead to decreased adherence to the treatment plan for that session. Poor adherence to sessions may further fail to change, or even worsen, the user's condition being addressed through the digital therapeutic application.

[0006] One attempt to address these issues could be to conduct A / B testing. For example, a messaging platform service could define user groups, where one group receives message variant "A" and another group receives message variant "B". In this setup, the messaging platform service can collect response data from these user groups during the testing phase to measure the performance of each message variant. The service can then select one variant to provide to all users based on its performance.

[0007] However, A / B testing methods can have many problems. For example, by selecting a message variant for the entire user population, it can be difficult to personalize the selection of a message variant for a specific user. This is true even if a particular message variant may have a high interaction rate with different users. Another example is that because personalization is not considered, the service may collect too many measurements of user responses for each group, leading to wasted computing resources and network bandwidth and poor quality HCI. Furthermore, since users only see the message variant for their own group, A / B testing methods may be unsuitable for digital therapeutic applications. This can limit the customization of sessions to address individual conditions and leave the problem of low adherence to treatment unresolved. [Means for solving the problem]

[0008] To address these and other technical challenges, the Services of this Disclosure can make personalized choices for a given user using a model of a specific user's response to a message variant. As a user begins to engage with the application, the Services can establish a machine learning model (or statistical model). The machine learning model may be, for example, a multi-armed bandit model updated according to Thompson sampling. Using this model, selection probabilities for each message variant can be defined, which are updated based on performance metrics shown from the user's responses to the message variants. As the Services begin to collect data on user behavior, specific user preferences may become apparent. This data may include, for example, whether the user interacted with a message variant, whether the user responded correctly to the message, and performance data regarding sessions to address the user's state.

[0009] In the context of digital therapeutic applications, message variants (e.g., notifications) can have a causal effect on user behavior. For example, a message received by a user device from a service can prompt the user to perform tasks in a therapeutic plan provided by the service. To ensure ease and consistency for users participating in a therapeutic plan, a messaging application or platform running on the device can be used to generate and push messages encouraging the user to participate in the plan's sessions. Sessions can be embedded in an application on the user's device to address the user's physical and mental goals. For example, an application running on the user's device may receive a message prompting the user to participate in a training plan task, after which the user can participate in the task.

[0010] Messages may include mechanisms for responses, such as links within the message (e.g., Uniform Resource Locators (URLs)), text boxes, or the use of different platforms or applications, which may also relate to the application. For example, if two types of message variants are initially presented to a user, the user may indicate their preference for the first message over the second by responding to the first message and not responding to the second. Based on the outcome of the message (e.g., if a response is elicited from the user), prior probabilities can be updated. The service can then randomly select the next message to send with probabilities weighted by the updated prior probabilities. For example, the service may run a Thompson sampling algorithm on a large number of variants and corresponding responses to establish a set of prior probabilities for each message variant to the user. This iterative process can continue until the service can converge to a user preference or treatment termination.

[0011] Users may prefer certain messages to others. Messages sent to a user may be of different variants. For example, one or more different types or variants of messages may be sent to a user's device corresponding to a task in their plan. Users are more likely to engage with the messages and content they prefer. Users may prefer or respond to one message variant more frequently than another. This indicates that a user is more engaged with a particular message variant related to their plan than another, and thereby is more fully participating in their plan as a result of receiving that variant. However, some users may not show any obvious or statistically significant preferences, in which case they may continue to receive message variants from the server randomly or at equal frequency.

[0012] As these preferences materialize, as indicated by machine learning models, services can begin to personalize user content by prioritizing the types of messages users respond to. Services can aggregate data from machine learning models (e.g., multi-armed bandits). The more user response data collected, the more confident the model may be in selecting one of the message variants. In digital therapeutic applications, by the time a user completes a session, services can update the model to converge on the user's specific messaging preferences, thereby increasing the likelihood of adherence to therapeutic interventions delivered through messages.

[0013] In this way, the consumption of computing resources (e.g., processing and memory) and network bandwidth of the service and user devices can be reduced. By using machine learning models to estimate message variants preferred by the user, the server and user devices can reduce the bandwidth, power, or transmission of messages. Furthermore, the use of machine learning models may make it possible to personalize the message variants selected for that user. By customizing the message variants sent to the user within a session to address the user's condition, the quality of HCI between the user and the device is improved. Moreover, in the context of digital therapeutic applications, personalizing the selection of such message variants can provide user-specific interventions and improve user adherence to treatment. This not only increases adherence to therapeutic interventions provided throughout the duration of the session, but may also lead to improvements in the user's condition that needs to be addressed.

[0014] The delivered message may include several variants. At least one variant may include an identification of the type or action the user should perform as part of a session. In some embodiments, at least one variant of the message may include an identification of the theme of the content presented via the user device (e.g., health, family, work, study). In some embodiments, at least one variant of the message may include an identification of the type of content presented via the user device (e.g., audio, text, images, video, or other multimedia content). In some embodiments, at least one variant may include an identification of whether or not to include user information (e.g., whether or not to include the user's name in the message when presented in the application). For example, a variant might include the user's name and then specify, "You have one or more big missions. Tap here to get started!" In some embodiments, at least one variant may specify, identify, or define the layout of individual UI elements (e.g., position, size, and color) when the message is presented via the user interface. Each variant of the message may be designed to facilitate user engagement or interaction. Engagement can be measured by the ratio of messages sent to messages in which users interact. Message variant selection can enhance user engagement by choosing variants that users are more likely to respond to.

[0015] Variants can be based on a user profile or a user. For example, a set of variants can correspond to the personality associated with a user profile. Personality can indicate a difference in humor, such as a message where the user interprets a variant as "serious" or "solemn" in contrast to a variant where the user interprets it as "funny" or "comical". Variants can be based on the tone of the content included in the message. For example, one variant of a message can be interpreted as empathetic, while a second variant of the same message can be interpreted as direct. The empathetic variant can be something like "It's okay to have strong emotions. In this lesson, you'll learn how to control them well.", while the direct variant might be something like "Perseverance and a sense of self - ownership are the keys to recovery. Log in now and take the lesson!"

[0016] Variants can also be related to the user's health knowledge or health literacy. Health literacy can correspond to the amount of technical or scientific details the users of the app have obtained based on past experiences. For example, regarding migraines, a variant of the message can explain scientifically what exactly is happening in the brain to a health - literate person or state the type of the user's symptoms via the application. However, a variant of the message for a user with low literacy can contain a simpler level of content. For example, if the user profile indicates that the user is interested in that field or lacks knowledge in that field, the variant can teach the user about the disease, treatment, or anatomical structure. In another example, the variant can be something like "This drug will improve your mood" instead of "This drug increases dopamine levels in the brain". Variants can also be related to the user's cognitive abilities. Cognitive abilities can be defined as the ability to read, learn, remember, make logical inferences, and pay attention. Variant 225 can be selected based on the cognitive abilities of the user performing the above tasks and can contain different levels of content based on the abilities.

[0017] Furthermore, the variant can be related to the user's learning style. For example, the user may prefer to learn through visual aids rather than text. The variant presented to the user may include graphics or images that are helpful according to the user's learning style. The variant can be related to the user's personal extrinsic or intrinsic motivation. For example, the variant can include messages that encourage motivation such as "Well done!" or "Excellent!", or can give the user motivation through sound effects, a point system, or other means. The variant can be related to the user's preferred messaging style. For example, the user may prefer messages with emojis or other images rather than plain text messages without emojis. The variant can be related to the user's values. For example, the user may value their family or study, and the received variant can correspond to something that reliably reminds them of those values or an encouragement based on those values. For example, messages can be "These lessons will enable you to enjoy time with your family and friends again. Let's start the first lesson", or "Think about why you are following such procedures. If you recover, you will be able to concentrate on tasks at home, school, or work. Let's start the first lesson", or "Don't forget that you are doing this for your children", or "This will make studying more enjoyable!". Each variant can be selected at least based on the user profile.

[0018] Aspects of this disclosure relate to systems, methods, and computer-readable media for controlling the transmission of messages to a user. During a first period of a user session toward achieving a behavioral endpoint, the server may send one or more first commands to a user device associated with the user, each command causing the message to be presented in at least one of one or more variants. During the first period, the server may receive one or more responses from the user device corresponding to one or more first commands, each of which is from the user toward each presentation of the message in at least one of one or more variants toward achieving the behavioral endpoint. As the first period of the session progresses, the server may use one or more responses to determine each performance metric for the message in each of the one or more variants. For a second period of the session, the server may select one variant from the one or more variants of the message based on the performance metric of that variant. During the second period, the server may send a second command to the user device associated with the user, causing the message to be presented in the selected variant.

[0019] In some embodiments, the server can determine that the second performance metric for a message in a selected variant during the second period differs from each performance metric during the first period by a threshold margin. Based on this determination, the server can select the second variant from several variants of the message to send.

[0020] In some embodiments, the server may, for a third period of a session, select one of one or more second variants of a second message to send, based on the selection of message variants from the second period. In some embodiments, the server may, using one or more responses, establish a model that tracks, for each variant of one or more variants, (i) each performance metric that identifies the degree of user interactivity with the message of the corresponding variant, and (ii) the variance of the performance metrics.

[0021] In some embodiments, the server may decide to select one of one or more variants of the message for the second period based on a determination that the variance of the performance evaluation metric is below a threshold. In some embodiments, the server may determine multiple prior probabilities for selecting corresponding variants of the message to send during the first period of the session, based on the user's profile.

[0022] In some embodiments, the server may provide a dashboard interface for presentations that identify one or more statistics of a session based on multiple responses from user devices associated with the user. In some embodiments, determining each performance metric further includes determining each performance metric for messages in each variant of one or more variants using one or more responses and a user profile that identifies information for the session.

[0023] In some embodiments, each variant of one or more variants of a message may include at least one of the following: (i) identification of the type of action performed by the user via the user device; (ii) identification of the theme of the content presented via the user device; (iii) identification of the type of content presented via the user device; (iv) identification of the inclusion of user information; (v) definition of each layout of the content; (vi) identification corresponding to the user's personality; (v) identification of the tone of the content presented via the user device; (vi) identification of content based on the user's literacy; (vii) identification of content based on the user's health knowledge; (viii) identification of the user's learning style; (ix) identification of the user's personal extrinsic or intrinsic motivations; (x) identification based on the user's messaging style; (xi) identification of the user's values; or (xii) the user's cognitive abilities.

[0024] In some embodiments, the server may maintain a profile in the database that identifies (i) the action endpoint to be achieved, (ii) a log of one or more responses from the user, and (ii) one or more performance metrics for one or more variants of the message. In some embodiments, the user may be taking medication related to addressing the action endpoint at least partially concurrently with the session.

[0025] The above and other purposes, aspects, features, and advantages of this disclosure will be made clearer and more readily understood by referring to the following description together with the accompanying drawings. [Brief explanation of the drawing]

[0026] [Figure 1] A block diagram of a system for controlling the sending of messages to a user device, according to an exemplary embodiment, is shown. [Figure 2] A block diagram is shown of a process in a transmission control system according to an exemplary embodiment, in which a message variant is selected and transmitted during the first period of a session. [Figure 3] A block diagram of a process for receiving a response to a message during the first period of a session in a system that controls transmission, according to an exemplary embodiment, is shown. [Figure 4] A block diagram of a process for selecting and sending a message variant during a second session in a system that controls transmission, according to an exemplary embodiment, is shown. [Figure 5] An example of a graph of prior probabilities for message variants is shown in an exemplary embodiment. [Figure 6A] An exemplary embodiment demonstrates a method for predicting variants of messages to be sent to a user device based on the user device's response. [Figure 6B] An exemplary embodiment demonstrates a method for predicting variants of messages to be sent to a user device based on the user device's response. [Figure 7] This is a block diagram of a server system and a client computer system according to an exemplary embodiment. [Modes for carrying out the invention]

[0027] When reading the descriptions of the various embodiments below, it may be helpful to list the sections of the specification and their contents as follows.

[0028] Section A describes systems and methods for adaptively selecting messages to improve session adherence.

[0029] Section B describes network and computing environments that may be useful for carrying out the embodiments described herein.

[0030] A. Systems and methods for adaptively selecting messages to improve session adherence Referring next to Figure 1, a block diagram of system 100 that controls the sending of messages to users is shown. In general, system 100 may include at least one session management service 105 and a set of user devices 110A-N (hereinafter collectively referred to as user device 110) that are interconnected and can communicate with each other via at least one network 115. At least one user device 110 (e.g., the first user device 110A shown) may include at least one application 120. Application 120 may include or provide at least one user interface 145 having one or more user interface (UI) elements 150A-N (hereinafter collectively referred to as UI elements 150). Session management service 105 may, in particular, include at least one session management unit 155, one variant selection unit 160, one response handling unit 165, or at least one performance calculation unit 170. The session management service 105 may include or access at least one database 175. The database 175 may, in particular, store, maintain, or otherwise include one or more user profiles 180A-N (hereinafter collectively referred to as user profile 180) and one or more messages 185A-N. The functionality of application 120 may be partially performed by the session management service 105.

[0031] More specifically, the session management service 105 (which may also be referred to herein as a computing system or service as a whole) may be any computing device capable of performing the various processes and tasks described herein, comprising one or more processors coupled with memory and software. The session management service 105 may communicate with one or more user devices 110 and database 175 via the network 115. The session management service 105 may be located, installed, or otherwise associated with at least one group of servers. The group of servers may correspond to a data center, branch office, or any location where one or more servers corresponding to the session management service 105 are located.

[0032] Within the session management service 105, the session management unit 155 can execute, start, or send messages 185 related to sessions initiated by users of the application 120 on each user device 110. The variant selection unit 160 can select a variant of the message to send to the user device 110 via the session management unit. The response handling unit 165 can receive responses to messages from the user device 110. The performance calculation unit 170 can calculate or update models of responses and messages.

[0033] The user device 110 (sometimes referred to herein as an end-user computing device) may be any computing device capable of performing the various processes and tasks described herein, comprising one or more processors coupled with memory and software. The user device 110 may communicate with the session management service 105 and the database 175 via the network 115. The user device 110 may be a smartphone, another mobile phone, a tablet computer, a wearable computing device (e.g., a smartwatch, glasses), or a laptop computer. The application 120 can be accessed using the user device 110. In some embodiments, the application 120 may be downloaded and installed on the user device 110 (e.g., via a digital distribution platform). In some embodiments, the application 120 may be a web application having resources accessible via the network 115.

[0034] An application 120 running on the user device 110 may be a digital therapeutic application and may provide a session (which may also be referred to herein as a therapeutic session) to address at least one of the user's conditions. Examples of user conditions include, for example, chronic pain (e.g., related to or including arthritis, migraine, fibromyalgia, back pain, Lyme disease, endometriosis, recurrent stress disorder, irritable bowel syndrome, inflammatory bowel disease, and cancer pain), skin conditions (e.g., atopic dermatitis, psoriasis, dermatophyte, and eczema), cognitive impairment (e.g., mild cognitive impairment (MCI), Alzheimer's disease, multiple sclerosis, and schizophrenia), and other diseases (e.g., narcolepsy and tumors).

[0035] While receiving sessions through Application 120, users may be taking medication to address their condition, at least partially simultaneously. For example, if the medication is for pain, users may be taking acetaminophen, nonsteroidal anti-inflammatory compositions, antidepressants, anticonvulsants, or other compositions. For skin conditions, users may be taking steroids, antihistamines, or topical disinfectants. For cognitive impairment, users may be taking cholinesterase inhibitors or memantine. For narcolepsy, users may be taking psychostimulants or antidepressants. Users of Application 120 may also receive other psychotherapies for these conditions.

[0036] Application 120 may include, present to, or otherwise provide to the user of a user device 110, a user interface 145 containing one or more UI elements 150, depending on the configuration on Application 120. The UI elements 150 may correspond to visual components of the user interface 145, in particular, command buttons, text boxes, checkboxes, radio buttons, menu items, and sliders. In some embodiments, Application 120 may be a digital therapeutic application that can provide a session (sometimes referred to herein as a therapeutic session) via the user interface 145 toward achieving behavioral endpoints of the user (sometimes referred to herein as a patient, person, or subject). Behavioral endpoints may be, for example, completion of a session, the user's physical or mental goals, completion of a medication plan, or behavioral endpoints indicated by the physician or user.

[0037] Database 175 may store and maintain various resources and data related to the session management service 105 and application 120. Database 175 may include a database management system (DBMS) for organizing and structuring the data maintained therein. Database 175 can communicate with the session management service 105 and one or more user devices 110 via the network 115. During the execution of various operations, the session management service 105 and application 120 can access database 175 and retrieve identified data from it. The session management service 105 and application 120 can also write data to database 175 from the execution of such operations.

[0038] Such operations may include sending and receiving messages 185. Messages 185 may be sent via MMS, SMS, or the messaging interface of application 120. Each message 185 may identify or include information presented via the user interface 145 of application 120. For example, the information in message 185 may include a reminder to perform a session task. Messages 185 can be delivered regularly, such as daily, weekly, or monthly. Messages 185 can be derived from a library of pre-generated psychotherapy messages and / or a library of pre-generated engagement (reminder) messages. Messages 185 may include reminders to complete a therapy session, take medication, and / or complete a plan task. Messages 185 can be personalized based on the user's activity, adherence, and / or performance with respect to the plan.

[0039] Message 185 may also include a mechanism for responding, such as a link, a chat box, or an instruction to respond to the message. The session management unit 155 can coordinate the sending of message 185 to the user device 110. The response handling unit 165 can coordinate the reception of responses to message 185 and subsequently update the user profile 180 according to the user's response.

[0040] A user profile 180 (which may also be referred to herein as a user account, user information, or subject profile) can store and maintain information about the user of application 120 via the user device 110A. Each user profile 180 may be associated with or correspond to each user 210 (as described herein) of application 120. In particular, the user profile 180 can identify various information about user 210, such as a user identifier, a condition to be addressed, information about the session being performed by user 210, message preferences, user demographic information, and the progress of addressing the condition. Information about the session may include various parameters of the previous session performed by user 210, and may initially be null. Message preferences may include treatment intentions and user input preferences, such as preferred message types or message timing. Message preferences may also include preferences determined by the session management service 105, such as the types or variants of messages that user 210 may respond to. The progress can be initially set to a starting value (e.g., null or "0") and may correspond to the mitigation, alleviation, or treatment of the condition.

[0041] In some embodiments, the user profile 180 may identify or include information about the treatment plan that user 210 is working on, in particular, the type of treatment (e.g., physiotherapy, medication, or psychotherapy), the duration (e.g., days, weeks, or years), and the frequency (e.g., daily, weekly, quarterly, or annually). The user profile 180 may be stored and maintained in the database 175 using one or more files (e.g., Extensible Markup Language (XML), comma-separated value (CSV) delimited text file, or Structured Query Language (SQL) file). The user profile 180 may be updated iteratively when user 210 performs an additional session or responds to an additional message 185. For example, the response handling unit 165 may update the user profile 180 based on a user response. The user response may indicate engagement with a message 185, a change in demographic information, or an update in user preferences.

[0042] Next, referring to Figure 2, a block diagram of process 200 in the system that controls transmission selects and transmits a message variant during the first period of the session. Process 200 is responsible for selecting and transmitting a message variant during the first period of the session (time T). A This may include, or correspond to, operations performed by system 100 to select and send a message. Process 200 allows a session management unit 155, running in session management service 105, to access database 175 to retrieve or identify the user profile 180 of user 210 of application 120 on user device 110. From the user profile 180, the session management unit 155 can determine or identify the user 210's behavioral endpoints.

[0043] The session management unit 155 can determine or identify a session of user 210 toward achieving a behavioral endpoint. The application 120 running on the user device 110 may be a digital therapeutic application and can provide a session (sometimes referred to herein as a therapeutic session) via the user interface 145 to address at least one condition of user 210 (sometimes referred to herein as a patient, person, or subject). The session provided via the application 120 may include a set of messages containing routines or activities that user 210 should perform or information that user 210 should read or view. The session may specify or have a set duration for the condition to be addressed. The duration may be any time range, particularly from a few hours (e.g., 2 hours) to several years (e.g., 3 years). Message prompts are presented to the user interface 145 on the user device 110 by the session management service 105.

[0044] To initiate a session, an application 120 on a user device 110 may provide, send, or otherwise transmit to the session management service 105 a request for at least one session for user 210 of application 120. The session may correspond to a set of routines or activities that user 210 is to perform and evaluate. The request may include or specify the user 210 to whom the session is to be provided. In some embodiments, application 120 may send a request in response to the detection of a corresponding interaction with interface 145 of application 120. In some embodiments, application 120 may send a request independently of user 210's interaction with interface 145, for example, according to a schedule for addressing user 210's state. The session management unit 155 may initiate a session upon receiving a request.

[0045] By identifying the user profile 180, the session management unit 155 can select or identify a set of messages 185 for the session. For example, the database 175 may have multiple sets of messages 185 for different states. The session management unit 155 can select a set of messages 185 for the state of user 210 identified in the user profile 180. The set of messages 185 can define or form user 210's session. Each message 185 may identify or have a set of variants 225A to N. Each variant 225 of message 185 may convey or include at least a common portion of the information of message 185 across the variants 225. For example, a first variant 225A may present the same prompt to perform a specific action in the same routine as a second variant 225B, but in a different format (e.g., layout), content type (e.g., text, audio, or video), or theme (e.g., a different tone).

[0046] In some embodiments, at least one variant 225 may include an identification of the type or action that the user 210 should perform as part of a session. In some embodiments, at least one variant 225 of message 185 may include an identification of the theme of the content presented via the user device 110 (e.g., health, family, work, study). In some embodiments, at least one variant 225 of message 185 may include an identification of the type of content presented via the user device 110 (e.g., audio, text, images, video, or other multimedia content). In some embodiments, at least one variant 225 may include an identification of whether or not to include user information (e.g., whether or not to include the user's name in the message when presented in application 120). For example, variant 225 may include the user's name and then specify, "You have one or more big missions. Tap here to get started!" In some embodiments, at least one variant 225 may specify, identify, or define the layout (e.g., position, size, and color) of individual UI elements 150 when the message 185 is presented via the user interface 145. Each variant of the message may be intended to facilitate engagement or interaction by the user 210. Engagement can be measured by the ratio of messages 185 sent to messages 185 with which the user interacts. The selection of variants 225 of the message 185 can facilitate user engagement by selecting variants 225 that the user 210 is more likely to respond to.

[0047] Message 185 may have or contain multiple variants 225. Variants 225 can be based on user profile 180 or user 210. For example, a set of variants 225 may correspond to a personality associated with user profile 185. Personality may show differences in humor, such as a variant of Message 185 that user 210 interprets as "serious" or "dignified," in contrast to a variant that user 210 interprets as "playful" or "humorous." Variants 225 can be based on the tone of the content contained in Message 185. For example, one variant 225 may be interpreted as empathetic, while a second variant 225 of the same Message 185 may be interpreted as direct. An empathetic variant 225 might say, "It's okay to have strong emotions. In this lesson, you'll learn how to control them effectively," while a direct variant 225 might say, "Perseverance and a sense of self-possession are key to recovery. Log in now and take the lesson!" The tone of the content can be identified in variant 225 of message 185.

[0048] Variant 225 may relate to user 210's health knowledge or health literacy. Health literacy may correspond to the amount of technical or scientific detail the app user has acquired based on past experiences. For example, regarding migraines, variant 225 could explain exactly what is happening scientifically in the brain to someone with health literacy, or it could describe the type of symptoms the user has through the application. However, variant 225 for users with lower literacy may contain a simpler level of content. For example, if user profile 180 indicates that user 210 is interested in or has little knowledge in that area, variant 225 could teach user 210 about a disease, treatment, or anatomical structure. For example, variant 225 might say "This drug will increase dopamine levels in your brain" rather than "This drug will make you feel good." Health knowledge or health literacy may be identified in variant 225 of message 185. Variant 225 may also relate to user 210's cognitive abilities. Cognitive abilities can be defined as the ability to read, learn, remember, logical reasoning, and pay attention. Variant 225 can be selected based on the cognitive abilities of the user performing the above tasks and may include different levels of content based on those abilities.

[0049] Furthermore, variant 225 may relate to user 210's learning style. For example, user 210 may prefer learning through visual aids rather than text. Variant 225 presented to user 210 may include graphics or images that are more helpful to the user's learning style. The user's learning style may be identified in variant 225. Variant 225 may relate to user 210's personal extrinsic or intrinsic motivation. For example, variant 225 may include motivational messages such as "Well done!" or "Great!", or motivate the user through sound effects, a point system, or other means. User 210's personal extrinsic or intrinsic motivation may be identified in variant 225 of message 185. Variant 225 may relate to user 210's preferred messaging style. For example, user 210 may prefer message 185 with emoticons or other images to message 185 with only plain text and no emoticons. The messaging style can be identified by a variant 225 of message 185. Variant 225 can relate to the user 210's values. For example, the user may value their family or their studies, and the received variant 225 can be one that reliably reminds them of those values ​​or one that provides encouragement based on those values. The user 210's values ​​can be identified by a variant 225 of message 285. For example, the message could be: "These lessons will help you enjoy time with family and friends again. Let's start the first lesson," or "Think about why you're going through this process. Once you recover, you'll be able to focus on tasks at home, school, or work. Let's start the first lesson," or "Remember you're doing this for your kids," or "This will make studying easier!" Each variant 225 can be selected by the variant selection unit 160 based at least on the user profile 180.

[0050] The variant selection unit 160, which operates in the session management service 105, can start or establish at least one selection model 205 for controlling and selecting variants 225 of message 185 to be provided to the user 210 associated with the user device 110 of message 185. The selection model 205 (which may also be referred to herein as a machine learning model or statistical model) operates during the first period T in which data is collected by the session management service 105. A It can develop repeatedly over a period of T. A Collecting data over a specified period (T) can facilitate the development of an accurate and robust selection model 205. In some cases, collecting data over a short period or in small amounts of data may lead to an inaccurate model. Therefore, collecting data over a specified period (T) is necessary. A During or up to a specified number of responses, the variant selection unit 160 can record responses to enough variants to establish a selection model 205. The selection model 205 can be based on various models or algorithms, in particular, such as statistical models (e.g., multi-armed bandits or multi-class queuing networks) or reinforcement models (e.g., Q-learning or Markov decision processes). Generally, the selection model 205 can be used to track, in particular, the selection probability for each variant 225, the performance evaluation metric for each variant 225, and the statistical variance of performance for each variant 225.

[0051] For example, selection model 205 can be implemented using a multi-armed bandit model updated with Thompson sampling. A multi-armed bandit model can be a probabilistic heuristic in which multiple resources have unknown recursive probabilities associated with various payoffs and it is desirable to maximize the payoff. Furthermore, each resource can provide a random payoff depending on the unknown probability distribution of a given resource. In a multi-armed bandit model, by iteratively selecting each resource over a certain period of time to converge to the maximized payoff, it is possible to pursue the development of metrics for the payoff of each resource (e.g., gain) and the probability distribution of payoffs for each resource.

number

[0052] Thompson sampling can define a heuristic for selecting variants demonstrated in multi-armed bandit models. This sampling may include a model for selecting variants that are expected to maximize a payoff (e.g., user response or performance rating measure). Thompson sampling may include several elements, including (1) a likelihood function: P(r|Θ,a,x); (2) a set of parameters θ of the distribution of r Θ; (3) a prior distribution P(θ); (4) a triplet of past observations D={(x;a;r)}; and (5) a posterior distribution P(θ|D), where x is the context (e.g., the user presented the message), a is the action (e.g., the user received a variant of the message), and r is the reward (e.g., a payoff or interaction between the user and the variant). Thompson sampling may rely on varying numbers of sampling (e.g., multiple message rounds with different variants) to maximize the expected outcome over time or rounds.

[0053] The selection model 205 can be updated and developed in conjunction with heuristics or other probability models detailed herein. The selection model 205 can be developed from iterative sampling of messages sent to a user. In one embodiment, this iterative sampling can be used to formulate the probability of success (e.g., eliciting a response from the user) for each variant 225 of message 185. For example, the variant selection unit 160 may initially assign equal success probabilities (e.g., selection probabilities 200) to each variant 225 of message 185. This prior probability (e.g., selection probability 220) can be iteratively updated as new information about the selection probability 220 becomes available. For example, responses to message 185, user-selected preferences, or user demographics may result in or change the selection probability 220 of variant 225. The selection probability 220 may change or deviate from the initial equalized probability as more information becomes available. In this way, the probability distribution of the likelihood of success for each variant 225 of message 185 can be converged.

[0054] User response patterns can introduce statistical variance. For example, a user may not exhibit predictable behavior regarding their preference for variant 225. This can lead to a large variance in variant preferences. In this case, variant 225 of message 185 can be sent randomly or at equal frequency. In another example, user 210 may respond to variant 225 of message 185 in a way that does not conform to the developed selection model 205 or the prior probabilities associated with variant 225 (stored in selection model 205). This can introduce statistical variance within the probability distribution of success for variant 225. The variance may be the expected value of its squared deviation from the mean of variant 225. In some cases, selection model 205 may continue for a period T until the variance among variants 225 falls below a threshold. A The system can continue to collect message response data from user 210. The variant selection unit 160 may include information other than user 210's response to message 185 in order to develop the selection model 205.

[0055] In some embodiments, the selection model 205 can determine the probability of a message variant by utilizing information contained in a database, such as a user profile 180. The user profile 180 may include demographic information of the user 210, such as age, weight, or health status. Demographic information can be provided by the user 210 (e.g., via the user interface 145) or collected about the user, and can identify various characteristics of the user 210. For example, the session management service 105 may identify that the user 210 has initiated a session corresponding to the management of prediabetes, and can update the user profile 180 to conclude that the user has prediabetes. Demographic information about the user may influence the selection probability 220 of variant 225.

[0056] In some embodiments, the variant selection unit 160 may use the user profile 180 to calculate, generate, or determine prior probabilities of selection in the selection model 205 for each variant 225 of message 185. For example, the variant selection unit 160 may use the user profile 180 to create subgroups of users with similar demographics. The variant selection unit 160 can use these subgroups to generate more accurate selection probabilities for each group of users. For example, the variant selection unit 160 may define classifications based on specific user characteristics (e.g., weight, height, health status, or any combination thereof) to determine commonalities among preferred variants 225 of a group. This definition allows the variant selection unit 160 to identify the selection model 205 for all variants 225 of that user group. Based on the combination of selection models 205 for all variants 225 of that user group, the variant selection unit 160 can establish a selection model 205 for user 210. The combined selection model 205 may have a prior selection probability biased towards at least one variant 225, based on selection models 205 from other users in the same group. In this way, the variant selection unit 160 can present variant 225 that is more likely to result in user 210 engagement with message 185.

[0057] The variant selection unit 160 can identify or otherwise select at least one variant 225 from the set of variants 225 of the message 185 to be transmitted. The selection is made during the first period of the session (for example, time T). AThe selection model 205 may be followed. For each variant 225, the variant selection unit 160 can calculate, determine, or identify the corresponding selection probability 220 as described herein. The variant selection unit 160 can continuously update the selection model 205, or it can update the selection model 205 periodically or in response to new data (e.g., user responses, changes in user demographics). The variant selection unit 160 can select a variant 225 of message 185 to send according to the variant 225 with the highest selection probability 220. In this way, variants 225 that are likely to encourage user engagement can be presented to the user 210 (via the application 120 on the user device 110).

[0058] Following the selection of variant 225, the session management unit 155 may generate at least one instruction 125 to send to the user device 110. The instruction may include the selected variant 225 of message 185. Instruction 215 may be code, data packets, or controls for presenting the message to the user 210 with at least one variant 225. Instruction 215 may include processing instructions for displaying the message on the application 120. Instruction 215 may also include instructions that the user 210 executes regarding their own session. For example, instruction 215 may display message 185 instructing the user to administer medication related to their session. Once generated, the session management unit 155 can send instruction 215 to the user device 210.

[0059] An application 120 on a user device 110 may retrieve, identify, or otherwise receive an instruction 215 from the session management service 105 that identifies a selected variant 225 of message 185. Upon receipt, the application 120 may display message 185 via the user interface 145. The user interface 145 may display variant 225A of message 185A in a different form than variant 225B of the same message 185A. For example, variant 225 may refer to a different display or presentation of the same message. For example, variant 225 may be a different layout of the UI element 150 displayed in the user interface 145. Message 185 may include or display prompts, buttons, text boxes, links (e.g., URLs), or other means that cause user 210 to interact with or engage with message 185. For example, a presentation of message 185 may include a button that affirms that user 210 has performed a task for their session. Engagement with the prompt in message 185 may occur in dialogue 305. Dialogue 305 may indicate that user 210 has engaged with variant 225 of message 185.

[0060] Next, referring to Figure 3, a block diagram of process 300, which receives a response to a message during the first period of a session in a system that controls transmission, is shown. Process 300 is during the first period of a session (for example, time T). AThe process may include, or correspond to, an operation to collect user responses related to the presentation of a variant of the message. In process 300, an application 120 running on the user device 110 may monitor at least one interaction 305 with the user interface 145. The interaction 305 may include a key press, touchscreen, click, or any other event related to the presentation of a message 185 via the user interface 145. In some embodiments, the interaction 305 may occur after the presentation of a variant 225 of the message 185. For example, the interaction 305 may correspond to a set of user interactions logging the completion of a routine after the presentation of a variant 225 of the message 185 prompting the user 210 to perform a specified routine. In some embodiments, the instruction 305 may occur simultaneously with the presentation of the message 185 via the user interface 145.

[0061] Upon detecting interaction 305, application 120 may create or generate at least one response 315 for transmission to session management service 105. Response 315 may identify or include information regarding user task completion, responses by user 210, or general engagement with message 185 or the session. In some embodiments, application 120 may maintain a timer to track the elapsed time since the presentation of message 185. Application 120 may compare the elapsed time to a time limit for variant 225 of message 185. If the elapsed time exceeds the time limit, application 120 may generate a response 315 indicating no user interaction with variant 225 of message 185. Once generated, application 120 may supply, transmit, or otherwise send the response 315 to session management service 105.

[0062] The response unit 165, which operates in the session management service 105, can retrieve, identify, or otherwise receive a response 315 from the user device 110. Upon receipt, the response unit 165 can analyze the response 315 to extract or identify the user 210's interaction 305 with the presented variant 225 of message 185. The response unit 165 can record or log the data from the response 315 in the database 175. The response unit 165 can store the association between the response 315 and the user profile 180. In some embodiments, the response unit 165 can store and maintain the identification information of the session variant 225 in the database 175. The response unit 165 can store the association between the response 315 and the user 210's variant 225.

[0063] The performance calculation unit 170, which runs on the session management service 105, can calculate, generate, or otherwise determine a performance evaluation metric 310 for variant 225 of message 185. The performance evaluation metric 310 may be a value (inquiry, a numerical value) that identifies or corresponds to the user 210's interaction 305 with the presented variant 225 of message 185 on the user interface 145. The performance evaluation metric 310 may be assigned or set to a certain value if the presented variant 225 elicits an interaction 305, and assigned or set to a different value if the variant 225 does not elicit an interaction 305. Generally, the more interaction there is with variant 225 of message 185, the higher the performance evaluation metric 310 is likely to be, while the less interaction there is with the presented variant 225 of message 185, the lower the performance evaluation metric 310 is likely to be. In some embodiments, the performance evaluation metric 310 may correspond to the percentage of accurate or appropriate responses by the user 210 to variants 225 of message 185. The percentage of accurate or appropriate responses may correlate with adherence to a session to address a state. The performance calculation unit 170 can analyze information about responses 315 and variants 225 and perform statistical calculations. For example, the performance calculation unit can calculate the variance of the frequency of responses 315, the relationship between a particular variant 225 and a particular response 315, or the relationship between the user profile 180 and variants 225.

[0064] In some embodiments, the performance evaluation metric 310 may correlate with the variance of responses 315 associated with a given user 210 variant 225. For example, the performance calculation unit 170 can calculate the variance between variants 225 that prompted an interaction 305 from user 210 and variants 225 that did not prompt an interaction 305 from user 210. The performance calculation unit 170 calculates the variance over time T AThe performance evaluation metric 310 for each variant 225 of message 185 can also be calculated using the collected responses 315 and the corresponding user profiles 180. In this way, the performance evaluation metric 310 can relate to the degree of user 210's interactivity with message 185 for the corresponding variant 225.

[0065] Using the performance evaluation scale 310, the performance calculation unit 170 can modify or change the selection model 205. As described above, the selection model 205 can track, in particular, the selection probability for each variant 225, the performance evaluation scale for each variant 225, and the statistical variance of performance for each variant 225. The performance calculation unit 170 can update the selection model 205 to change the selection probability 220 of variant 225A based on the performance evaluation scale 310. For example, if user 210 does not consistently respond to variant 225A, the performance calculation unit 170 may assign a lower selection probability 220 to the presented variant 225. Conversely, if user 210 responds to variant 225, the performance calculation unit 170 may assign a higher selection probability 220. In either scenario, by adding responses 315, the performance calculation unit 170 can reduce the statistical variability of variant 225 in the selection model 205.

[0066] The performance calculation unit 170 can modify or update the user profile 180 to identify or include performance evaluation metrics 310, selection probabilities 220, or other statistics related to the user 210 represented by the user profile 180. The performance calculation unit 170 can maintain information about the user 210 on the user profile 180 in the database 175. The user profile 180 may identify user behavior endpoints, logs of user-related responses 315, or a set of performance evaluation metrics 310 related to the user 210 for variants 225 presented to the user 210. The performance calculation unit 170 can update the selection model 205 based on the information in the user profile 180, the performance evaluation metrics 310, or a combination thereof.

[0067] By updating the selection model 205, the performance calculation unit 170 can determine whether the first period T of the session has elapsed. Period T A can be determined. Period T A may correspond to a part of the session in which any variant 225 of the message 185 can be selected for transmission to the user 210. During this time, the session management service 105 can continue to collect the response 315 from the user device 110, which is used for the selection of one variant 225 of the message 185 for providing to at least a part of the remaining session. Period T A The elapse of may, in particular, correspond to at least one variance in the selection model 205 being less than a threshold, a threshold number of responses 315 received by the session management service 105, or a set limit time.

[0068] In some embodiments, the performance calculation unit 170 can determine whether the limit time of the first period of the session has elapsed. The limit time can be defined relative to the start of the session and can particularly be in the range of several seconds (e.g., 3 seconds) to several weeks (e.g., 10 weeks). When the limit time elapses, the performance calculation unit 170 can determine that the first period T of the session has elapsed. Otherwise, if the limit time has not yet elapsed in the session, the performance calculation unit 170 can determine that the first period T of the session has not elapsed. A has elapsed. Otherwise, if the limit time has not yet elapsed in the session, the performance calculation unit 170 can determine that the first period T of the session has not elapsed. A has not elapsed.

[0069] In some embodiments, the performance calculation unit 170 can determine whether at least one statistical variance of the corresponding variant 255 of the message 185 of the selection model 205 is less than a threshold. The threshold can clarify or specify a value of the statistical variance that can obtain a sufficient reliability for the selection of the variant 225 for at least a part of the remaining session. If the variance is less than the threshold, the performance calculation unit 170 can determine that the first period T of the session has elapsed. Otherwise, if the variance exceeds the threshold, the performance calculation unit 170 can determine that the first period T of the session has not elapsed. A has elapsed. Otherwise, if the variance exceeds the threshold, the performance calculation unit 170 can determine that the first period T of the session has not elapsed. AIt can be determined that the time has not elapsed. The session management service 105 may then repeat processes 200 and 300 as described herein.

[0070] In some embodiments, the performance calculation unit 170 may determine whether the number of responses 315 from the user 210 exceeds a threshold. The threshold may clarify or specify the number of responses 315 that provides sufficient confidence in selecting variant 225 for at least the remainder of the session. If the number of responses is equal to or greater than the threshold, the performance calculation unit 170 determines whether the number of responses exceeds a threshold for the first period T of the session. A It can be determined that the first period T of the session has elapsed. If not, and the number of responses is less than the threshold, the performance calculation unit 170 determines that the first period T of the session has elapsed. A It can be determined that the time has not elapsed. The session management service 105 may then repeat processes 200 and 300 as described herein.

[0071] Following the determination of the completion of the first period, the second period of the session T B This may begin. In some embodiments, following the elapsed first period, the session management service 105 logs the response, calculates the distribution, or T A Other operations described in relation to this can be stopped. In some embodiments, the elapsed first period may indicate that enough data has been taken up by the selection model 205 for the session management service 105 to select a variant 225 that is likely to elicit a response from user 210.

[0072] Next, referring to Figure 4, a block diagram of process 400 in the system that controls transmission selects and transmits a message variant during the second period of the session. Process 400 is the second period of the session (period T). BProcess 400 may include operations corresponding to sending messages and receiving responses within the process. Process 400 may include one or more operations described with respect to process 200 or 300. In process 400, the session management unit 155 may continue the session (which is then in the second period). In the second period of the session, the session management unit 155 may identify or select at least one message 185' to send to the user 210. In some embodiments, message 185' may be the same as message 185 from the first period of the session. In some embodiments, message 185' may be different from message 185 sent during the first period of the session.

[0073] In some embodiments, the session management unit 155 may identify or select another set of messages 185' to send to the user 210 for at least a portion of the second period of the session. The selection model 205 is first established during the first period T. A Subsequently, the session management unit 155 may identify or select a second message set 185'. The second message set 185' may be for the same session established to address the state of user 210. The second message set 185' may be related to the first message set 185 used in the first period. The relationships between message sets can be defined according to the session specifications. For example, the second message set 185' may be for period T A This may be for a subsequent routine or activity prompted by message 185. In this example, T A The message could be about performing breathing exercises, B The message can be related to the motion of running.

[0074] Each message 185' may have a set of variants 225'A-N, as well as the set of variants 225 described above. Each variant 225' of message 185' may convey or include at least common parts of the information of message 185', in particular with all variants 225' having different action types, themes, content types, inclusion of user information (e.g., name), or layouts. Each variant 225' of message 185' may correspond to one of the variants 225 of message 185. For example, variants 225' and variant 225 may correspond in particular to the same type of action, theme, content type, presence or absence of user information, or layout.

[0075] By identification, the variant selection unit 160 can identify or select a variant 225' of message 185' using the selection model 205. The variant selection unit 160 may select a variant 225' of message 185' that has been determined to have the highest selection probability 220' from the first period of the session. In some embodiments, the variant selection unit 160 can select a variant 225 of message 185' that is different from the message 185 selected during the first period using the selection model 205. To make a selection, the variant selection unit 160 may identify the variant 225 having the highest selection probability 220' from the selection model 205. By identification, the variant selection unit 160 can find, determine, or otherwise select a variant 225' of message 185' that has a correspondence with the variant 225 of message 185. For example, the variant selection unit 160 may select a variant 225' of message 185' that has the same layout as the variant 225 of message 185.

[0076] In some embodiments, the variant selection unit 160 may identify or select a variant 225' of message 185' that is different from the highest selection probability 220' obtained using the selection model 205 from the first period. The selection probability 220' of the performance evaluation metric 310 of variant 225 is obtained from the first period T A From the second period TB This could change. This change could be due to a variety of reasons, including changes in user 210 preferences, changes in user status or environment, changes in one or more variants 225 (such as the addition or deletion of further variants 225), or changes in user device 110 that cause a different presentation of variant 225 to the user (for example, the user purchasing a different mobile device for displaying application 120).

[0077] To make a determination, the variant selection unit 160 may determine that the selection probability 220' in the second period differs from the selection probability 200 in the first period by a threshold margin. The threshold margin can identify or define the value of the difference between the selection probability 220 and the selection probability 220' in which the user 210's preference deviates. In some embodiments, a performance evaluation metric from the user 210 can be used instead of or in addition to the selection probability. If the difference is greater than the threshold margin, the variant selection unit 160 may select a variant 225' different from the initially selected variant 225. If, however, the difference is less than the threshold margin, the variant selection unit 160 may retain the initially selected variant 225.

[0078] Depending on the selection, the session management unit 155 can generate at least one instruction 215' to send to the user device 110. The instruction may include a selected variant 225' of message 185'. Instruction 215' may be code, data packets, or controls for presenting the message to the user 210 in at least one of the variants 225'. Instruction 215' may include processing instructions for displaying the message on the application 120. Instruction 215' may also include instructions for the user 210 to perform in relation to their own message. For example, instruction 215' may display message 185' instructing the user to perform a specific activity related to their session. Once generated, the session management unit 155 can send instruction 215' to the user device 210.

[0079] An application 120 on user device 110 may retrieve, identify, or otherwise receive an instruction 215' that identifies a selected variant 225' of message 185' from session management service 105. Upon receipt, application 120 may display message 185' via user interface 145. Following the display, application 120 may monitor at least one interaction 305' with user interface 145. In some embodiments, the interaction may occur after the display of variant 225' of message 185'. For example, interaction 305' may correspond to a set of user interactions logging the completion of a routine after the display of variant 225' of message 185' prompting user 210 to perform a specified routine. In some embodiments, the instruction 305' may occur simultaneously with the display of message 185' via user interface 145.

[0080] Upon detecting interaction 305', application 120 may create or generate at least one response 315' for transmission to session management service 105. Response 315' may identify or include information regarding user task completion, responses by user 210, or general engagement with message 185' or the session. In some embodiments, application 120 may maintain a timer to track the elapsed time since the presentation of message 185'. Application 120 may compare the elapsed time to a time limit for variant 225' of message 185'. If the elapsed time exceeds the time limit, application 120 may generate a response 315' indicating no user interaction with variant 225' of message 185'. Once generated, application 120 may supply, transmit, or otherwise send the response 315 to session management service 105.

[0081] The response unit 165 can retrieve, identify, or otherwise receive a response 315' from the user device 110. Upon receipt, the response unit 165 can parse the response 315' to extract or identify the user 210's interaction 305 with the presented variant 225' of message 185'. The response unit 165 can record or log the data from the response 315' in the database 175. The response unit 165 can store the association between the response 315' and the user profile 180. In some embodiments, the response unit 165 can store and maintain the identification of the session variant 225' in the database 175. The response unit 165 can store the association between the response 315 and the user 210's variant 225'.

[0082] In some embodiments, the performance calculation unit 170 can calculate, generate, or otherwise determine a performance evaluation metric 310' for a variant 225' of message 185'. The performance evaluation metric 310' may be a value (inquiry, a numerical value) that identifies or corresponds to the user 210's interaction 305 with the presented variant 225' of message 185' on the user interface 145. The performance evaluation metric 310' may be assigned or set to a certain value if the presented variant 225' elicits an interaction 305, and assigned or set to a different value if the variant 225' does not elicit an interaction 305. Generally, the more interaction there is with the variant 225' of message 185', the higher the performance evaluation metric 310' may be, while the less interaction there is with the presented variant 225' of message 185', the lower the performance evaluation metric 310' may be. In some embodiments, the performance evaluation metric 310' may correspond to the percentage of accurate or appropriate responses by the user 210 to variant 225' of message 185'. The performance calculation unit 170 can analyze information regarding responses 315' and variant 225' and perform statistical calculations. For example, the performance calculation unit can calculate the variance of the frequency of responses 315', the relationship between a particular variant 225' and a particular response 315', or the relationship between the user profile 180 and variant 225'.

[0083] Using the performance evaluation metric 310', the performance calculation unit 170 can modify or change the selection model 205. As described above, the selection model 205 can track, in particular, the selection probability for each variant 225', the performance evaluation metric for each variant 225', and the statistical variance of performance for each variant 225'. The performance calculation unit 170 can update the selection model 205 to change the selection probability 220' of variant 225'A based on the performance evaluation metric 310'. Furthermore, the performance calculation unit 170 can modify or update the user profile 180 to identify or include the performance evaluation metric 310', the selection probability 220, or other statistics related to user 210 represented by the user profile 180. The performance calculation unit 170 can maintain information about user 210 on the user profile 180 in the database 175. The user profile 180 may identify user behavior endpoints, logs of user-related responses 315, or a set of performance metrics 310' related to user 210 for variants 225' presented to user 210. The performance calculation unit 170 can update the selection model 205 based on the information in the user profile 180, the performance metrics 310', or a combination thereof. In some embodiments, the performance calculation unit 170 does not need to perform the calculation of the performance metrics 310' during the second period of the session.

[0084] In some embodiments, the developed selection model 205 may no longer suit the user 210's preferences. For example, user 210 may provide a response 315' that deviates from the selection model's estimates or exceeds a threshold variance within the selection model 205. Therefore, the selection model 205 may need to be modified or changed. The selection model 205 may need to be modified in response to the statistical variability or variance exceeding a threshold amount as a result of a change in the variant 225 selected by user 210. The performance calculation unit 170 performs the second period T BThe system may determine that the performance evaluation metric 310 of an internal variant exceeds a threshold. For example, the variance of response 315 to message 185 may exceed the threshold. The selection model 205 may need to be edited as a result of changes in user input. For example, the user may add or remove data about their demographic information that may affect the selection probability 220 of variant 225. For example, the user may input data for a period T. A It is possible that the pre-diabetic condition was not included in the list, but then, during period T... B The prediabetic state is then added to the list via the user interface 145. Subsequently, the performance calculation unit 170 can recalculate the performance evaluation scale 310 associated with one or more variants 225.

[0085] In some embodiments, the performance calculation unit 170 may provide a dashboard interface for presentation. The dashboard interface may be presented on the display of the session management service 105 or a computing device communicatively coupled to the session management service 105. The dashboard interface may provide a presentation of one or more statistics about the session based on responses received from the user interface 110 related to the user 210. For example, the dashboard interface may display or present, in particular, individual responses 315 from the user 210, performance evaluation metrics 310 related to each variant 225 of message 185, selection probabilities and statistical variances shown by the selection model 205, and information related to the user 210. The dashboard interface may display different time periods (e.g., T A or T B The dashboard interface may show the changes in the selected model 205. The dashboard interface may appear as a graphical user device (e.g., GUI), a table, a series of graphs, or another display medium on the server. The dashboard interface can change dynamically as responses 315 are collected by the response response unit 165.

[0086] In this way, the session management service 105 can control the transmission of messages in the network environment. By updating the selection model 205 with response data during the first period, the session management service 105 may be able to identify a variant 225 of message 185 that has the best performance evaluation metric for a particular user 210. In doing so, the session management service 105 can reduce the computational resources across the system 100 from providing the variant 225 of message 185 that is most likely to encourage engagement with or by the user 210. Furthermore, the session management service 105 can reduce the overall wasted transmission of message 185 instructions 215 that would not have elicited interaction from the user 210. By selecting a variant 225 of message 185 in this way, the session management service 105 can also improve the quality of human-computer interaction (HCI) between the user 210 and the user interface 145 of the application 120 on the user device 110.

[0087] Next, referring to Figure 5, an example of a set of graphs of prior probabilities for message variants is shown. These illustrated examples can relate to the iterative development of a selection model 205, where each variant 225 in the set of variants has an equal selection probability 220. When a response 315 is received from user 210, the prior probability (e.g., selection probability 220) of each variant changes over a period T A Once this process is complete, the distribution can converge to its final form.

[0088] In the illustrated example, variants 225A and 225B can be represented by the plots shown in Figure 5. Each variant corresponds to a period T AIt can be sent to user 210 over the course of the process. In this example, variant 225A may have a higher engagement rate with user 210 than variant 225B, with an actual interaction rate of 30%, while variant 225B may have an interaction rate of approximately 20%. Initially, there may be no reason to think that one variant is more attractive than the other, so each user in the set of users has an equal chance of receiving either variant.

[0089] As the response handling unit 165 collects more responses 315, it may come to believe that variant 225A has a slightly higher interaction rate. Therefore, the session management service 105 can associate variant 225A with a higher probability (i.e., more frequently than variant 225B). However, the session management unit 155 does not have to stop sending variant 225B until the system is statistically convinced that variant 225A is preferable. In this way, the more attractive message can be prioritized.

[0090] Referring next to Figures 6A and 6B, a method 600 is shown that predicts a variant of a message to send to a client based on the client's response. Each figure illustrates an operation performed by the client and an operation performed by the service. Method 600 may be implemented or performed using any of the components detailed herein, such as the session management service 105 and the user device 110. Starting with Figure 6A, in method 600, a service (e.g., the session management service 105) may identify a user profile (e.g., user profile 180) (605). The service may establish a model (e.g., selection model 205) of variants (e.g., a set of variants 255) of a message (e.g., message 185) (610). The service may select a variant (e.g., variant 225) according to a prior probability (e.g., selection probability 220) ((615)). The service may send an instruction (e.g., instruction 215) regarding a variant of the message to a user device (e.g., user device 110) (620). The user device may receive the instruction (e.g., instruction 215) (625). The user device may present the variant of the message identified in the instruction (630). The user device may send a response (e.g., response 315) to the service (635). The service may receive a response from the user device (640). The service may obtain a performance evaluation metric (e.g., performance evaluation metric 310) (645). The service may update the model (650).

[0091] Moving to Figure 6B, the service may check whether there is sufficient data (655). The decision may be based on the amount of time elapsed in the session or the degree of variance in the variant model. If there is not enough data, the service may return to (615) and repeat method 600. If there is sufficient data, the service may select the variant with the best performance (660). The service may send an instruction (e.g., instruction 215') containing the selected variant of the message (665). The client may receive an instruction from the service (670). The user device may present a variant of the message (675). The user device may send a response (e.g., response 315') to the server (680). The service may receive a response from the user device (685). The service may receive a response from the user device (685). The service may check whether the session has completed (690). If the session has not completed, the service may return to (660) and repeat method 600. If the session has completed, the service may terminate the session (695).

[0092] B. Network and computing environment The various operations described herein can be performed on a computer system. Figure 7 shows a simplified block diagram of a typical server system 700, a client computer system 714, and a network 726 that can be used to implement a particular embodiment of this disclosure. In various embodiments, the server system 700 or a similar system can implement the services or servers or parts thereof described herein. The client computer system 714 or a similar system can implement the clients described herein. System 100 described herein may be similar to the server system 700. The server system 700 may have a modular design incorporating a plurality of modules 702 (e.g., blades in a blade server embodiment). Two modules 702 are shown, but any number can be provided. Each module 702 may include one or more processing units 704 and local storage 706.

[0093] A processing unit (one or more) 704 may include a single processor or multiple processors, each containing one or more cores. In some embodiments, a processing unit (one or more) 704 may include a general-purpose primary processor and one or more dedicated coprocessors, such as a graphics processor or a digital signal processor. In some embodiments, some or all of the processing units 704 may be implemented using customized circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored in the circuit itself. In other embodiments, a processing unit (one or more) 704 may execute instructions stored in local storage 706. Any combination of any type of processors may be included in a processing unit (one or more) 704.

[0094] The local storage 706 may include volatile storage media (e.g., DRAM, SRAM, SDRAM, etc.) and / or non-volatile storage media (e.g., magnetic or optical disks, flash memory, etc.). The storage media incorporated into the local storage 706 may be fixed, removable, or upgradeable as desired. The local storage 706 may be physically or logically divided into various subunits such as system memory, read-only memory (ROM), and permanent storage. The system memory may be volatile read-write memory such as a read-write memory device or dynamic random-access memory. The system memory may store some or all of the instructions and data required by the processing unit(s) 704 at runtime. The ROM may store static data and instructions required by the processing unit(s) 704. The permanent storage may be a non-volatile read-write memory device that can store instructions and data even when the module 702 is powered down. As used herein, the term "storage medium" includes a medium capable of storing data indefinitely (subject to overwriting, electrical disturbances, power loss, etc.), but does not include carrier waves or transient electronic signals propagated wirelessly or via wired connections.

[0095] In some embodiments, the local storage 706 can store one or more software programs executed by the processing unit(s) 704, such as programs that implement various server functions, such as functions of the operating system and / or system 100 or any other system described herein, or any other server(s) associated with system 100 or any other system described herein.

[0096] "Software" generally refers to a sequence of instructions that, when executed by a processing unit (one or more) 704, cause the server system 700 (or a part thereof) to perform various operations, and thus defines one or more specific machine embodiments that execute and carry out the operations of the software program. Instructions can be stored as firmware residing in read-only memory and / or as program code stored in a non-volatile storage medium that can be loaded into volatile working memory for execution by the processing unit (one or more) 704. Software can be implemented as a single program or as a collection of separate programs or program modules that interact as desired. From local storage 706 (or non-local storage as described below), the processing unit (one or more) 704 can retrieve program instructions to execute and data to process in order to perform the various operations described above.

[0097] Depending on the server system 700, multiple modules 702 can be interconnected via a bus or other interconnect 708 to form a local area network that facilitates communication between the modules 702 and other components of the server system 700. The interconnect 708 can be implemented using various technologies such as server racks, hubs, and routers.

[0098] The wide area network (WAN) interface 710 can provide data communication capabilities between the local area network (e.g., via interconnect 708) and a network 726 such as the Internet. The server system can be connected to the network 726 in a communicative manner using other technologies, including wired technology (e.g., Ethernet, IEEE 802.3 standard) and / or wireless technology (e.g., Wi-Fi, IEEE 802.11 standard).

[0099] In some embodiments, local storage 706 is intended to provide working memory to processing unit(s) 704 to accelerate access to programs and / or data being processed while reducing traffic on interconnect 708. Storage for larger amounts of data can be provided on the local area network by one or more mass storage subsystems 712 that can be connected to interconnect 708. The mass storage subsystem 712 may be based on magnetic, optical, semiconductor, or other data storage media. Direct-attached storage, storage area networks, network-attached storage, etc., can be used. Any data store or other collection of data described herein as being generated, consumed, or maintained by a service or server can be stored in the mass storage subsystem 712. In some embodiments, additional data storage resources may be accessible via WAN interface 710 (although this may increase latency).

[0100] System 700 can operate in response to requests received via the WAN interface 710. For example, one of the modules 702 can implement a monitoring function and assign individual tasks to other modules 702 in response to received requests. Work allocation techniques can be used. Once a request is processed, the results can be returned to the requester via the WAN interface 710. Such operations can generally be automated. Furthermore, in some embodiments, the WAN interface 710 can interconnect multiple server systems 700 to provide a scalable system capable of managing large volumes of activity. Other techniques for managing server systems and server farms (collections of cooperating server systems), including dynamic resource allocation and reallocation, can be used.

[0101] The server system 700 can interact with devices owned or operated by various users via a wide area network such as the Internet. An example of a user-operated device is shown in Figure 7 as a client computing system 714. The client computing system 714 can be implemented as a consumer device such as a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smartwatch, glasses), desktop computer, or laptop computer.

[0102] For example, the client computing system 714 can communicate via the WAN interface 710. The client computing system 714 may include computer components such as processing units (one or more) 716, storage devices 718, network interfaces 720, user input devices 722, and user output devices 724. The client computing system 714 can be implemented in various form factors, such as desktop computers, laptop computers, tablet computers, smartphones, other mobile computing devices, and wearable computing devices.

[0103] The processing unit 716 and storage device 718 may be similar to the processing unit(s) 704 and local storage 706 described above. Appropriate devices can be selected based on the requirements imposed on the client computing system 714; for example, the client computing system 714 may be implemented as a "thin" client with limited processing power, or as a high-power computing device. The client computing system 714 may include program code executable by the processing unit(s) 716 to enable various interactions with the server system 700.

[0104] The network interface 720 can provide a connection to a network 726, such as a wide area network (e.g., the Internet), to which the WAN interface 720 of the server system 700 is also connected. In various embodiments, the network interface 720 may include a wired interface (e.g., Ethernet) and / or a wireless interface implementing various RF data communication standards such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, LTE, etc.).

[0105] The user input device 722 may include any device (or a number of devices) on which the user can supply signals to the client computing system 714. The client computing system 714 can interpret the signals as indicating a specific user request or information. In various embodiments, the user input device 722 may include any or all of the following: a keyboard, touchpad, touchscreen, mouse, or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, etc.

[0106] The user output device 724 may include any device on which the client computing system 714 can supply information to the user. For example, the user output device 724 may include inter-display images generated by or delivered to the client computing system 714. The display may incorporate various image generation technologies, such as liquid crystal displays (LEDs), light-emitting diode (LED) displays including organic light-emitting diodes (OLEDs), projection systems, cathode ray tubes (CRTs), etc., together with supporting electronic equipment (e.g., digital-to-analog converters or analog-to-digital converters, signal processors, etc.). Depending on the embodiment, it may include a device such as a touchscreen that functions as both an input and output device. In some embodiments, other user output devices 724 may be provided in addition to or instead of the display. Examples include indicator lights, speakers, haptic "display" devices, printers, etc.

[0107] In some embodiments, the system includes electronic components such as a microprocessor, storage, and memory that store computer program instructions in a computer-readable storage medium. Many of the features described herein can be implemented as a process, which is specified as a set of program instructions encoded in a computer-readable storage medium. When these program instructions are executed by one or more processing units, they cause the processing unit(s) to perform the various operations indicated by the program instructions. Examples of program instructions or computer code include machine code, such as that generated by a compiler, and files containing higher-level code that is executed by a computer, electronic component, or microprocessor using an interpreter. With appropriate programming, the processing units(s) 704 and 716 can provide the server system and client computing system 714 with a variety of functionalities, including one or more of the functionalities described herein, which are executed by a server or client.

[0108] It should be understood that the server system 700 and client computing system 714 are illustrative and are subject to modification and alteration. Computer systems used in connection with embodiments of this disclosure may have other capabilities not specifically described herein. Furthermore, while the server system 700 and client computing system 714 are described with reference to specific blocks, it should be understood that these blocks are defined for convenience and are not intended to imply a specific physical arrangement of components. For example, different blocks may, but are not required to, be located in the same facility, in the same server rack, or on the same motherboard. Moreover, blocks do not need to correspond to physically separate components. Blocks can be configured to perform various operations, for example, by programming a processor or providing appropriate control circuits, and depending on how the initial configuration is obtained, the various blocks may or may not be reconfigurable. Embodiments of this disclosure can be realized in various devices, including electronic devices implemented using any combination of circuitry and software.

[0109] While this disclosure has described specific embodiments, it will be recognized by those skilled in the art that numerous modifications are possible. Embodiments of this disclosure can be implemented using a variety of computer systems and communication technologies, including but not limited to the specific examples described herein. Embodiments of this disclosure can be implemented using any combination of dedicated components and / or programmable processors and / or other programmable devices. Various processes described herein can be implemented on the same processor or different processors in any combination. Where a component is described as being configured to perform a particular operation, such configuration can be achieved, for example, by designing an electronic circuit to perform that operation, by programming a programmable electronic circuit (such as a microprocessor) to perform that operation, or by any combination thereof. Furthermore, while the embodiments described above may refer to specific hardware and software components, it will be understood by those skilled in the art that different combinations of hardware and / or software components may be used, and that certain operations described as being implemented in hardware may also be implemented in software, and vice versa.

[0110] Computer programs incorporating the various features of this disclosure can be encoded or stored on a variety of computer-readable storage media, including, but not limited to, magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, and other non-temporary media. The computer-readable storage media encoded with the program code may be packaged with a compatible electronic device, or the program code may be provided separately from the electronic device (for example, by internet download or as a computer-readable storage medium in a separate package).

[0111] Therefore, although this disclosure has been described in relation to specific embodiments, it will be understood that this disclosure covers all modifications and equivalents within the scope of the attached claims.

Claims

1. A method for controlling the sending of messages to users, The server sends a number of first commands to the user device associated with the user, each of which causes the user to present a message in at least one of several variants, during the first period of the user's session toward achieving the behavioral endpoint. The steps include: the server receiving a plurality of responses from the user device during the first period corresponding to the plurality of first commands, each of which is a user response to each presentation of the message in at least one of the plurality of variants toward achieving the action endpoint; The server performs the steps of determining each performance evaluation metric for each variant of the message in each of the multiple variants using the multiple responses, in accordance with the elapsed time of the first period of the session. The server selects one variant from the plurality of variants of the message for a second period of the session based on the performance evaluation metric of the variant. The server sends a second command to the user device associated with the user during the second period, causing the user device to present the message with the selected variant. A method that includes this.

2. In the method according to claim 1, The server determines that the second performance evaluation metric for the message in the selected variant during the second period differs from each performance metric during the first period by a threshold margin. The server, in response to its determination that the second performance evaluation metric is different, selects a second variant from the plurality of variants of the message to be transmitted. A method that further includes this.

3. A method according to claim 1, further comprising the step of the server selecting one of a plurality of second variants of a second message to send for a third period of the session, based on the selection of the variants of the message for the second period.

4. A method according to claim 1, further comprising the step of the server establishing a model that tracks, for each variant of the plurality of variants, (i) a performance metric that identifies the degree of user interactivity with the message of the corresponding variant, and (ii) the variance of the performance metric.

5. A method according to claim 1, further comprising the step of the server determining, in response to a determination that the variance of the performance evaluation metric is less than a threshold, the selection of one of the plurality of variants of the message for the second period.

6. A method according to claim 1, further comprising the step of the server determining a plurality of prior probabilities for selecting a plurality of corresponding variants of the message to be sent during the first period of the session, based on the user's profile.

7. A method according to claim 1, further comprising the step of the server providing a dashboard interface for presentations that identify one or more statistics of the session based on the plurality of responses from the user device relating to the user.

8. A method according to claim 1, further comprising the step of determining each performance metric, using the plurality of responses and the user profile that identifies information for the session, to determine each performance metric for the message in each variant of the plurality of variants.

9. The method according to claim 1, wherein each variant of the plurality of variants of the message includes at least one of the following: (i) identification of the type of action performed by the user via the user device; (ii) identification of the theme of the content presented via the user device; (iii) identification of the type of content presented via the user device; (iv) identification of the inclusion of user information; (v) definition of each layout of the content; (vi) identification corresponding to the personality associated with the user; (v) identification of the tone of the content presented via the user device; (vi) identification of content based on the user's literacy; (vii) identification of content based on the user's health knowledge; (viiii) identification of a learning style associated with the user; (ix) identification of the user's personal extrinsic or intrinsic motivations; (x) identification based on the user's messaging style; (xi) identification of the user's values; or (xii) the user's cognitive abilities.

10. A method according to claim 1, further comprising the step of the server maintaining a profile in a database that identifies (i) action endpoints to be achieved, (ii) logs relating to the plurality of responses from the user, and (iii) one or more performance metrics of the plurality of variants of the message.

11. The method according to claim 1, wherein the user is being administered a drug related to addressing the behavioral endpoint at least partially concurrently with the session.

12. A system that controls the sending of messages to users, A server having one or more processors coupled to memory, During the first period of a user session aimed at achieving the behavioral endpoint, a plurality of first commands are sent to the user device associated with the user, each command causing the user to present a message in at least one of a plurality of variants. During the first period, the user device receives a plurality of responses corresponding to the plurality of first commands, each of which is a user response to each presentation of the message in at least one of the plurality of variants toward achieving the behavioral endpoint. In accordance with the progression of the first period of the session, the performance evaluation metrics for each variant of the message in each of the multiple variants are determined using the multiple responses. For the second period of the session, one variant of the message is selected from the plurality of variants based on the performance evaluation metric of the variant, and During the second period, the second command causing the message to be presented in the selected variant is sent to the user device associated with the user. A system comprising at least one server configured in such a way.

13. In the system according to claim 12, the at least one server is It is determined that the second performance evaluation metric for the message in the selected variant during the second period differs from each performance metric during the first period by a threshold margin, and In response to the determination that the second performance evaluation metric is different, the second variant is selected from the plurality of variants of the message to be transmitted. A system that is further composed of these elements.

14. The system according to claim 12, wherein the at least one server is further configured to select one of a plurality of second variants of a second message to send for a third period of the session, based on the selection of the variant of the message for the second period.

15. The system according to claim 12, wherein the at least one server is further configured to use the plurality of responses to establish a model for each variant of the plurality of variants that tracks (i) a performance metric that identifies the degree of user interactivity with the message of the corresponding variant, and (ii) the variance of the performance metric.

16. The system according to claim 12, wherein the at least one server is further configured to determine the selection of one of the plurality of variants of the message for the second period in response to a determination that the variance of the performance evaluation metric is less than a threshold.

17. The system according to claim 12, wherein the at least one server is further configured to determine a plurality of prior probabilities for selecting a plurality of corresponding variants of the message to be sent during the first period of the session, based on the user's profile.

18. The system according to claim 12, wherein the at least one server is further configured to provide a dashboard interface for presentations that identify one or more statistics of the session based on the plurality of responses from the user device relating to the user.

19. The system according to claim 12, wherein the at least one server is further configured to determine each performance metric for the message in each variant of the plurality of variants using the plurality of responses and the user profile that identifies information for the session.

20. The system according to claim 12, wherein each variant of the plurality of variants of the message includes at least one of the following: (i) identification of the type of action performed by the user via the user device; (ii) identification of the theme of the content presented via the user device; (iii) identification of the type of content presented via the user device; (iv) identification of the inclusion of user information; (v) definition of each layout of the content; (vi) identification corresponding to the personality associated with the user; (v) identification of the tone of the content presented via the user device; (vi) identification of content based on the user's literacy; (vii) identification of content based on the user's health knowledge; (viiii) identification of the learning style associated with the user; (ix) identification of the user's personal extrinsic or intrinsic motivations; (x) identification based on the user's messaging style; (xi) identification of the user's values; or (xii) the user's cognitive abilities.

21. The system according to claim 12, wherein the at least one server is further configured to maintain a profile in a database that identifies (i) action endpoints to be achieved, (ii) logs relating to the plurality of responses from the user, and (iii) one or more performance metrics of the plurality of variants of the message.

22. The system according to claim 12, wherein the user is being administered a drug related to addressing the behavioral endpoint at least partially concurrently with the session.

Images