Automated change system clock to simulate test environment for application triggers generated using machine learning

Abstract

To provide a system and method for simulating a test environment of a digital therapy application for addressing a medical condition over a number of time frames.SOLUTION: The computing system 100 can select a trigger from the plurality of triggers of the event scheduler to address the medical condition and set a clock to a timestamp of the trigger, wherein the clock is accessible to the service and the user device. The computing system can also cause, responsive to the timestamp of the clock matching the timestamp of the trigger, the service to send a message to the user device to address the medical condition, identify that the message is being communicated between the service and the user device in accordance with the trigger, and determine, responsive to identifying that the message has been communicated, that the trigger of the event scheduler has been validated.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and benefit of U.S. Non-Provisional Application No. 18 / 748,002, filed June 19, 2024, entitled "Automatically Varying System Clock to Simulate a Test Environment for Application Triggers Generated Using Machine Learning," the entire contents of which are incorporated herein by reference. [Background technology]

[0002] Prior to deployment to user devices, applications may undergo testing and validation to ensure that the application's functionality operates and performs properly as intended. If one of the design considerations for the application includes communication with a server that hosts the application's resources, the exchange of messages and responses between the application and the server may also be tested and validated over a period of time. Such testing may include evaluation of the application's behavior and functionality, including the interactivity provided by the application to the user. This validation may include evaluation of whether the application meets specified requirements and objectives, such as the successful delivery of messages and responses.

[0003] One technical challenge for testing and validation can arise from the sequence dependencies of application functionality and the complexity of message and response communication, especially over long timeframes. These testing and validation issues can be exacerbated by the type of application and the messages and responses exchanged over the network. For example, for digital therapeutic applications, the timeframes involved in functionality and message and response communication can be on the order of weeks to months. Digital therapeutic applications can directly provide evidence-based interventions (e.g., in the form of content or messages) with the intent of treating, ameliorating, or otherwise addressing the user's condition during this timeframe. Furthermore, the interventions themselves can be complex, involving myriad sequences, branches, or dynamic variations in application functionality and message and response communication between the application and the server.

[0004] Due to this complexity, it can be impractical and difficult to test and validate all functionality in real time, as well as every potential sequence of messages and responses transmitted between the application and the server. As a result, it can be difficult to pinpoint the cause of poor application or communication performance and identify ways to optimize overall system performance. The inability to test and validate all functionality and potential sequences in real time can result in suboptimal application and server performance, which can lead to wasted computing resources (e.g., processors, memory, and network bandwidth). Furthermore, some functionality or communication that was not tested or validated may not function properly, reducing the overall functionality of the system. Malfunctioning applications can also lead to poor quality human-computer interaction (HCI) between users and applications.

[0005] In digital therapeutic applications, testing and validating all functionality and all sequences of messages and responses may require waiting a significant amount of time (e.g., weeks or months) until the end of the intervention. This may ultimately delay the deployment of the digital therapeutic application, potentially requiring even larger timeframes. Furthermore, releasing a digital therapeutic application before full testing and validation may result in some functionality and communications not working properly, as well as suboptimal performance of the entire system (e.g., digital therapeutic application and server). Furthermore, breakdowns in functionality and communications during execution may lead to a degradation of the HCI between the user and the application. In the context of digital therapeutics, the application may become ineffective in addressing the condition or even worsen the condition associated with the user. Summary of the Invention

[0006] Presented herein are systems and methods for simulating an application's testing environment over several time frames. To address these and other technical challenges, a test management service can automatically advance clock time on a digital therapeutic application to trigger message transmission. The test management service can maintain and operate a central clock and insert accurate timestamps from the central clock into the respective clocks on the application and server. This automatic change in clock time can provide several advantages and benefits when testing and validating digital therapeutic applications. For example, by using a centralized clock to advance the system clocks on the server and application, the test management service can reduce the testing and validation of triggers for a digital therapeutic application from weeks or months to minutes or hours. For example, testing and validation that would take 30 to 90 days without automated clock advancement can now take only 10 to 30 minutes. This allows time and effort that would otherwise be spent waiting for testing and validation to complete to be redirected toward optimizing and improving the efficiency and functionality of the digital therapeutic application and the content of its messages. Needless to say, this shortened testing and validation period could allow digital therapeutic applications to be more quickly introduced and deployed to users.

[0007] Second, because of the shorter testing and validation time, test management services may be able to test and validate more aspects of a digital therapeutic application. This ensures that more of the digital therapeutic application's functionality operates as intended and designed, and that more messages (following sequences, branching, or dynamic changes) are communicated properly. Doing so may save computational resources (e.g., processing and memory) and network bandwidth that would otherwise be wasted on improper functionality or inaccurate communication. Furthermore, technical issues that are apparent to users (e.g., issues with content display) may be resolved before deployment, thereby improving the quality of the HCI between the user and the digital therapeutic content. From the perspective of the treatment provided, test management services may more reliably improve the effectiveness of digital therapeutic applications in mitigating, curing, or otherwise addressing a user's condition.

[0008] To that end, automatic changes in clock time progression can enhance the nature of digital therapeutic applications. The delivery of therapeutic interventions in the form of messages may depend on triggers defined in terms of a time schedule (e.g., defined by hours, minutes, seconds, days, and months), among other factors and conditions. For example, the passage of a specific message time can trigger a server to send a message containing digital therapeutic content to an application on a computing device. A challenge here in testing is that time on the application and server may elapse according to the system clocks on the computing system and server. Test management services (e.g., in virtual testing environments) can be configured with a centralized clock that defines the time to which clocks on the server and applications are set. By systematically advancing the system clocks of the server and applications using the centralized clock, the server can be triggered to send these messages at an earlier point in time than the actual time.

[0009] The test management service can identify a set of triggers for the transmission of messages and responses defined by the digital therapeutic application's event scheduler. This set of triggers can be fixed or predefined for the digital therapeutic application, or can be dynamically generated during runtime (e.g., by an artificial intelligence or machine learning model) based on various inputs (e.g., user profile or prior responses) over a set time frame (e.g., one week to six months). Each trigger can specify a timestamp defining the time at which the server sends a specified message containing digital therapeutic content. From each trigger, the test management service can identify the timestamp specified by that trigger and set a centralized clock to the identified timestamp. This setting can cause the centralized clock to skip or fast-forward several hours, days, weeks, or months of real time in fractions of a second. This setting of the centralized clock can also cause the application's or server's clock to be overridden and set to this timestamp (e.g., via a communicated command or injection). As a result, the clocks on the application and server can be synchronized with the test management service's centralized clock.

[0010] With this clock thus set, the server can determine whether a trigger corresponding to the timestamp is activated. This determination can be performed by the server according to a fixed interval (e.g., every 30 seconds to every hour) or on demand to evaluate whether to send a message. For example, every five minutes, the server can use data associated with the computing device or user to check against the trigger criteria. If the criteria are not met and therefore the trigger is not activated, the server may refrain from sending the message. Conversely, if the criteria are met and the trigger is determined to be activated, the server can select a specified message and send the message to the computing device for presentation via the digital therapeutic application. Upon presenting the message, the application can monitor for user interaction or dialogue with the message's user interface elements (e.g., actions taken to address a medical condition). The application can use the data associated with the user interaction to generate a response and send the response to the server.

[0011] Additionally, the test management service may monitor for communications between the computing device and the server. From the monitoring, the test management service can determine whether a message to the user to address the condition was communicated, i.e., communicated, between the server and the computing device as specified by the trigger. Monitoring of a particular message for a trigger can be performed at a time interval used by the server (e.g., between 30 seconds and 1 hour). Based on the trigger and the communication of the message or response, the test management service can determine whether the associated trigger is validated. If the trigger is determined not to have been fired and a message is determined not to have been sent, the test management service can determine that the trigger is validated. If the trigger is determined to have been fired and a message is determined to have been sent, the test management service can determine that the trigger is validated. Conversely, if the trigger is determined to have been fired and a message is determined not to have been sent, the test management service can determine that the trigger is not validated. If the trigger is determined to have been fired and a message is determined to have been sent, the test management service can determine that the trigger is not validated.

[0012] The test management service may repeat this over a set of triggers over a configured time frame, as specified by the event scheduler. The test management service may use the response from the digital therapeutic application as part of the validation process. If it is determined that any triggers have not been validated by the end of the time frame, the test management service may determine that the event scheduler has not been validated. Conversely, if it is determined that all triggers have been validated by the end of the time frame, the test management service may determine that the event scheduler has been validated. The test management service may generate output based on the results of the validation and present it to an administrator of the digital therapeutic application or to a server hosting the application's resources.

[0013] By using a centralized clock to advance the system clocks of servers and applications, test management services can significantly reduce the testing and validation of digital therapeutic application triggers from weeks or months to minutes or hours. Thus, test management services can increase the likelihood that the functionality of a digital therapeutic application and the communication and transmission between the application and the server will function properly during runtime, compared to approaches that do not rely on clock time advancement. The functionality of a digital therapeutic application and the communication between the application and the server can be optimized and improved, thereby saving computational resources (e.g., processing and memory) and network bandwidth that would be wasted if the application functioned improperly. In the context of digital therapeutics, test management services can further ensure that a user's condition is addressed more efficiently.

[0014] The present disclosure relates to a system and method for simulating a test environment for a digital therapeutic application for addressing a pathological condition over a time frame. One or more processors can select a trigger from a plurality of triggers in an event scheduler to address a pathological condition. Each of the plurality of triggers can identify a respective timestamp from a plurality of timestamps, at which a service sends a corresponding message to a user device to address the pathological condition. The one or more processors can set a clock to the timestamp of the one trigger, the clock being accessible to the service and the user device. In response to the timestamp of the clock matching the timestamp of the one trigger, the one or more processors can cause the service to send a message to the user device to address the pathological condition. The one or more processors can determine that the message is being communicated between the service and the user device in accordance with the one trigger. In response to determining that the message has been communicated, the one or more processors can determine that the trigger of the event scheduler is validated.

[0015] In some embodiments, the one or more processors may select a second trigger from the plurality of triggers in response to the one trigger being validated. The second trigger may identify a second timestamp corresponding to an end of a time period spanned by the plurality of timestamps. The one or more processors may set the clock to the second timestamp of the second trigger. In response to the second timestamp of the clock matching the second timestamp of the one trigger, the one or more processors may cause the service to send a second message to the user device to address the condition. The one or more processors may determine that the service has sent the second message to the user device in accordance with the second trigger. In response to determining that the service has received the second response, the one or more processors may determine that the event scheduler is validated prior to the end of the time period. In some embodiments, the one or more processors may determine that the event scheduler is validated in response to determining that the service has successfully sent a respective message for each trigger of the plurality of triggers to the user device.

[0016] In some embodiments, the one or more processors can set the clock to a second timestamp of a second trigger of the plurality of triggers. In response to the second timestamp of the clock matching the second timestamp of the second trigger, the one or more processors can cause the service to send a second message to the user device to address the condition. In response to determining that the second message has not been delivered, the one or more processors can determine that the second trigger of the event scheduler has not been validated. In some embodiments, in response to determining that the second message has not been delivered on a first attempt, the one or more processors can reattempt to cause the service to send the second message on a second attempt.

[0017] In some embodiments, the one or more processors may, in response to determining that the second trigger is not validated, retrieve a transaction log identifying one or more of a plurality of messages communicated between the service and the user device. The one or more processors may generate an output including at least one of an indication of the failure to validate the second trigger and an identification of the transaction log. In some embodiments, the one or more processors may (i) cause the service to set a service clock from a first timestamp to the timestamp of the clock, and (ii) cause the user device to set a device clock from a second timestamp to the timestamp of the clock.

[0018] In some embodiments, the event scheduler may include a machine learning model that generates at least a portion of the plurality of triggers for the schedule. In some embodiments, at least one of the plurality of triggers may specify respective criteria that must be satisfied for the service to transmit the corresponding message to address the condition. The one or more processors may cause the service to transmit the message, further including causing the service to transmit the message in response to data associated with the user device satisfying the criteria specified by the trigger. In some embodiments, the one or more processors may correct the clock from an initial timestamp corresponding to a current time to a timestamp corresponding to a subsequent time specified by the one trigger prior to the occurrence of the subsequent time.

[0019] In some embodiments, the one or more processors may determine that the message has been communicated by at least one of: (i) determining that the service sent the message using a transmission log associated with the service, (ii) determining that the user device received the message from the service using a reception log associated with the user device, or (iii) determining that the service received a response from the user device within a certain time frame for sending the message. In some embodiments, the message may include at least one of a short message service (SMS) message, a multimedia messaging service (MMS), or an in-app message, and the message is presented at least partially contemporaneously with the user taking the medication. [Brief explanation of the drawings]

[0020] The above and other objects, aspects, features, and advantages of the present disclosure will become more apparent and will be better understood by referring to the following description taken in conjunction with the accompanying drawings. [Figure 1] FIG. 1 illustrates a block diagram of a system for simulating a testing environment for a digital therapeutic application to address a pathological condition over several time frames, according to an exemplary embodiment. [Figure 2] FIG. 2 illustrates a block diagram of a process for obtaining event schedule data and correcting a global clock in a system for simulating a test environment according to an exemplary embodiment. [Figure 3] FIG. 3 illustrates a block diagram of a process for validating message propagation in a system for simulating a test environment in accordance with an illustrative embodiment. [Figure 4] FIG. 4 illustrates a block diagram of a process for recorrecting a global clock and validating message propagation in a system for simulating a test environment, according to an example embodiment. [Figure 5] 5A-C show a flowchart of a method for simulating a test environment for a digital therapeutic application to address a pathological condition over several time frames, according to an example embodiment. [Figure 6] FIG. 6 is a block diagram of a server system and a client computer system according to an example embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] In reading the following description of the various embodiments, it will be helpful to refer to the descriptions and respective contents listed following each section of the specification.

[0022] Section A presents a system and method for simulating a test environment for an application over several time frames.

[0023] Section B is a description of network and computing environments that may be useful in implementing the embodiments described herein.

[0024] A. A system and method for simulating a test environment for an application over several time frames is presented. Referring now to FIG. 1 , a block diagram of an environment or system 100 for simulating a test environment for an application over several time frames is shown. Broadly, the system 100 may include at least one test management service 105, at least one session management service 110, at least one test environment 115, and at least one database 120, communicatively coupled to each other via at least one network 125. The test management service 105 may include at least one schedule reader 130, at least one clock orchestrator 135, at least one trigger validator 140, at least one test evaluator 145, and at least one global clock 150, etc. The session management service 110 may include at least one event scheduler 160, at least one session handler 165, and at least one server clock 150′, etc. The test environment 115 may include at least one user device 170, etc. The user device 170 may include at least one application 175, such as having at least one local clock 150″. The application 175 may provide or include at least one user interface 185. The user interface 185 may include one or more user interface (UI) elements 190A-N (hereinafter generally referred to as UI elements 190). The database 120 may include a set of messages 195A-N (hereinafter generally referred to as messages 195). Each component of the system 100 (e.g., the test management service 105, the session management service 110, the test environment 115, and the user device 170) may be implemented using a computing system such as that described in Section B.

[0025] More specifically, the test management service 105 (sometimes generally referred to herein as a service or test service) may be any computing device including one or more processors coupled with memory and software, capable of performing the various processes and tasks described herein. The test management service 105 may be associated with an entity that oversees or manages the testing, verification, validation, etc., of the application 175 or the session management service 110. The test management service 105 may communicate with the session management service 110, the test environment 115 including the user device 170, and the database 120, etc. The test management service 105 may be located, collocated, or otherwise associated with at least one server group. The server group may correspond to a data center, branch office, or campus where one or more servers corresponding to the test management service 105 are located. In some embodiments, the test management service 105 may be separate from the session management service 110 (e.g., as shown). For example, the test management service 105 may correspond to an administrator's computing device. In some embodiments, the test management service 105 may be part of the session management service 110 and may perform all of the functions assigned to the session management service 110 .

[0026] In the test management service 105, the schedule reader 130 interfaces with the session management service 110, retrieves triggers defined by the event scheduler 160, and communicates messages to and from the application 175. The clock orchestrator 135 can modify or set the global clock 150 according to the time specified by the trigger. The trigger validator 140 can determine whether messages are being communicated between the session management service 110 and the application 175 to validate the trigger. The test evaluator 145 can generate output based on the validation of the trigger. The global clock 150 can keep track of the time for the test management service 105. This time can be defined in units of year, month, day, hour, minute, second, millisecond, etc. The global clock 150 can be a hardware clock (e.g., a real-time clock, a system timer, or a process clock) or a software clock (e.g., a virtual clock), etc. The global clock 150 can be accessible by the session management service 110 and the user device 170 that includes the application 175. The global clock 150 functions or serves as the reference clock to which the server clock 150' and the local clock 150" are synchronized.

[0027] Session management service 110 (sometimes generally referred to herein as a service or messaging service) may be any computing device including one or more processors coupled with memory and software capable of performing the various processes and tasks described herein. Session management service 110 may be associated with an entity that manages the provision of resources and content for application 175 or the provision of digital therapeutics to users of user devices 170. In some embodiments, session management service 110 may be associated with the same entity that oversees or manages testing, testing, validation, etc. of application 175 or session management service 110. Session management service 110 may communicate with test management service 105, test environment 115 including user devices 170, database 120, etc. Session management service 110 may be located, collocated, or otherwise associated with at least one server group. This server group may correspond to a data center, branch office, or campus where one or more servers corresponding to session management service 110 are located. In some embodiments, session management service 110 may be separate from test management service 105 (e.g., as shown). In some embodiments, session management service 110 may be part of test management service 105 and may perform all of the functions assigned to test management service 105.

[0028] On the session management service 110, the event scheduler 160 can maintain a set of triggers that cause one or more messages 195 to be sent to the user device 170 and presented via the application 175. A session handler 165 can manage sessions with the application 175, where one or more messages 195 and responses are communicated according to these triggers. A server clock 150' can keep track of the time on the session management service 110. This time can be defined in units of year, month, day, hour, minute, second, millisecond, etc. The server clock 150 can be a hardware clock (e.g., a real-time clock, a system timer, or a process clock) or a software clock (e.g., a virtual clock). The server clock 150' can be synchronized with the global clock 150 of the test management service 105.

[0029] The test environment 115 may correspond to or include an environment for testing, verifying, validating, or otherwise evaluating an application 175 of a user device 170. In some embodiments, the test environment 115 may be created, instantiated, or otherwise generated by the test management service 105 to facilitate evaluation of the application 175 on the user device 170. For example, the test environment 115 may be an isolated, controlled, sandbox environment or a secure container to facilitate testing of the application 175 on the user device 170. In some embodiments, the test environment 115 may include a session management service 110 along with the user device 170 to facilitate evaluation of the application 175 and the session management service 110 on the user device 170.

[0030] User device 170 (sometimes referred to herein as end-user computing device) may be any computing device including one or more processors coupled with memory and software capable of performing the various processes and tasks described herein. User device 170 may communicate with test management service 105, test environment 115, database 120, etc. Applications 175 may be accessed using user device 170. In some embodiments, applications 175 may be downloaded (e.g., via a digital distribution platform) and installed on user device 170. In some embodiments, applications 175 may be web applications with resources accessible via network 125.

[0031] In some embodiments, user device 170 may correspond to a virtual machine running on hardware. For example, user device 170 may be a virtual machine having an operating system and applications 175 running on a physical computing device, such as a server, and may be managed by a hypervisor. The virtual machine may be part of an isolated and controlled sandbox environment corresponding to test environment 115. In some embodiments, user device 170 may be a physical device. For example, user device 170 may be a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, glasses), or laptop computer.

[0032] The application 175 executing on the user device 170 may be a digital therapeutic application. The application 175 may present or provide sessions (sometimes referred to herein as therapy sessions) to address at least one condition of the user. End user conditions include, for example, chronic pain (e.g., associated with or including arthritis, migraine, fibromyalgia, back pain, Lyme disease, endometriosis, repetitive stress injury, irritable bowel syndrome, inflammatory bowel disease, and cancer pain), skin conditions (e.g., atopic dermatitis, psoriasis, excoriation disorder, and eczema), cognitive disorders (e.g., mild cognitive impairment (MCI), Alzheimer's disease, multiple sclerosis, schizophrenia, etc.), psychiatric disorders (e.g., affective disorders, bipolar disorder, obsessive-compulsive disorder, borderline personality disorder, attention deficit / hyperactivity disorder, etc.), substance use disorders (e.g., opioid use disorder, alcohol use disorder, tobacco use disorder, hallucinogen use disorder, etc.), other conditions (e.g., narcolepsy, tumors or cancer, etc.), and the like.

[0033] The end user may also be taking medication to address a medical condition at least partially (e.g., over several sessions) concurrently with use of the application 175. For example, if the medication is for pain, the end user may be taking acetaminophen, a nonsteroidal anti-inflammatory composition, an antidepressant, an anticonvulsant, or other composition. For skin conditions, the end user may be taking steroids, antihistamines, or topical antiseptics. For cognitive impairment, the end user may be taking cholinesterase inhibitors or memantine. For psychiatric disorders, the end user may be taking antidepressants, mood stabilizers, antipsychotics, anti-anxiety medications, or stimulants. For substance abuse disorders, the end user may be receiving or taking naltrexone, disulfiram, acamprosate, or nicotine replacement therapy. The end user may also be receiving other psychotherapy for these conditions. In some embodiments, digital therapeutic content can be provided to the end user within the digital therapeutic application toward achieving the end user's endpoints. The endpoint may be, for example, an end-user's physical or behavioral goal, completion of a medication regimen, or an endpoint indicated by a physician or end-user. At least one of the end-user devices 170 may have a digital therapeutic application and may provide sessions (sometimes referred to herein as therapy sessions) to address at least one condition of the end-user.

[0034] The application 175 may include, present, or otherwise provide to a user of the user device 170 at least one user interface 185 including one or more user interface elements 190A-N (hereinafter generally referred to as UI elements 190). The user interface 185 may be provided according to the configuration of the application 175. The UI elements 190 may correspond to visual components of the user interface 185, such as command buttons, text boxes, check boxes, radio buttons, menu items, and sliders. In some embodiments, the application 175 may be a digital therapeutic application, and sessions for addressing a pathological condition (sometimes referred to herein as therapy sessions) may be provided via the user interface 185. The user interface 185 may include a set of UI elements 190 for presenting digital therapeutic content. In some embodiments, the application 175 may include functionality attributed to the session management service 110 herein, such as an event scheduler 160 or a session handler 165. Although application 175 is described herein primarily as a digital therapeutic application, application 175 may include other types of applications, such as a messaging application, a word processor, a spreadsheet program, an email agent, a web browser, a video game, database management software, or computer-aided design software.

[0035] Database 120 may store and maintain various resources and data associated with test management service 105, session management service 110, application 175, and the like. Database 120 may include a database management system (DBMS) for organizing and arranging data maintained on the database, such as instances of data elements. Database 120 may communicate with test management service 105, session management service 110, and application 175 via network 125. While performing various operations, test management service 105, session management service 110, and application 175 may each access database 120 to send and retrieve identified data therefrom. Test management service 105, session management service 110, and application 175 may also write data to database 120 from the performance of such operations.

[0036] The database 120 can store and maintain a set of messages 195 for addressing a condition of a user of the application 175. Each message 195 can include digital therapeutic content. The messages 195 can be transmitted, sent, or otherwise provided and presented to the user device 170 by the session management service 110. The messages 195 themselves can be in any format. In some embodiments, at least one message 195 can be a short message / messaging service (SMS) message. The SMS message can include text of a configured length (e.g., limited number of alphanumeric characters) to be transmitted over a cellular network. In some embodiments, at least one message 195 can be a multimedia messaging service (MMS) message. The MMS message can include multimedia content such as text, images, video, or audio. In some embodiments, at least one message 195 can be an in-app message. The in-app message can include instructions for presenting multimedia content (e.g., in the form of text, images, video, audio content, or any combination thereof) via a UI element 190 of the user interface 185 of the application 175. The instructions may identify or specify one or more UI elements 190 to be rendered, displayed, or otherwise presented within the user interface 185 of the application 175 .

[0037] The digital therapeutic content of the message 195 may be any modality, such as text, image, audio, video, or multimedia content, or any combination thereof. The message 195 may be stored and maintained in the database 120 using one or more files. For example, in the case of text, the digital therapeutic content may be stored as a text file (TXT), a rich text file (RTF), an extensible markup language (XML), and a hypertext markup language (HTML), among others. For images, the digital therapeutic content may be stored in the Joint Photographic Experts Group (JPEG) format, the Portable Network Graphics (PNG) format, the Graphic Interchange (GIF) format, or the Scalable Vector Graphics (SVG) format, among others. For audio, the digital therapeutic content may be stored as a waveform audio file (WAV), a moving image experts group format (e.g., MP3 and MP4), and the Ogg Vorbis (OGG) format, among others. For video, the digital therapeutic content can be stored as a Moving Image Experts Group format (e.g., MP3 and MP4), QuickTime Movie (MOV), Windows Movie Video (WMV), etc. For multimedia content, the digital therapeutic content can be Audio Video Interleave (AVI), a Moving Image Experts Group format (e.g., MP3 and MP4), QuickTime Movie (MOV), Windows Movie Video (WMV), etc.

[0038] In some embodiments, the digital therapeutic content of the message 195 can include a set of stimuli (e.g., in audio, visual, or text format) for the user to perform a specific task. Tasks can include, for example, an implicit association task (IAT) (e.g., associating stimuli with concepts), attention bias modification training (ABMT) (e.g., training a user to distract from specific stimuli), an emotional faces memory task (EFMT) (e.g., testing a user's recognition and memory of emotions displayed in specific faces), a digital support tool (DST) (e.g., providing messages based on a user's state), or adaptive goal setting (AGS) (e.g., providing messages based on a user's dynamic goals). Although the message 195 is described herein in terms of digital therapeutic content, the message 195 can include other types of content.

[0039] Referring now to FIG. 2 , a block diagram of a process 200 for obtaining event schedule data and modifying a global clock in a system 100 for simulating a test environment is shown. The process 200 may include or correspond to operations performed in the system 100 for identifying triggers and setting a clock according to the triggers. Under the process 200, the event scheduler 160 executing in the session management service 110 may create, generate, or otherwise generate at least one schedule 205 for sending and presenting messages 190 to an application 175 to address a condition (e.g., of a potential user). The schedule 205 may define, specify, or otherwise include a set of triggers 210A-N (hereinafter generally referred to as triggers 210) and a corresponding set of timestamps 215A-N (hereinafter generally referred to as timestamps 215). In some embodiments, the schedule 205 may define, specify, or specify a sequence of the set of triggers 210. The sequence may include or specify one or more branches between the set of triggers 210. For example, the sequence of schedule 205 may specify that trigger 210B is evaluated when trigger 210A is satisfied and fired. Conversely, the sequence of schedule 205 may specify that trigger 210C is evaluated if trigger 210A fails to be satisfied.

[0040] Each trigger 210 can define, specify, or otherwise identify a respective timestamp 215 at which the session management service will transmit, provide, or otherwise transmit at least one corresponding message 195 to the user device 170 to address the condition. Each timestamp 215 can define, specify, or otherwise identify the time at which the corresponding message 195 is to be transmitted. The timestamps 215 can be defined in units of years, months, days, hours, minutes, seconds, milliseconds, etc. A set of timestamps 215 spans at least one time period 220. The time period 220 can range, for example, from five days to six months. The time period 220 can correspond to a period during which a digital therapy is provided (e.g., in the form of a message 195). In some embodiments, the time period can correspond to or be defined by the session management service 110 with respect to the application 175. In some embodiments, the time period can correspond to or be defined by the duration between the first timestamp 215A and the last timestamp 215N in the schedule 205.

[0041] In some embodiments, at least one of the set of triggers 210 may define, specify, or otherwise identify at least one corresponding criterion that must be satisfied in order for the session management service 110 to send the corresponding message 195. The criterion may specify, define, or otherwise identify one or more conditions that data associated with the user device 170 (or an application 175 running on the user device 170) must satisfy in order for the session management service 110 to send the corresponding message 195. The data may include user profile information, interaction data, an indication of a previous activity or task completion, the device type, or the location of the user device 170, etc. In some embodiments, the criterion condition may be in addition to the timestamp 215. For example, the criterion of the trigger 210 may specify that the user must have completed a previous task by the time specified by the timestamp 215 in order to provide the corresponding message 195. In some embodiments, the criterion condition may be in lieu of or independent of the timestamp 215. For example, the criteria of the trigger 210 may specify that a user, via the application 175, requests that one or more corresponding messages 195 be provided, regardless of the time of day. Once the schedule 205 is generated, the event scheduler 160 may save and maintain the schedule 205 in storage at the session management service 110 using one or more files (e.g., Extensible Markup Language (XML), Comma Separated Values ​​(CSV) delimited text files, or Structured Query Language (SQL) files).

[0042] In some embodiments, the event scheduler 160 may generate the schedule 205 in a fixed manner. For example, the event scheduler 160 may generate the schedule 205 to include a set of triggers 210 during an initialization phase of a digital therapeutic intervention. The set of triggers 210 may be fixed and unmodifiable throughout the period of a set of timestamps 215. In some embodiments, the event scheduler 160 may dynamically generate the schedule 205 as additional data (e.g., user profile information or user interaction data) is received from the user device 170 (or application 175). For example, the event scheduler 160 may update or modify at least a portion of the set of triggers 210 using data received from the user device 170 during the period in which the digital therapeutic is provided (e.g., in the form of messages 195). The event scheduler 160 may add new portions of triggers 210 and timestamps 215 to the schedule 205 using new data received from the user device 170. The event scheduler 160 may continue to generate triggers 210 and corresponding timestamps 215 throughout the entire period 220 (e.g., for a digital therapeutic). As each trigger 210 and corresponding timestamp 215 for the schedule 205 is generated, the event scheduler 160 may send, provide, or transmit the trigger 210 and corresponding timestamp 215 to the test management service 105.

[0043] In some embodiments, the event scheduler 160 may apply a machine learning (ML) model to the data to identify at least portions of the triggers 210 and corresponding portions of the timestamps 215 and generate the schedule 205 in a dynamic manner. The event scheduler 160 may include an ML model for generating the triggers 210 and corresponding portions of the timestamps 215. The ML model may be any architecture, such as an artificial neural network (ANN) (e.g., an autoencoder, a convolutional neural network (CNN), or a transformer), a support vector machine (SVM), a clustering model (e.g., a k-nearest neighbor model), a Bayesian classifier, a decision tree, or a random forest. Generally, the ML model may include at least one input (e.g., data), at least one output (e.g., the schedule 205), and a set of weights relating the input to the output.

[0044] Subsequently, the ML model may be trained using a training dataset. The training dataset may include, for example, historical data associated with a user device or application (e.g., user profile information or user interaction data) and corresponding portions of triggers and timestamps. The weights of the ML model may be trained (e.g., by the session management service 110 or another service, according to supervised, unsupervised, or weakly supervised learning) to learn patterns in the training dataset. Once the ML model is established, the event scheduler 160 can apply the ML model to received data from the user device 170 (or application 175). The event scheduler 160 can process the data according to a set of weights of the ML model. From the processing, the event scheduler 160 may generate the trigger 210 and timestamp 215 portions of the schedule 205.

[0045] A schedule reader 130 executing on the test management service 105 can retrieve, receive, or otherwise identify a schedule 205 including a set of triggers 210 and a set of corresponding timestamps 215 from the event scheduler 160. The schedule 205 may be generated in a fixed manner or may be continually generated in a dynamic manner. In some embodiments, the schedule reader 130 can access the event scheduler 160 to identify, retrieve, or otherwise obtain the set of triggers 210 and the set of corresponding timestamps 215. If the schedule 205 is generated in a fixed manner, the schedule reader 130 can retrieve a schedule 205 including all of the set of triggers 210 and all of the set of corresponding timestamps 215 within a time period. If the schedule 205 is generated in a dynamic manner, the schedule reader 130 can retrieve, identify, or otherwise receive each of the triggers 210 and the corresponding timestamps 215 as they are generated and provided by the event scheduler 160.

[0046] The schedule reader 130 can access the event scheduler 160 to obtain or otherwise identify a set of triggers 210 and a set of corresponding timestamps 215. In some embodiments, the schedule reader 130 can process or parse the schedule 205 to extract or identify the set of triggers 210 and the set of corresponding timestamps 215 from the schedule 205. The schedule reader 130 can identify or select at least one trigger 210 (e.g., trigger 210A) from the set of triggers 210. In the illustrated example, trigger 210A may be the first trigger 210 in the set of triggers 210 and may correspond to the beginning of a digital intervention provided in the form of a message 195 corresponding to the user device 170. Trigger 210A may also correspond to the beginning of a portion of the digital intervention that is being tested and validated. The schedule reader 130 can select or identify a trigger 210 from the beginning of a digital intervention defined by the event scheduler 160. Once the triggers 210 are identified, the schedule reader 130 may extract or identify the timestamps 215 for the triggers 210. In the illustrated example, the schedule reader 130 may identify the timestamp 215A associated with the first trigger 210A. The timestamp 215A may correspond to the start of the period 220.

[0047] A clock orchestrator 135 executing on the test management service 105 can assign or set the global clock 150 to a timestamp 215 specified by a trigger 210. In the illustrated example, when a first trigger 210A is selected, the clock orchestrator 135 can assign or set the global clock 150 to a first timestamp 215A. In setting the global clock 150, the clock orchestrator 135 can change, alter, or otherwise modify the global clock 150 from an initial timestamp to a timestamp 215 (e.g., timestamp 215A) specified by a trigger 210 (e.g., trigger 210A). The initial timestamp can correspond to a current time (e.g., an actual time). The timestamp 215 specified by the trigger 210 can correspond to a time subsequent to the current time. In this manner, the clock orchestrator 135 can modify the global clock 150 to a timestamp 215 prior to the actual time that corresponds to the timestamp 215, overwriting the current time. Once set, the global clock 150 can keep track of the time and increment the time from the timestamp 215 .

[0048] In response to setting the global clock 150 to the timestamp 215, the clock orchestrator 135 may change, set, or otherwise modify the server clock 150′ on the session management service 110 to be set to the timestamp 215 (e.g., timestamp 215A). Additionally, the clock orchestrator 135 may change, set, or otherwise modify the local clock 150″ on the user device 170 (or application 175) to be set to the timestamp 215 (e.g., timestamp 215A). As described above, the global clock 150 on the test management service 105 may have access to the server clock 150′ on the session management service 110 and the local clock 150″ on the user device 170. In some embodiments, the clock orchestrator 135 may cause the server clock 150′ to be set to the timestamp 215 of the global clock 150 from an initial time (e.g., the current time). In some embodiments, the clock orchestrator 135 may cause the local clock 150″ to be set from an initial time (eg, the current time) to the timestamp 215 of the global clock 150.

[0049] In some embodiments, the clock orchestrator 135 may create, generate, or otherwise generate at least one instruction 225. The instruction 225 may include or identify a timestamp 215 at which the global clock 150 is set. The instruction 225 may be generated in response to the setting of the global clock 150. The instruction 225 may change, set, or otherwise modify other clocks (e.g., the server clock 150′ and the local clock 150″) to the timestamp 215 at which the global clock 150 is set (e.g., timestamp 215A). For example, the instruction 225 may include commands for recipient clocks (e.g., the server clock 150′ and the local clock 150″) to synchronize their time to the timestamp 215 of the global clock 150. The instruction 225 may be generated in accordance with the Network Time Protocol (NTP), Precision Time Protocol (PTP), timestamp insertion, or the like. In some embodiments, the instruction 225 may include a command to the session management service 110 to evaluate a trigger 210 associated with the timestamp 215. This generation allows the clock orchestrator 135 to provide, transmit, or otherwise send instructions 225 to the session management service 110 and the user device 170.

[0050] The session management service 110 (of the event scheduler 160) can change, set, or modify the server clock 150' from the initial timestamp to the timestamp 215. In some embodiments, the session management service 110 can interface with or access the global clock 150 on the test management service 105 to obtain, obtain, or identify the timestamp 215 of the global clock 150. The session management service 110 can access the global clock 150 according to a defined cycle (e.g., every 5 to 30 minutes). This identification allows the session management service 110 to modify or set the server clock 150' to the timestamp 215. The timestamp 215 on the server clock 150' may be very similar (e.g., within 95%) to the timestamp 215 on the global clock 150. In some embodiments, the session management service 110 can obtain, identify, or receive instructions 225 from the test management service 105. Upon receipt, the session management service 110 can extract or identify the timestamp 215 from the instructions 225. This determination allows the session management service 110 to correct or set the server clock 150' to the timestamp 215. Once set, the server clock 150' can keep track of the time and increment the time from the timestamp 215.

[0051] The user device 170 (or its application 175) can change, set, or otherwise modify the local clock 150" from the initial timestamp to the timestamp 215. In some embodiments, the user device 170 can interface with or access the global clock 150 on the test administration service 105 to obtain, obtain, or identify the timestamp 215. The user device 170 can access the global clock 150 according to a defined cycle (e.g., every 5 to 30 minutes). This identification allows the user device 170 to modify or set the local clock 150" to the timestamp 215. In some embodiments, the user device 170 can obtain, identify, or otherwise receive instructions 225 from the test administration service 105. Upon receipt, the user device 170 can extract or identify the timestamp 215 from the instructions 225. The timestamp 215 on the local clock 150" may be very similar (e.g., within 95%) to the timestamp 215 on the global clock 150. This determination allows the user device 170 to correct or set the local clock 150" to the timestamp 215. Once set, the local clock 150" can keep track of the time and increment the time from the timestamp 215.

[0052] Referring now to FIG. 3 , a block diagram of a process 300 for validating message transmission in a system 100 for simulating a test environment is shown. Process 300 may include or correspond to operations performed in system 100 for evaluating triggers to validate message transmission. In process 300, with server clock 150′ set to timestamp 215 (e.g., timestamp 215A shown), a session handler 165 executing on session management service 110 may identify or determine whether to provide, send, or otherwise forward at least one message 195′ to user device 170. In some embodiments, in response to the time of global clock 150 being set or changed to timestamp 215, test management service 105 may cause session handler 165 to determine whether to transmit message 195′. In some embodiments, in response to receiving instruction 225, test management service 105 may cause session handler 165 to evaluate trigger 210 and determine whether to transmit message 195′ identified by trigger 210. Message 195' can correspond to at least one of the set of messages 195 identified by trigger 210. In some embodiments, message 195' can be identified by criteria defined by trigger 210 and sent to user device 170. For example, message 195' can correspond to the message identified by initial trigger 210A that is sent to user device 170 if the criteria are met.

[0053] To determine whether transmission is necessary, the session handler 165 can identify or determine whether the data associated with the user device 170 satisfies criteria defined by the trigger 210. The evaluation of whether the criteria defined by the trigger 210 are satisfied can be performed according to a defined cycle (e.g., every 30 seconds to every hour). The data can identify or include, for example, user profile information, interaction data, indications of previous activities or task completion, device type, or location of the user device 170. The data associated with the user device 170 can be aggregated, collected, or otherwise received from the user device 170 (or application 175, or another device associated with the user device 170) or from the database 120. The time period of this data can span the timestamp 215 associated with the trigger 210. For example, the session handler 165 can retrieve, identify, or otherwise receive data to check against the criteria over a time period up to the time corresponding to the timestamp 215A specified by the trigger 210A. In some embodiments, this data may be included as part of the test data of a test environment 115 that tests, validates, or evaluates the event scheduler 160 (or schedule 205).

[0054] If the data associated with the user device 170 meets the criteria of the trigger, the session handler 165 may determine to send a message 195′ to the user device 170. In some embodiments, the session handler 165 may determine that the data obtained up to the timestamp 215 meets the criteria of the trigger. Once this determination is made, the session handler 165 may determine that the message 195′ should be sent to the user device 170. Once the determination to send the message 195′ is made, the session handler 165 may identify or select the message 195′ from the set of messages 195 in the database 120. The message 195′ may correspond to at least one of the set of messages 195 defined by the trigger 210. The message 195′ may include, for example, a short message / messaging service (SMS) message, a multimedia messaging service (MMS) message, an in-app message, or the like. The message 195′ may be presented in part concurrently with a medication that the user (e.g., a potential user of the user device 170 or application 175) is receiving to address a medical condition. In certain cases, session management service 110 may send or erroneously refrain from sending message 195' due to a software bug, code defect, soft error, or the like. The behavior of session management service 110 may be unexpected, unexpected, or deviate from the developer's intent.

[0055] Once the selection is made, the session handler 165 can send, provide, or otherwise transmit the message 195′ to the user device 170. In some embodiments, the session handler 165 can also update at least one transmission log 305 (also referred to herein as a transaction log). This transmission log 305 can include or identify a set of messages (or responses) exchanged between the session management service 110 and the user device 170. The transmission log 305 can be stored and maintained in a database (e.g., a database local to the session management service 110 or the database 120) using one or more files (e.g., Extensible Markup Language (XML), Comma Separated Values ​​(CSV) text files, or Structured Query Language (SQL) files). The transmission log 305 can also include or identify a timestamp corresponding to the time each message was sent. The transmission log 305 is accessible by the test management service 105. In some embodiments, the transmission log 305 can be maintained and updated by the test management service 105. Once the message 195' is sent, the session handler 165 may update the transmission log 305 to include the identification of the message 195' and a timestamp corresponding to the time the message 195' was sent.

[0056] Conversely, if the data associated with user device 170 does not meet the criteria of the trigger, session handler 165 may decide not to send message 195' to user device 170. In some embodiments, session handler 165 may determine that the data obtained up to timestamp 215 does not meet the criteria of the trigger. If this determination is made, session handler 165 may decide not to send message 195' to user device 170. If a decision not to send the message is made, session handler 165 may refrain from sending the message associated with trigger 210 to user device 170. Session handler 165 may proceed to evaluate the next trigger 210 (e.g., trigger 210B) after the time of server clock 150' exceeds the timestamp 215 (e.g., timestamp 215A) associated with the previous trigger 210 (e.g., trigger 210A). The session handler 165 may send, provide, or otherwise communicate the result of the decision (eg, whether to send the message 195' associated with the trigger 210 of the timestamp 215) to the test management service 105.

[0057] The user device 170 (or application 175) can retrieve, identify, or receive the message 195′ from the session management service 110. Upon receiving the message, the user device 170 can display, render, or otherwise present the message 195′. If the message 195′ is an SMS or MMS message, the user device 170 can present the contents of the message 195′ using a messaging application. This messaging application can be, for example, a default application used by the operating system on the user device 170 and can present and communicate the message in the SMS or MMS protocol. If the message 195′ is an in-app message, the application 175 running on the user device 170 can present the contents of the message 195′ via a UI element 190 of the user interface 185, as prompted by the message 195′.

[0058] In some embodiments, the user device 170 can update at least one reception log 315 (also referred to herein as a transaction log). The reception log 315 can include or identify a set of messages or responses exchanged between the user device 170 and the session management service 110. The reception log 315 can be stored and maintained in a database (e.g., a database local to the session management service 110 or the database 120) using one or more files (e.g., Extensible Markup Language (XML), Comma Separated Values ​​(CSV) text files, or Structured Query Language (SQL) files). The reception log 315 can include or identify a timestamp corresponding to the time of transmission of each message or response. The reception log 315 is accessible by the test management service 105. In some embodiments, the reception log 315 can be maintained and updated by the test management service 105. When a message 195′ is received, the user device 170 can update the reception log 315 to include an identification of the receipt of the message 195′ and a timestamp corresponding to the receipt of the message 195′.

[0059] Upon presentation of the message 195′, the user device 170 (or application 175) can create, generate, or otherwise generate at least one response 310. The response 310 can include an acknowledgement of receipt of the message 195′ or an indication that the message 195′ was presented via the user device 170. In some embodiments, the user device 170 can retrieve, obtain, or identify data associated with the presentation of the message 195′. This data can include interaction data or event data of a process on the user device 170 and can also be part of test data for the test environment 115 for testing, validating, or evaluating the event scheduler 160 (or schedule 205). In some embodiments, the user device 170 can generate the response 310 to include data (e.g., user interaction data or event data).

[0060] Once the response 310 is generated, the user device 170 may send, provide, or otherwise transmit the response 310 to the session management service 110. Once the response 310 is sent, the user device 170 may update the reception log 315 to include an identification of the response 310 and a timestamp corresponding to the time the response 310 was sent. The session handler 165 may then retrieve, identify, or receive the response 310 from the user device 170. Upon receiving the response 310, the session handler 165 may update the transmission log 315 to include an identification of the response 310 and a timestamp corresponding to the time the response 310 was received. Additionally, the session handler 165 may store and maintain the response 310 in the database 120.

[0061] Additionally, the trigger validator 140 executing on the test management service 105 can determine or identify whether the message 195′ is being communicated between the session management service 110 and the user device 170 in accordance with the trigger 210. Based on whether the message 195′ is being properly communicated as specified in the trigger 210, the trigger validator 140 can decide whether to validate the trigger 210 associated with the timestamp 215. To do so, the trigger validator 140 can determine whether the criteria of the trigger 210 are satisfied. The trigger validator 140 can interface with or access the session handler 165 to obtain, acquire, or otherwise identify the results of the evaluation of the trigger 210 executed by the session handler 165 of the session management service 110. Additionally, the trigger validator 140 can monitor or verify the communication of the message 195′ between the session management service 110 and the user device 170. In some embodiments, the trigger validator 140 may monitor or verify that the transmission of the response 310 from the user device 170 to the session management service 110 occurs within a time frame (e.g., 30 seconds to 1 hour) relative to the transmission of the previous message 195′. In some embodiments, the trigger validator 140 may monitor updates to a transaction log (e.g., the sending log 305 or the receiving log 315) of the transmitted message or response.

[0062] If the criteria of the trigger 210 are satisfied, the trigger validator 140 can determine or determine whether the message 195′ is being communicated in accordance with the trigger 210 between the session management service 110 and the user device 170. If the message 195′ is identified as being sent from the session management service 110 and the user device 170 (e.g., from a monitor or from a transaction log), the trigger validator 140 can determine that the message 195′ is being communicated in accordance with the trigger 210. In some embodiments, the trigger validator 140 can further determine that the message 195′ is being communicated in accordance with the trigger 210 if a response 310 to the message 195′ is received within the time frame of the message 195′. If the message 195′ is further determined to be being communicated in accordance with the trigger 210, the trigger validator 140 can determine or determine that the trigger 210 of the event scheduler 160 (or schedule 205) is validated.

[0063] On the other hand, if it is determined that the message 195′ has not been sent from the session management service 110 and the user device 170, the trigger validator 140 may determine that the message 195′ has not been communicated in accordance with the trigger 210. Furthermore, if it is determined that the message 195′ has not been communicated as specified in the trigger 210, the trigger validator 140 may determine or determine that the trigger 210 of the event scheduler 160 (or schedule 205) has not been validated. In some embodiments, in response to determining that the message 195′ has not been communicated in accordance with the trigger 210 on a first attempt, the trigger validator 140 may retry or reattempt to have the session management service 110 send the message 195′. For example, the trigger validator 140 may create, generate, or otherwise generate an instruction to the session management service 110 to re-evaluate the trigger 210. Upon receiving this, the session handler 165 may re-evaluate the criteria of the trigger 210 and the system 100 may repeat the functions detailed herein for a second attempt.

[0064] Conversely, if the criteria of the trigger 210 are not satisfied, the trigger validator 140 can determine or identify whether the message 195′ has not been communicated in accordance with the trigger 210 between the session management service 110 and the user device 170. If it is determined that the message 195′ has not been sent from the session management service 110 and the user device 170 (e.g., from a monitor or from a transaction log), the trigger validator 140 can determine that the message 195′ has not been communicated in accordance with the trigger 210. Furthermore, if it is determined that the message 195′ has not been communicated as specified in the trigger 210, the trigger validator 140 can determine or determine that the trigger 210 of the event scheduler 160 (or schedule 205) has been validated. On the other hand, if it is determined that the message 195′ has been sent from the session management service 110 and the user device 170, the trigger validator 140 can determine that the message 195′ has not been communicated in accordance with the trigger 210. Further, if message 195′ is confirmed to have been communicated in accordance with trigger 210, trigger validator 140 may identify or determine that a trigger 210 of event scheduler 160 (or schedule 205) has not been validated. In some embodiments, in response to determining that at least one trigger 210 (e.g., trigger 210A) has not been validated, trigger validator 140 may stop testing and validating the set of triggers 210 of event scheduler 160 (or scheduler 205).

[0065] Referring now to FIG. 4, a block diagram of a process 400 for recalibrating a global clock and validating message transmission in a system 100 for simulating a test environment is shown. Process 400 can include or correspond to operations of evaluating a subsequent trigger to recalibrate the clock and validate message transmission. Process 400 may overlap or repeat operations of processes 200 and 300 detailed herein for another trigger 210 or timestamp 215. Under process 400, schedule reader 130 can identify or select at least one subsequent trigger 210 from the set of triggers 210 of event scheduler 160 (or schedule 205). The selection of the subsequent trigger 210 can be similar to the selection of a previous trigger 210 (e.g., trigger 210A), as detailed herein. In some embodiments, schedule reader 130 can select a subsequent trigger 210 upon determining that the previous trigger 210 has been validated. In some embodiments, the schedule reader 130 may select a subsequent trigger 210 regardless of whether the previous trigger 210 has been validated. In some embodiments, the subsequent trigger 210 is identified by the previous trigger 210 as the next trigger to be evaluated in the schedule 205.

[0066] The schedule reader 130 can access the event scheduler 160 (or schedule 205) to obtain, retrieve, or otherwise identify a subsequent trigger 210. The subsequent trigger 210 can be generated by the event scheduler 160 in a static or dynamic manner. Once a trigger 210 is identified, the schedule reader 130 can extract or identify a timestamp 215 for the trigger 210. In the illustrated example, trigger 210N is the last trigger 210 in the set of triggers 210 and can define a timestamp 215N that corresponds to the end of the period 220. In some embodiments, trigger 210N can correspond to the end of a digital intervention that is provided to the user device 170 in the form of a corresponding message 195.

[0067] The clock orchestrator 135 may assign or set the global clock 150 to the timestamp 215 identified by the trigger 210. Setting the global clock 150 using the timestamp 215 (e.g., timestamp 215N) may be similar to setting the global clock 150 described herein (e.g., using timestamp 215A). In the illustrated example, when the last trigger 210N is selected, the clock orchestrator 135 may assign or set the global clock 150 to the last timestamp 215N. In setting the global clock 150, the clock orchestrator 135 may change, alter, or otherwise modify the global clock 150 from a previously tested timestamp to the timestamp 215 (e.g., timestamp 215N) identified by the trigger 210 (e.g., trigger 210N). The previous timestamp matches the timestamp 215 of the previous trigger 210 and is incremented by some amount from the previous evaluation of the previous trigger 210. In response to setting the global clock 150 to the timestamp 215, the clock orchestrator 135 may change, set, or otherwise modify the server clock 150′ and the local clock 150″ to be set to the timestamp 215 (e.g., timestamp 215N).

[0068] In some embodiments, the clock orchestrator 135 can create, generate, or otherwise generate at least one instruction 225′. The instruction 225 can include or specify a timestamp 215 (timestamp 215N) to which the global clock 150 is set. Once generated, the clock orchestrator 135 can provide, transmit, or otherwise send the instruction 225 to the session management service 110 and the user device 170. The session management service 110 (of the event scheduler 160) can change, set, or otherwise modify the server clock 150′ from a previous timestamp to the timestamp 215. Additionally, the user device 170 (or its application 175) can change, set, or otherwise modify the local clock 150″ from a previous timestamp to the timestamp 215, where the previous timestamp matches the timestamp 215 of the previous trigger 210 and is incremented by some amount since the previous evaluation of the previous trigger 210.

[0069] A session handler 165 on the session management service 110 can identify or determine whether to provide, transmit, or otherwise forward at least one message 195″ to the user device 170 with the server clock 150′ set to a timestamp 215 (e.g., the illustrated timestamp 215N). The transmission of the message 195″ can be similar to the transmission of the message 195′ described above. To determine whether to transmit, the session handler 165 can identify or determine whether the data associated with the user device 170 satisfies criteria defined by a trigger 210. The evaluation of whether the criteria defined by the trigger 210 are satisfied can be performed according to a predefined cycle. As described above, the data associated with the user device 170 can be aggregated, collected, or otherwise received from the user device 170 (or an application 175, or another device associated with the user device 170) or from the database 120. The time of this data can extend to the timestamp 215 associated with the trigger 210 (e.g., up to timestamp 215N).

[0070] If the data associated with the user device 170 meets the criteria of the trigger, the session handler 165 can determine to send the message 195" to the user device 170. In some embodiments, the session handler 165 can determine that the data obtained up to the timestamp 215 meets the criteria of the trigger. Once this determination is made, the session handler 165 can determine that the message 195" should be sent to the user device 170. Once the determination is made to send the message 195", the session handler 165 can identify or select the message 195" from the set of messages 195 in the database 120. Once the selection is made, the session handler 165 can send, provide, or otherwise transmit the message 195" to the user device 170. In some embodiments, the session handler 165 can also update the transmission log 305 to include or identify the sending of the timestamp 214. Conversely, if the data associated with the user device 170 does not meet the criteria of the trigger, the session handler 165 can determine not to send the message 195" to the user device 170. In some embodiments, the session handler 165 may determine that the data obtained up to the timestamp 215 does not meet the criteria of the trigger. If this determination is made, the session handler 165 may decide not to send the message 195" to the user device 170.

[0071] The user device 170 (or application 175) can obtain, identify, or receive a message 195" from the session management service 110. Upon receiving the message, the user device 170 can display, render, or otherwise present the message 195". In some embodiments, the user device 170 may also update at least one receipt log 315 to include or identify receipt of the message 195″. Upon presentation of the message 195′, the user device 170 (or application 175) may create, generate, or otherwise generate at least one response 310′. The generation of the response 310′ may be similar to the generation of the response 310 described above. The response 310′ may include an acknowledgement of receipt of the message 195″ or an indication that the message 195″ was presented via the user device 170. Once the response 310 is generated, the user device 170 may send, provide, or otherwise transmit the response 310 to the session management service 110. Once the response 310 is sent, the user device 170 may update the receipt log 315 to include an identification of the response 310 and a timestamp corresponding to the time the response 310 was sent.

[0072] The trigger validator 140 can determine or identify whether the message 195" is being communicated between the session management service 110 and the user device 170 in accordance with the trigger 210. The identification and validation for the message 195" can be similar to the identification and validation for the message 195', as described above. Based on whether the message 195" is being properly communicated as specified in the trigger 210, the trigger validator 140 can determine whether to validate the trigger 210 associated with the timestamp 215.

[0073] If the criteria of the trigger 210 are satisfied, the trigger validator 140 can determine or identify that the message 195" is being communicated in accordance with the trigger 210 between the session management service 110 and the user device 170. If the message 195" is identified as being sent from the session management service 110 and the user device 170 (e.g., from a monitor or from a transaction log), the trigger validator 140 can identify that the message 195" is being communicated in accordance with the trigger 210. In some embodiments, the trigger validator 140 can further identify that the message 195" is being communicated in accordance with the trigger 210 if a response 310 to the message 195" is received within the time frame of the message 195". Furthermore, if it is determined that message 195" has been transmitted in accordance with trigger 210, trigger validator 140 may identify or determine that trigger 210 of event scheduler 160 (or schedule 205) has been validated. On the other hand, if it is determined that message 195" has not been sent from session management service 110 and user device 170, trigger validator 140 may identify that message 195" has not been transmitted in accordance with trigger 210. Furthermore, if it is determined that message 195" has not been transmitted as specified in trigger 210, trigger validator 140 may identify or determine that trigger 210 of event scheduler 160 (or schedule 205) has not been validated.

[0074] Conversely, if the criteria of the trigger 210 are not satisfied, the trigger validator 140 can determine or identify whether the message 195" has not been communicated in accordance with the trigger 210 between the session management service 110 and the user device 170. If it is determined that the message 195" has not been sent from the session management service 110 and the user device 170 (e.g., from a monitor or from a transaction log), the trigger validator 140 can determine that the message 195" has not been communicated in accordance with the trigger 210. Furthermore, if it is determined that the message 195" has not been communicated as specified in the trigger 210, the trigger validator 140 can determine or determine that the trigger 210 of the event scheduler 160 (or schedule 205) has been validated. On the other hand, if it is determined that the message 195" has been sent from the session management service 110 and the user device 170, the trigger validator 140 can determine that the message 195" has not been communicated in accordance with the trigger 210. Furthermore, if the message 195" is confirmed to have been communicated in accordance with the trigger 210, the trigger validator 140 may identify or determine that the trigger 210 of the event scheduler 160 (or schedule 205) has not been validated.

[0075] The test evaluator 145 executing on the test management service 105 can identify or determine whether the event scheduler 160 (or schedule 205) is validated based on determining whether the triggers 210 are validated. The validation can be performed before the actual time corresponding to the end of the time period 220. For example, the test evaluator 145 can determine that a set of triggers 210 all validate successfully, and thus that the event scheduler 160 (or schedule 205) will be validated within 30 minutes, even though the time period 220 spans 60 to 90 days. If it is determined that the messages 195 for each trigger 210 in the set of triggers 210 were successfully communicated, the test evaluator 145 can identify or determine that the event scheduler 160 (or schedule 205) is validated. In some embodiments, the test evaluator 145 may determine that the event scheduler 160 (or schedule 205) is validated when it is determined that each trigger 210 in the set of triggers 210 has successfully validated. In some embodiments, the test evaluator 145 may identify or determine that the event scheduler 160 is validated when it is determined that the last trigger 210 in the time period 220 (e.g., trigger 210N) has successfully validated. Conversely, the test evaluator 145 may identify or determine that the event scheduler 160 (or schedule 205) is not validated when it is determined that the message 195 of at least one trigger 210 in the set of triggers 210 has not been successfully communicated. In some embodiments, the test evaluator 145 may determine that the event scheduler 160 (or schedule 205) is not validated when it is determined that at least one trigger 210 in the set of triggers has not successfully validated.

[0076] Based on the results of the validation, the test evaluator 145 can create, generate, or otherwise produce at least one output 405. If the event scheduler 160 (or schedule 205) is determined to be validated, the test evaluator 145 can generate an output 405 that includes an indication that the event scheduler 160 (or schedule 205) is validated or that the event scheduler 160 (or schedule 205) is validated. In some embodiments, the test evaluator 145 can generate an output 405 that indicates that the set of triggers 210 are all validated. On the other hand, if the event scheduler 160 (or schedule 205) is determined to be not validated, the test evaluator 145 can generate an output 405 that includes an indication that the event scheduler 160 (or schedule 205) is not validated. In some embodiments, the test evaluator 145 can generate an output 405 that includes at least one identifier corresponding to at least one trigger 210 of the set of triggers that is not validated.

[0077] In some embodiments, the test evaluator 145 can use at least a portion of a transaction log (e.g., the sending log 305 or the receiving log 315) to generate the output 405. The output 405 can be generated to include an identifier of a transaction log corresponding to at least one of the sending log 305 or the receiving log 315. The identifier can uniquely reference the transaction log and can include, for example, a uniform resource locator (URL). A recipient can use the identifier to access the transaction log, which includes a set of messages and responses communicated between the session management service 110 and the user device 170. If the event scheduler 160 (or schedule 205) is determined to be validated, the test evaluator 145 can generate the output 405, which includes an indication that the event scheduler 160 (or schedule 205) has been successfully validated and the identifier of the transaction log. If the event scheduler 160 (or schedule 205) is determined to be not validated, the test evaluator 145 may generate an output 405 that includes an indication that the event scheduler 160 (or schedule 205) is not validated and an identifier of the transaction log. In some embodiments, the test evaluator 145 may generate an output 405 that includes at least one identifier corresponding to at least one trigger 210 in the set of triggers that is not validated and an identifier of the transaction log.

[0078] Once the output 405 is generated, the test evaluator 145 can emit, transmit, or otherwise provide the output 405. The output 405 can be provided for display on a display (or computing device) communicatively coupled to the test management service 105. A user of the test management service 105 can use the output 405 to view the results of the testing and validation of the event scheduler 160. If any triggers 210 have not been validated, the user can further inspect the contents of the output 405 to determine which triggers 210 contributed to the failure of the validation of the event scheduler 160. Based on the output 405, the user can adjust, change, or otherwise modify configurations (e.g., programming code or scripts) of the session management service 110, the user device 170, the application 175, or the like.

[0079] By using the global clock 150 to systematically set, disable, and advance the time on the server clock 150′ and the local clock 150″, the test management service 105 can significantly shorten and compress the time it would take to test and validate a set of triggers 210 and event scheduler 160 of an application 175. The required time can be reduced from weeks or months (e.g., as with approaches disclosed herein that do not rely on regular clock advances) to minutes or hours. Thus, the test management service 105 can improve the feasibility of the testing and validation process itself. The test management service 105 can further enhance the performance of the session management service 110 and the user device 170, increasing the number of functions that can be tested in a defined period of time. Because more functions can be tested and validated, the overall capabilities of, and communication between, the session management service 110 and the user device 170 can be significantly improved and optimized. This optimization can enable savings in computational resources (e.g., processing and memory) on both the session management service 110 and the user device 170, as well as reduced network bandwidth consumption. In the context of digital therapeutics, the test management service 105 can further ensure that the session management service 110 and the application 175 more efficiently address the user's condition.

[0080] 5A-C, a block diagram of a method 500 for simulating a test environment for an application over several time frames is shown. Method 500 may be implemented or performed using any of the components described herein, such as test management service 105, session management service 110, or user device 110, or any combination thereof. According to method 500, a computing system may identify a set of triggers for an event scheduler (502). The computing system may select a trigger from the set of triggers (504). The computing system may set a clock to the timestamp of the trigger (506). The computing system may cause a service to evaluate the trigger (508). The computing system may determine if the trigger is satisfied (510).

[0081] If the trigger is satisfied, the computing system may determine (530) whether the message has been communicated. If it is determined that the message has not been communicated, the computing system may determine (532) that the trigger has not been validated. The computing system may determine (534) that the event scheduler has not been validated. On the other hand, if it is determined that the message has been communicated, the computing system may determine (536) that the trigger has been validated. The computing system may determine (538) whether there are additional triggers. If there are no additional triggers, the computing system may generate an output (540). In contrast, if there are additional triggers, the computing system may identify (542) the next trigger and repeat the functions from step (506).

[0082] Conversely, if the trigger is not satisfied, the computing system may determine (560) whether the message has been propagated. If it is determined that the message has been propagated, the computing system may determine (562) that the trigger has not been validated. The computing system may determine (564) that the event scheduler has not been validated. On the other hand, if it is determined that the message has not been propagated, the computing system may determine (566) that the trigger has been validated. The computing system may determine (568) whether there are additional triggers. If there are no additional triggers, the computing system may generate an output (570). In contrast, if there are additional triggers, the computing system may identify (572) the next trigger and repeat the functions from step (506).

[0083] C. Network and Computing Environment Various operations described herein may be performed on a computer system. FIG. 6 illustrates a simplified block diagram of a representative server system 600, a client computer system 614, and a network 626 that may be used to implement certain embodiments of the present disclosure. In various embodiments, the server system 600 or a similar system may implement the services or servers described herein, or portions thereof. The client computer system 614 or a similar system may implement the clients described herein. The system 100 described herein may be similar to the server system 600. The server system 600 may have a modular design incorporating many modules 602 (e.g., blades in a blade server embodiment); although two modules 602 are shown, any number may be provided. Each module 602 may include one or more processing units 604 and local storage 606.

[0084] The one or more processing units 604 can include a single processor, which can have one or more cores, or multiple processors. In some embodiments, the one or more processing units 604 can include a general-purpose primary processor as well as one or more special-purpose coprocessors, such as a graphics processor, digital signal processor, etc. In some embodiments, some or all of the processing units 604 can be implemented using customized circuitry, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA). In some embodiments, such integrated circuits execute instructions stored on the circuitry itself. In other embodiments, the one or more processing units 604 can execute instructions stored on local storage 606. Any combination of any type of processor can be included in the one or more processing units 604.

[0085] The local storage device 606 may include volatile storage media (e.g., DRAM, SRAM, SDRAM, etc.) and / or non-volatile storage media (e.g., magnetic or optical disks, flash memory, etc.). The storage media incorporated in the local storage device 606 may be fixed, removable, or upgradeable, as desired. The local storage device 606 may be physically or logically divided into various sub-units, such as system memory, read-only memory (ROM), and permanent storage. The system memory may be a read-and-write memory device or a volatile read-and-write memory, such as dynamic random access memory. The system memory may store some or all of the instructions and data needed by the one or more processing units 604 during execution. The ROM may store static data and instructions needed by the one or more processing units 604. The permanent storage device may be a non-volatile read-and-write memory device that can store instructions and data even when the module 602 is powered down. As used herein, the term "storage medium" includes any medium capable of storing data indefinitely (albeit through overwriting, electrical disturbance, power loss, etc.), and does not include carrier waves or ephemeral electronic signals propagated over wireless or wired connections.

[0086] In some embodiments, local storage 606 may store one or more software programs executed by one or more processing units 604, such as an operating system and / or programs that implement the functionality of system 100 or any other system described herein or various server functions, such as any other one or more servers associated with system 100 or any other system described herein.

[0087] "Software" generally refers to sequences of instructions that, when executed by one or more processing units 604, cause server system 600 (or portions thereof) to perform various operations, and thus define one or more specific machine implementations that execute and carry out the operations of the software program. The instructions may be stored as firmware resident in read-only memory for execution by one or more processing units 604 and / or as program code stored on a non-volatile storage medium that can be loaded into volatile working memory. The software may be implemented as a single program or as a collection of separate programs or program modules that interact as desired. To perform the various operations described above, one or more processing units 606 may retrieve program instructions to execute and data to process from local storage 604 (or non-local storage, as described below).

[0088] In some server systems 600, multiple modules 602 may be interconnected via a bus or other interconnect 608 to form a local area network that supports communication between the modules 602 and other components of the server system 600. The interconnect 608 may be implemented using a variety of technologies, including server racks, hubs, routers, etc.

[0089] A wide area network (WAN) interface 610 may provide data communication capabilities between a local area network (e.g., via interconnect 608) and a network 626, such as the Internet. Other technologies may be used to communicatively couple the server system to the network 626, including wired technologies (e.g., Ethernet, IEEE 802.3 standard) and / or wireless technologies (e.g., Wi-Fi, IEEE 802.11 standard).

[0090] In some embodiments, local storage 606 is intended to provide working memory for one or more processing units 604, providing fast access to programs and / or data being processed while reducing traffic on interconnect 608. Storage for larger amounts of data can be provided on a local area network by one or more mass storage subsystems 608 connectable to interconnect 612. Mass storage subsystems 612 can be based on magnetic, optical, semiconductor, or other data storage media. Direct-attached storage, storage area networks, network-attached storage, etc. can be used. Any stored data or other collection of data described herein as generated, consumed, or maintained by a service or server can be stored in mass storage subsystem 612. In some embodiments, additional data storage resources are accessible (albeit potentially with increased latency) via WAN interface 610.

[0091] The server system 600 may operate in response to requests received via the WAN interface 610. For example, one of the modules 602 may implement management functions and assign individual tasks to the other modules 602 in response to received requests. Work allocation techniques may be used. Once a request is processed, results may be returned to the requestor via the WAN interface 610. Typically, such operations may be automated. Furthermore, in some embodiments, the WAN interface 610 may connect multiple server systems 600 together to provide a scalable system capable of managing high volumes of activity. Other techniques for managing server systems and server farms (collections of server systems cooperating with each other) may be used, including dynamic resource allocation and reallocation.

[0092] Server system 600 can interact with a variety of user-owned or user-operated devices over a wide area network, such as the Internet. An example of a user-operated device is shown in Figure 6 as client computing system 614. Client computing system 614 can be implemented as a consumer device, such as, for example, a smartphone, other mobile phone, tablet computer, wearable computing device (e.g., smart watch, eyeglasses), desktop computer, laptop computer, etc.

[0093] For example, client computing system 614 may communicate over WAN interface 610. Client computing system 614 may include computer components such as one or more processing units 616, storage devices 618, network interfaces 620, user input devices 622, and user output devices 624. Client computing system 614 may be computing devices implemented in a variety of form factors, such as desktop computers, laptop computers, tablet computers, smartphones, other mobile computing devices, wearable computing devices, etc.

[0094] The processing unit 616 and storage unit 618 may be similar to the one or more processing units 604 and local storage unit 606 described above. Suitable devices may be selected based on the demands placed on the client computing system 614. For example, the client computing system 614 may be implemented as a "thin" client having limited processing power or as a high-performance computing device. The client computing system 614 may comprise program code executable by the one or more processing units 616 to enable various interactions with the server system 600.

[0095] The network interface 620 may provide a connection to a network 626, such as a wide area network (e.g., the Internet), to which the WAN interface 610 of the server system 600 is also connected. In various embodiments, the network interface 620 may include a wired interface (e.g., Ethernet) and / or a wireless interface implementing various wireless data communication standards, such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, LTE, etc.).

[0096] User input device 622 may include any device (or devices) that allows a user to send signals to client computing system 614, which can be interpreted by client computing system 614 as indicating a particular user request or information. In various embodiments, user input device 622 may include any or all of a keyboard, touchpad, touchscreen, mouse or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, etc.

[0097] The user output device(s) 624 may include any device(s) that enable the client computing system 614 to provide information to a user. For example, the user output device(s) 624 may include a display-to-display shared image generated by or transmitted to the client computing system 614. The display may incorporate various image generation technologies, such as, for example, a liquid crystal display (LCD), a light emitting diode (LED) display including an organic light emitting diode (OLED), a projection system, a cathode ray tube (CRT), etc., along with supporting electronics (e.g., digital-to-analog or analog-to-digital converters, signal processors, etc.). Some embodiments may include devices such as a touchscreen that function as both an input and output device. In some embodiments, other user output device(s) 624 may be provided in addition to or instead of a display. Examples include indicator lights, speakers, tactile "display" devices, printers, etc.

[0098] Some embodiments include electronic components, such as a microprocessor, storage devices, and memory, that store computer program instructions on a computer-readable storage medium. Many of the features described herein can be implemented as processes specified as a set of program instructions encoded on a computer-readable storage medium. Execution of these program instructions by one or more processing units causes the one or more processing units to perform various operations indicated by the program instructions. Examples of program instructions or computer code include machine code produced by a compiler and files containing higher-level code that are executed by a computer, electronic component, or microprocessor using an interpreter. With appropriate programming, one or more processing devices 604 and 616 can provide various functions to the server system 600 and the client computing system 614, including any of the functions described herein as being performed by a server or client, or other functions.

[0099] It will be understood that the server system 600 and the client computing system 614 are illustrative and that variations and modifications are possible. Computer systems used in connection with embodiments of the present disclosure may have other functionality not specifically described herein. Additionally, while the server system 600 and the client computing system 614 are described with reference to certain blocks, it should be understood that these blocks are defined for convenience of explanation and are not intended to imply a particular physical arrangement of components. For example, different blocks may, but need not, be located in the same facility, in the same server rack, or on the same motherboard. Furthermore, the blocks need not correspond to physically separate components. The blocks may be configured to perform various operations, for example, by programming a processor or providing appropriate control circuitry, and various blocks may be reconfigurable or non-reconfigurable depending on how the initial configuration is obtained. Embodiments of the present disclosure may be realized in a variety of apparatuses, including electronic devices implemented using any combination of circuitry and software.

[0100] While the present disclosure has been described with reference to specific embodiments, those skilled in the art will recognize that numerous variations are possible. Embodiments of the present disclosure can be implemented using a variety of computer systems and communication technologies, including but not limited to the specific examples described herein. Embodiments of the present disclosure can be implemented using any combination of dedicated components and / or programmable processors and / or other programmable devices. The various processes described herein can be performed by any combination of the same or different processors. Where a component is described as being configured to perform a particular operation, such configuration can be achieved, for example, by designing electronic circuitry to perform the operation, by programming a programmable electronic circuit (such as a microprocessor) to perform the operation, or by any combination thereof. Furthermore, while the above-described embodiments may refer to particular hardware and software components, those skilled in the art will recognize that different combinations of hardware and / or software components can also be used, and that particular operations described as being implemented in hardware can also be implemented in software, or vice versa.

[0101] A computer program incorporating various features of the present disclosure may be encoded and stored on a variety of computer-readable storage media. Suitable media include magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, and other non-transitory media. A computer-readable medium encoded with program code may be packaged with a compatible electronic device, or the program code may be provided separately from the electronic device (e.g., via internet download or as a separately packaged computer-readable storage medium).

[0102] Thus, although the present disclosure has been described in terms of specific embodiments, it will be understood that the present disclosure is intended to cover all modifications and equivalents that fall within the scope of the following claims.

Claims

1. 1. A method of simulating a testing environment for a digital therapeutic application for addressing a disease state over a time frame, comprising: selecting, by one or more processors, a trigger from a plurality of triggers of an event scheduler to address the condition, each of the plurality of triggers identifying a respective timestamp from a plurality of timestamps, at which the service sends a corresponding message to the user device to address the condition; setting, by the one or more processors, a clock to a timestamp of the one trigger, the clock being accessible to the service and the user device; causing, by the one or more processors, the service to send a message to the user device to address the condition in response to the timestamp of the clock matching the timestamp of the one trigger; identifying, by the one or more processors, that the message is being communicated between the service and the user device in accordance with the one trigger; determining, by the one or more processors, in response to determining that the message has been communicated, that the one trigger of the event scheduler is validated.

2. selecting, by the one or more processors, a second trigger from the plurality of triggers in response to the one trigger being validated, the second trigger identifying a second timestamp corresponding to an end of a period spanned by the plurality of timestamps; setting, by the one or more processors, the clock to the second timestamp of the second trigger; causing, by the one or more processors, the service to send a second message to the user device to address the condition in response to the second timestamp of the clock matching the second timestamp of the one trigger; determining, by the one or more processors, that the service sent the second message to the user device in accordance with the second trigger; determining, by the one or more processors, in response to determining that the service has received the second response, that the event scheduler is validated prior to the end of the period.

3. 3. The method of claim 2, wherein determining that the event scheduler is validated further comprises determining that the event scheduler is validated in response to determining that the service has successfully sent a respective message for each of the plurality of triggers to the user device.

4. setting, by the one or more processors, the clock to a second timestamp of a second trigger of the plurality of triggers; causing, by the one or more processors, the service to send a second message to the user device to address the condition in response to the second timestamp of the clock matching the second timestamp of the second trigger; and determining, by the one or more processors, in response to determining that the second message has not been communicated, that the second trigger of the event scheduler has not been validated.

5. 5. The method of claim 4, further comprising, in response to determining that the second message was not delivered on a first attempt, attempting, by the one or more processors, to again cause the service to send the second message on a second attempt.

6. In response to determining, by the one or more processors, that the second trigger is not validated, obtaining a transaction log identifying one or more of a plurality of messages communicated between the service and the user device; and generating, by the one or more processors, an output comprising at least one of an indication of a failure to validate the second trigger and an identification of the transaction log.

7. 2. The method of claim 1, further comprising: causing the one or more processors to: (i) cause the service to set a service clock from a first timestamp to the timestamp of the clock; and (ii) cause the user device to set a device clock from a second timestamp to the timestamp of the clock.

8. The method of claim 1 , wherein the event scheduler includes a machine learning model that generates at least a portion of the plurality of triggers for the schedule.

9. at least one of the plurality of triggers further identifies a respective criterion that must be satisfied for the service to send the corresponding message to address the condition; 2. The method of claim 1, wherein causing the service to send the message further comprises causing the service to send the message in response to data associated with the user device satisfying the criteria specified by the trigger.

10. 2. The method of claim 1, wherein setting the clock further comprises correcting the clock from an initial timestamp corresponding to a current time to a timestamp corresponding to a subsequent time specified by the one trigger prior to the occurrence of the subsequent time.

11. 2. The method of claim 1, wherein determining that the message has been conveyed further comprises: (i) determining that the service sent the message using a transmission log associated with the service; (ii) determining that the user device received the message from the service using a reception log associated with the user device; and (iii) determining that the service received a response from the user device within a certain time frame for sending the message.

12. 14. The system of claim 13, wherein the message comprises at least one of a short message service (SMS) message, a multimedia messaging service (MMS), or an in-app message, and the message is presented at least partially concurrently with the user taking the medication.

13. 1. A system for simulating a testing environment for a digital therapeutic application for addressing a disease state over a time frame, comprising: One or more processors coupled to a memory, the processors: selecting a trigger from a plurality of triggers of an event scheduler to address the condition, each of the plurality of triggers identifying a respective timestamp from a plurality of timestamps, and at the respective timestamp, a service sending a corresponding message to the user device to address the condition; setting a clock to a timestamp of the one trigger, the clock being accessible to the service and the user device; in response to the timestamp of the clock matching the timestamp of the one trigger, causing the service to send a message to the user device to address the condition; Identifying that the message is being communicated between the service and the user device in accordance with the one trigger; In response to determining that the message has been communicated, the system determines that the one trigger of the event scheduler is validated.

14. The one or more processors: selecting a second trigger from the plurality of triggers in response to the one trigger being validated, the second trigger identifying a second timestamp corresponding to an end of a period spanned by the plurality of timestamps; setting the clock to the second timestamp of the second trigger; in response to the second timestamp of the clock matching the second timestamp of the one trigger, causing the service to send a second message to the user device to address the condition; Identifying that the service sent the second message to the user device in accordance with the second trigger; 14. The system of claim 13, further configured to, in response to determining that the service has received the second response, determine that the event scheduler is validated prior to the end of the period.

15. 15. The system of claim 14, wherein the one or more processors are further configured to determine that the event scheduler is validated in response to determining that the service has successfully sent a respective message for each trigger of the plurality of triggers to the user device.

16. The one or more processors: setting the clock to a second timestamp of a second trigger of the plurality of triggers; in response to the second timestamp of the clock matching the second timestamp of the one trigger, causing the service to send a second message to the user device to address the condition; 14. The system of claim 13, further configured to, in response to determining that the second message has not been communicated, determine that the second trigger of the event scheduler has not been validated.

17. 17. The system of claim 16, wherein the one or more processors are further configured, in response to determining that the second message was not delivered in a first attempt, to again attempt to cause the service to send the second message in a second attempt.

18. The one or more processors: In response to determining that the second trigger is not validated, obtaining a transaction log identifying one or more of a plurality of messages communicated between the service and the user device; 14. The system of claim 13, further configured to generate an output comprising at least one of an indication of a failure to validate the second trigger and an identification of the transaction log.

19. 14. The system of claim 13, wherein the one or more processors are further configured to: (i) cause the service to set a service clock from a first timestamp to the timestamp of the clock; and (ii) cause the user device to set a device clock from a second timestamp to the timestamp of the clock.

20. The system of claim 13 , wherein the event scheduler includes a machine learning model that generates at least a portion of the plurality of triggers for the schedule.

21. at least one of the plurality of triggers further identifies a respective criterion that must be satisfied for the service to send the corresponding message to address the condition; 14. The system of claim 13, wherein the one or more processors are further configured to cause the service to send the message, which further comprises causing the service to send the message in response to data associated with the user device satisfying the criteria specified by the trigger.

22. 14. The system of claim 13, wherein the one or more processors are further configured to correct the clock from an initial timestamp corresponding to a current time to a timestamp corresponding to a subsequent time identified by the one trigger prior to the occurrence of the subsequent time.

23. 14. The system of claim 13, wherein the one or more processors are further configured to determine that the message has been communicated by at least one of: (i) determining that the service sent the message using a transmission log associated with the service, (ii) determining that the user device received the message from the service using a reception log associated with the user device, or (iii) determining that the service received a response from the user device within a certain time frame for sending the message.

24. 14. The system of claim 13, wherein the message comprises at least one of a short message service (SMS) message, a multimedia messaging service (MMS), or an in-app message, and the message is presented at least partially concurrently with the user taking the medication.

Images